Overview

In a small mail system you have one POP3 server that has stored all mailboxes locally on an attached disk. All clients directly connect to this server to access the mail.

This modell works as long as one server can cope with the load. If you have too many mailboxes or too many accesses per mailbox, you want to split the mailboxes over several servers. But now the client has to know to which server to connect.

Telling every client about the new server is probably not feasable every time you add new servers. One possible solution is to put all mailboxes on a NFS server and allow access to all mailboxes from all servers. But this will create a new bottleneck: the NFS server. But there is another solution: Put a proxy server between the client and the server.

The proxy server will accept the connection from the client, get the username and password from the client, authenticate the user and find out on which backend server this users mailbox is located.

For authentication and the mapping of username to server, the proxy needs some kind of database. This database can either be on the proxy server itself or somewhere else on the net.

After the client is authenticated, the proxy will connect to the proper backend server and blindly forward data back and forth between the client and server until one of them closes the connection.

At that point the proxy might develop to be the bottleneck, but there is no reason not to have several proxies. Through DNS round robin or a load balancing server between the client and the proxy, the load will be distributed over all proxies.

The POPular system implements this architecture including the POP proxy and storage server software.

The sequence of events in a POP session

This section explains the several steps in a POP3 session as they are implemented in the POPular system.

When a new connection is opened from a client, the POP3 proxy ("pproxy") forks and sends a welcome message to the client. The client sends the username and password. Username, password, the client IP and some other information is then checked against one or more databases.

To be as flexible as posssible, these database checks are performed in so called POPular database modules (PDM) implemented as shared libraries loaded at runtime into the pproxy process. Several databases can be checked after another until one finds and authenticates the user.

The proxy supports IP based virtual servers. It can listen to more than one IP address and port. Depending on which IP address and port a new connection is coming in on, the proxy can use different namespaces for authentication.

From the database modules, the proxy will get back the name of the backend server, where the mailbox of this user is stored, and the protocol, which is to be used to connect to this backend. Depending on the protocol either the name of the mailbox or a username and password are also returned.

If the backend server is a POPular server, the proxy now sends a mail check requests to the backend server. This is a small UDP datagram only containing the name of the mailbox. The pcheckd process on the server will receive this mailbox name and check whether there are any mails in this mailbox It will tell the proxy by sending a UDP datagram back. Statistics have shown that about 80% of all accesses to a mailbox will find an empty mailbox, so it makes sense to check for this and optimize the common case. See below for what happens if the mailbox is empty.

If there are any mails in the mailbox the proxy will now open a TCP connection to the storage server. Depending on the procotol to be used, two different things can happen:

If the protocol is POP3 indicating a normal POP3 server is used as backend, the username and password that the proxy got from the PDM modules is send to the server. (This username and password might well be the same as the one the client sent, but this is not required.) The backend server will then authenticate the user again and answer with a positive or negative answer, which is relayed back to the client.

Because the user is already authenticated it doesn't really make sense to send the username and password and check them again on the backend server. That is why the POPular package contains a special server ("pserv") that will not check the user again, but just use a mailbox name supplied by the proxy.

If the protocol is XPOP indicating that a POPular backend server is used, the proxy first sends the name of the mailbox in a line by itself, then an ID which is used for all logging messages to easily find corresponding entries in the proxy and server log and then a line with flags. The POP server will answer with "+OK", which is relayed back to the client.

From now on the client and server will exchange POP3 requests and responses. The proxy ferries the data back and forth without interfering until one or the other closes the connection.

If the mailbox is empty the proxy server will handle this POP connection alone, i.e. it will answer all requests with "no mail" or whatever is appropriate for an empty mailbox. This reduces the load on the storage servers considerably.

Packages

There might be precompiled POPular packages available for your system. See the POPular web page.

If you are installing POPular with a package manager like rpm or dpkg you can probably ignore most information in the rest of this chapter.

Compilation

Install POPular by unpacking the distribution, running ./configure, make and make install. This will compile all programs and install them in /usr/local/bin and /usr/local/sbin. Some POPular database modules will be compiled and installed into /usr/local/lib/popular. Also, all man pages are made and installed into /usr/local/man. Call configure with the option --prefix=PREFIX if you want to install somewhere else.

The utility scripts, configuration files, etc. are not automatically installed.

The main documentation is created from a SGML/XML source file using the DocBook standard. As there are quite a lot of strange programs and files needed to get the DocBook stuff running, the documentation is provided in the distribution in the compiled HTML, Postscript and RTF version. Look in the doc directory for the files named popular.html, popular-htmldoc.tar.gz, popular.ps and popular.rtf. These files are not installed automatically. If you want to make the documentation yourself, use the --enable-docbook option to configure.

Along with the normal GNU configure options, the following options can be used:

POPular is being developed on Linux systems and currently only tested on Linux. It should mostly work on any POSIX system, but there are some things (like shared mmaped files and file descriptor passing), which might not work on other systems. Currently it mostly compiles on Solaris, but is not tested very much. If you can help porting POPular to other systems, you are most certainly welcome.

Common setup for proxy and storage server

First you have to decide which user and group to use for the POPular programs. If you are (for testing purposes) running the proxy and the backend server on the same machine, both have to use the same user and group, otherwise you can use different users and groups though it is not recommended. Choose a user and group that are not used for anything else on the system. Don't use 'root'. All mailbox directories and mail message files must belong to the user and group you choose. This manual will use the user and group name 'pop'.

The following script can be used to set up the default directories used by POPular. If you want to use any other directories, change the script and call the programs with the proper options.

#!/bin/sh

POPULAR_USER=pop
POPULAR_GROUP=pop

mkdir -p             /etc/popular
chmod 0755           /etc/popular

mkdir -p             /etc/popular/capa
chmod 0755           /etc/popular/capa

mkdir -p             /var/log/popular
chown $POPULAR_USER  /var/log/popular
chgrp $POPULAR_GROUP /var/log/popular
chmod 0755           /var/log/popular

mkdir -p             /var/run/popular
chown $POPULAR_USER  /var/run/popular
chgrp $POPULAR_GROUP /var/run/popular
chmod 0755           /var/run/popular
      

Setting up the proxy server

On the proxy server two program (pproxy and ringd) will be running.

You should have a file named pproxy.rc in the /etc/popular directory. Sample files can be found in the conf directory of the POPular distribution. Make sure pproxy.rc is executable.

First start ringd as user 'root'. ringd will run in the background and bind TCP ports for pproxy because pproxy runs as a normal user.

Next start pproxy as the user and group you chose. You should see some log messages appearing in /var/log/popular/pproxy.

To configure the running pproxy use the pcontrol program. Always start pcontrol as the user you have chosen for the pproxy. If everything is set up properly you can just call /etc/popular/pproxy.rc to do the first configuration.

Setting up the storage server

You should have a file named pserv.rc in the /etc/popular directory and it should be executable for the user you have chosen to run the server as. A sample file can be found in the conf directory of the POPular distribution.

Decide where to put the mailboxes, the sample config file uses the directory /pop for this. All mailbox directories must be readable and writeable by the user you chose for running pserv.

Start the server as the user and group you chose: pserv. You should see some log messages appearing in /var/log/popular/pserv.

To configure the running pserv use the pcontrol program. Always start pcontrol as the user you have chosen for the pserv. If everything is set up properly you can just call /etc/popular/pserv.rc to do the first configuration.

If you want to use the pcheckd daemon, start it as the same user you used for pserv. You need at least to use the option -m to tell pcheckd where the mailbox directory is (/pop by default).

Command line options

Runtime configuration

The configuration of the POPular proxy is done by sending commands to the running server via a UNIX domain socket. This socket is in the run directory (default: /var/run/popular) and named pproxy.PID.ctrl. PID, in this case, is the process id of the parent process. For easy access, there is a link from pproxy.ctrl to the real socket file.

The pcontrol utility program is provided to send commands to this socket. See the man page for details.

Normally, after pproxy has started, the file /etc/popular/pproxy.rc should be used to set the start configuration of the server. There is an example of this file in the conf directory of the distribution. The supplied RedHat and Debian init scripts will use this file to do the initial configuration.

Signals

The pproxy parent understands the following signals:

The child processes have no special signal handling, i.e. they will die on receiving SIGTERM, SIGQUIT, SIGINT, or SIGHUP.

Starting pserv

pserv is always started as the 'pop' user (or whatever user you decided should be used). Normally there are no command line options required.

The --logfile option can be used to name the initial log file (default is /var/log/popular/pserv). The name of the logfile can later be changed with the set logfile directive.

The --rundir option can be used to change the run directory, where some files needed for the operation of pserv are stored. This can't be changed while the program is running.

All the rest of the configuration is done while the server is running. A server that has just started doesn't listen to any ports and has only default values set for its configuration variables.

Runtime configuration

The configuration of the POPular server is done by sending commands to the running server via a UNIX domain socket. This socket is in the run directory (default: /var/run/popular) and named pserv.PID.ctrl. PID, in this case, is the process id of the parent process. For easy access, there is a link from pserv.ctrl to the real socket file.

The pcontrol utility program is provided to send commands to this socket. See the man page for details.

Normally, after pserv has started, the file /etc/popular/pserv.rc should be used to set the start configuration of the server. There is an example of this file in the conf directory of the distribution. The supplied RedHat and Debian init scripts will use this file to do the initial configuration.

Signals

The pserv parent understands the following signals:

The child processes have no special signal handling, i.e. they will die on receiving SIGTERM, SIGQUIT, SIGINT, or SIGHUP.

Command line options

Signals

pcheckd understands the following signals:

Introduction

POPular uses a very flexible database lookup and authentication method based on shared libraries loaded at runtime. These libraries are called "POPular Database Modules (PDM)". Several modules are currently provided with POPular. More modules can be written easily as the API is quite simple.

After a client program sends the username and password all loaded modules will be called in order and given the clients IP number, the used protocol, the used namespace, and the username and password. Each module can check, whether it can identify and authenticate the user. If a module authenticates the user, no more modules are called. If the module can identify the user, but doesn't allow access for any reason (like a wrong password), all modules are again called and asked whether they can authenticate the user.

Database modules can be loaded into and unloaded from pproxy at runtime. See the description of the pproxy configuration for details.

For testing database modules alone and together with other modules, the ptestpdm program is provided. For details see the documentation for this command.

Any module (libpdm_any.so)

This is a test module which will accept any user. If you are new to all this, this is a good place to start.

Master password module (libpdm_master.so)

Using a master authentication file for access to all mailboxes with special master password. The name of the file is given as an argument when initializing the module (default: /etc/popular/masterauth.conf). This is a plain text file. Each line contains an IP number in dotted quad notation, a colon, a namespace name, another colon and a crypted password. If the IP number matches the IP number of the POP3 client, the namespace name matches the accessed namespace, and the password matches the given POP3 password, access is granted. Empty lines and lines beginning with a '#' symbol are ignored.

This feature is intended to be used by the postmaster or support staff. Connections authenticated with the master password are flagged. New mail read is not marked as read, so that the customers MUA will not get confused. Mail that is deleted, WILL REALLY BE DELETED.

Berkeley DB Version 2 module (libpdm_db2.so)

Using Berkeley DB Version 2 hash files. The hash file used is named after the namespace and is looked for in the directory that was given as argument when initializing the module. (default: /etc/popular/db/). An example: /etc/popular/db/test.db is the file for the namespace 'test'.

The lookup key in the DB file is the POP3 username. There is no trailing zero after the key or data. The data should be a colon separated list of the crypted password, the ID of the backend server and the mailbox name. A Perl script db-create.pl is supplied (along with example files) in the auth subdirectory to create this DB file from a text file containing lines with a colon separated list of username, crypted password, backend server name and mailbox. The 'source' file can contain empty lines and comment lines beginning with a '#' symbol.

Berkeley DB Version 3 module (libpdm_db3.so)

Using Berkeley DB Version 3. Apart from that its the same as db2.

CDB (constant data base) module (libpdm_cdb.so)

Using Dan Bernstein's CDB.

Logfile format

The log file format is carefully designed to be easily greppable in shell or Perl scripts. The date and time stamps use the compact ISO 8601 format, which is concise, unambiguous and easily sortable. See https://www.cl.cam.ac.uk/~mgk25/iso-time.html for details.

Each log entry is in one line and contains the following fields separated by a space character:

Debug messages use a slightly different format: The message number is always '0000' and instead of the message name a string containing the following parts separated by colons (':') are printed: The debug type, the current C function, the C source file and the line number in the source file.

Log levels

The following log levels are used. They are chosen based on the idea that it should be easy for the system administrator to find out, whether any action on his part is needed.

Reopening log files

A SIGHUP can be sent to the pproxy, pserv, pcheckd, and psmtpd commands to reopen the log file. If the new log files can't be opened, the all programs except psmtpd will keep logging to the old file. psmtpd will not log anything anymore! Note that child processes don't know about this log file change and will keep logging to the old file. This is considered a feature, because all entries from one child will be in the same log file. Children are normally not running for a long time, so this is no problem for log file rotation. Just make sure that after rotating the log file it is not immediately gzipped or copied away and deleted.

Max session/load warnings

The POPular servers will log warnings when certain load and session limits are reached.

The session and load limit can be configured through the maxsession and maxlocalload config variables. Both are integers.

When 90% of the session or load limit is reached, POPular will switch from state 'OK' to state 'WARN', when 100% of the session or load limit is reached, POPular will switch to state 'MAX'. When the session count or load falls below 90% in state 'MAX', it will go to state 'WARN'. When the session count or load falls below 80% in state 'WARN', it will go to state 'OK'.

Each change of state is logged. For session state changes a message number 0x0170 ('session_state') is logged, for load state changes a message number 0x0172 ('load_state') is logged. The message contains the name of the new state.

As long as there are session slots available, new connections will be accepted regardless of the session state.

If the load state is 'MAX', no new connections will be accepted until the state drops back to 'WARN'.

Mail delivery

The pdeliver program described elsewhere in this document is used for local delivery of mails into mailboxes. For delivering mails from remote hosts, there is another program, which can be used. scripts/psmtpd is a Perl script, which implements a minimal SMTP server. Basically the only thing it can do is deliver a mail to a single mailbox.

There are some variables at the beginning of the file, which need to be defined properly.

This program is not intended to be used as general purpose MTA. It can only be used behind a carefully configured other MTA, that feeds it properly. Never, ever, permit the general Internet access to this server.

Cleanup

There is a script pclean provided, that can be used for all sorts of cleanup chores. At the moment this script is in beta state and not all that well tested.

In the meantime the following three scripts in the scripts directory can be used for nightly cleanup chores:

SSL/TLS support

Starting in version 1.5.0 POPular supports SSL and TLS if compiled with the OpenSSL library. You need at least version 0.9.6b. SSL/TLS is only available if POPular was configured with the --enable-ssl.

Each virtual server can be configured to either use unencrypted connections or to use SSL/TLS connections or use the RFC2595 style STARTTLS command. Use the prot and starttls options of the vserv command to set this as follows:

1. For usage without SSL/TLS set prot to "pop3" and starttls to "off".

2. For direct SSL/TLS usage without STARTTLS set prot to "pop3s" and starttls to "off".

3. For usage of an optionally encrypted connection set prot to "pop3" and starttls to "optional". The connection will start out unencrypted and can be switched over to encrypted with the STARTTLS command before authentication.

4. For forced use of the STARTTLS command set prot to "pop3" and starttls to "force". The client will have to send a STARTTLS command before he is allowed to do anything else.

Certificate files must be stored in the directory named by the tlsdir variable. The files must have the name of the virtual server they are used for plus a ".pem" extension. Each file contains the RSA private key and a certificate for this virtual server.

See the prng for instructions how to seed the pseudo random number generator.

SSLv2 is considered insecure and disabled by default. Use the allowsslv2 variable to change this behaviour.

The connection between pproxy and pserv is always unencrypted. Use a secure tunnel if they are not in the same LAN.

Usage of the user 'root'

Usage of the user 'root' is kept to a minimum. The only program that needs to run as 'root' is the ringd server that is used by pproxy to bind sockets to low TCP ports. Everything else runs with the privileges of a normal user.

Input checks

All input from network sockets is checked for ASCII NUL ('\0') characters and the connection is immediately and silently dropped if a NUL character is found. This event is logged as message 0x0035 ('null_byte_in_input'). A NUL byte can never be part of a valid POP3 dialogue.

pserv and pcheckd check the mailbox names coming from pproxy. Only the following characters are allowed in mailbox names: a-Z, A-Z, 0-9, '.' (dot), '_' (underscore), '-' (hyphen), '+' (plus), '/' (forward slash), '=' (equal sign), and '%' (percent). Two adjacent dots ("..") are not allowed.

Using chroot

I see no reason why POPular can't be run in a chroot environment, although I haven't tried it. In the chroot environment you need the binaries of the servers and any shared libraries they use, the log directory (/var/log/popular), and the run directory (/var/run/popular). Depending on your configuration, you might need some config files from /etc/popular. For the proxy all files needed for authentication have to be included and for the storage server all mailbox directories. If you have the mailboxes on several disks, you have to mount them all inside the chroot environment.

One feature of POPular is going to make difficulties: The server reads /proc/loadavg to determine the load and react accordingly. It is probably not a good idea to mount /proc in the chroot environment, so you either have to live without the feature or find some way around the limitation of not being able to read /proc/loadavg.

That said, I don't really see much reason for going through all the hassle of setting up and maintaining the chroot environment. All the important data, that you want to protect either has to be in the chroot environment anyway (like the mailboxes) or at least has to be accessible from it through the network (like authentication data). Of course, it will be harder for an attacker, but it will be a bit harder for the sysadmin, too. Decide for yourself, whether you want that extra bit of security.

If somebody is using POPular in a chroot environment, I like to hear from you. Especially if I need to put some changes into POPular to make it easier to use in a chroot environment.

The extended Maildir format

The POPular POP server uses a slightly modified Maildir format for storing mailboxes. The format is for most uses compatible to the original mailbox format, allowing standard software components to be used.

In contrast to the original Maildir format the line ending used is CRLF and not only LF. Both SMTP and POP3 use CRLF as a line ending and there is no sense converting to LF endings when saving the mail and back to CRLF when reading the mail. The added advantage is that the size of the file is exactly the file size that is reported in POP3 or with the ESMTP SIZE extension in SMTP.

With the Maildir format each mail in a mailbox is saved in its own file instead of all mails together in one file. A mailbox is a directory with the three subdirectories new, cur and tmp. New, unread mails are in the new directory. Read mails are in the cur directory. The tmp is used for delivery. A mail is saved in this directory with a unique name. Only after the mail has been written completely to disk it is moved to the new directory. With this scheme no locking or copying of mails or mailboxes is needed when delivering, reading or deleting mails.

The filenames used for the mails must be unique. Somewhat differently than in the original description of the Maildir format the following naming scheme is used: A filename has three components, the UNIX system time (in seconds after 1970/1/1), the process id of the process delivering the mail and the size of the mail. The components are separated by a dot (.) and an underscore (_), respectively. An example for a file name would be 945113196.14341_1067. The size of the mail is saved in the filename, so that we don't have to stat the file in order to find out about its size.

With the original Maildir format, the hostname of the delivering system is written into the filename. This is only needed if NFS is used for mailbox access or mail files are copied between hosts to the same mailbox. Because we don't do any of this, we don't put the hostname in there.

A Status-Header is never saved inside a mail message file, because it would mean that the file has to be rewritten after it is RETRieved, but not DELEted. Instead, pserv will add a Status-Header on the fly while sending the mail out if the statusheader configuration variable is set and the mail file is in the cur directory.

POPular can cope with several pathologic cases of mail file contents like totally empty mails, mails without headers etc. Empty mails will not be accessible, an error is logged. Other strange mails are accessible, but the client might not understand it. Mails that don't end in CRLF have a CRLF appended, so that the end-of-file-marker 'CRLF.CRLF' is recognized. The appended CRLF don't count towards the mail size.

The XPOP protocol

The POPular POP proxy and POP servers communicate through the XPOP protocol, which is a slightly modified POP3 protocol (defined in RFC 1939 [RFC1939]). The modifications are needed for telling the POP server the mailbox to use and some more information. This extension is used only by the POPular proxy and server. If you use a different server, use the POP3 protocol.

The XPOP protocol is really simple. As soon as the connection between the proxy and the server is established the proxy sends three lines of text, each ending with a CRLF combination. The content of the lines is, in this order:

1. The mailbox to use. This is a directory name relative to the POP spool defined in the configuration of the POP server. It can include any number of subdirectories. It does not start with a /. No ".." is allowed in the name.

2. An ID string used for this connection. This ID is generated by the POP proxy and used in all logging messages on the proxy and server to be able to easily find corresponding entries.

3. A list of flags. Flags are denoted by ASCII chars. In the absence of the char, the flag is 'false', otherwise 'true'. Currently there is only one flag 'M' defined.

After this three lines are sent the POP server answers with '+OK' or '-ERR' according to the POP3 protocol. This message is passed through to the client. From then on everything is just like in the POP3 protocol but starting in the transaction phase, not in the authorization phase.

When configuring a backend server the protocol CXPOP can be used. This just means that the XPOP protocol should be used after checking for available mail with the MAILCHECK protocol, which is described in the next chapter.

The MAILCHECK protocol

On every storage server a pcheckd program is running. It implements the server side of the MAILCHECK protocol. The client side is implemented in the pproxy program. There is a test program called pcheck, which fully implements the protocol. It can be called from the command line.

The MAILCHECK protocol uses a single UDP request packet and a single UDP packet with the answer. The request packet contains with an asterisk ('*') a single uppercase letter denoting the type of request, extra parameters and an optional carriage return (CR) and an optional line feed (LF) character.

Currently there are two types of requests defined:

Note: For compatibility with earlier versions of this programm a slightly different syntax is allowed. If the first character is not an asterisk ('*'), the request is assumed to be a mailbox check and the only content of the packet is the mailbox. Support for this format will be discontinued in a future version.

The answer is a single UDP packet formatted as follows: If an error has occured on the server side it will return '-ERR ', an error message and a line feed character. If there was no error the server will return '+OK ' and an additional message as follows:

Mailbox locking

RFC 1939 describing the POP protocol askes for the mailbox to be locked after authentication of the user and before any access to the mailbox is granted. The mailbox stays locked until after the any changes are done to the mailbox after the client sends the QUIT command.

If mailboxes are saved in any of the formats where all the mails are saved in one big file, locking this file is actually necessary for insuring mailbox integrity. However, if mails are saved in one file per message (maildir format as POPular uses it), this is not strictly necessary.

If no locking is used the worst thing that can happen is that a user connects to a mailbox twice (or more times) and sees a message in the message list and when he tries to retrieve the message he will get an error message. While this is obviously not the best solution it isn't that bad.

On the other hand there is no reason why anyone will open several POP connections to the same mailbox. The only common case where this happens is when a POP connection breaks down on the client side, but the server hasn't noticed that yet. This can happen either because the client program or system crashes or the network connection (often going through a modem) fails.

In this situation not locking the mailbox will actually help. The first connection is inactive anyways and having a lock means that the server has to wait for a timeout before it unlocks the mailbox. Only after a timeout the mailbox is accessable again. Without locking the mailbox will be accessible immediately with no ill effects.

To make matters worse, indicating a 'mailbox locked' error to the user will often confuse the user and will cause him to call the providers helpline reporting a problem that has probably solved itself by the time the helpdesk knows about it.

For this reason there is no mailbox locking in pserv.

TCP keepalive option

On all TCP sockets used by pproxy and pserv the SO_KEEPALIVE socket option is set. The default timeout is 2 hours, it can be set in most UNIX systems through a kernel parameter and will then affect all TCP connections with the option set.

On Linux systems the timeout can be set by writing the time in seconds to the file /proc/sys/net/ipv4/tcp_keepalive_time.

POP connections are shortlived, if there is no data flowing, it doesn't make sense to keep the connection open. A short keepalive timeout will not produce much more traffic. On the other hand POP connections at large ISPs are often hanging, because clients behind dialups sometimes shut down modem connection without properly closing all TCP connections (or the modem connection might have failed). A short keepalive timeout will help detect these cases and free resources on the proxy and server.

If there are no other services running on the same machines as the POPular services, that might be negatively affected, it is recommended to set the keepalive timeout to a value somewhere between 10 and 30 minutes.

Note that connections where data is still flowing (even if flowing slowly) are not affected by this timer, as the timer is reset on each packet that is sent or received on this connection.

POP3 extensions and capabilities

RFC 2449 describes a POP3 extension mechanism. A client can use the CAPA command to ask the server about any optional or enhanced capabilities. In the context of POPular there is one special problem with capabilities: The CAPA command can be send before or after authentication. But that means that the proxy and the server both have to be able to send the right answer back. To not confuse the client the answer in both cases must be the same or similar (for details see RFC 2449). Because the POPular proxy can be used with any server and it doesn't know about its capabilities it can't send a proper answer to a CAPA request on its own.

To solve this problem the proxy and the server can be configured to answer anything you like to a CAPA query. For different virtual servers, different CAPA listings can be saved. Note that, because a CAPA request can be sent before authentification, it is not possible to bind a list of capabilities to a backend server.

Setting up a capabilities list happens in two steps. First you edit a file in /etc/popular/capa (the directory can be changed through pcontrol with set capadir) and put all the capabilities you want to offer in that file. You can give the file any name you want. This name will later be used as the ID of this capability list in pcontrol.

The format of the capabilities file is easy. Just list the capabilities one per line, exactly as they are supposed to appear as answer to the CAPA command. Have a look at RFC 2449 for the details. If you get this wrong, the answer to the CAPA command might be not conforming to RFC 2449! Use a normal LF as line ending, POPular will automatically translate that to a proper CRLF line ending for the POP protocol. Note that there is a compiled in upper limit (MAXLEN_CAPA) on the length of all the capabilities joined together in one list.

After you have set up the file (or files) you can load the capabilities list into pproxy with the pcontrol utility. With capa load FILENAME the file is loaded. capa show NAME will show you the loaded capability list. Note that for printing purposes the list of capabilities will appear on one line, separated by commas. Capability lists can be reloaded at any time with the capa load command and deleted with capa del NAME. With capa list you can get a list of all currently loaded capability lists.

There are three special capability lists build into POPular: ERROR, NONE and DEFAULT. ERROR means to answer with an error to any CAPA command sent by the client, emulating an old server without CAPA support. NONE will send an empty capabilities list and DEFAULT will send a small list of capabilities that are supported by both pproxy and pserv. That means that if you use only POPular programs and no external servers you can use DEFAULT.

The last step is assigning a capability list to each virtual server. This can be done with the vserv conf VSERV capa CAPABILITY-ID command. You can set any of the three builtin capability lists or any of the ones you loaded.

The following capabilities are always supported by POPular: TOP, UIDL, USER and PIPELINING. SASL, RESP-CODES, and LOGIN-DELAY are not supported. The EXPIRE capability is outside the scope of the POPular software, but you might want to use it. The default capability list does not include the IMPLEMENTATION capability. If you want to use it, please use 'IMPLEMENTATION POPular-<version>'.

High availability issues

If you want to build a real high available mail system with the POPular server suite, some things have to be considered. The POPular proxy and server in itself are not enough to get a highly available system. This documenation can't go into all details of building a highly available system, but some issues to get you startet will be discussed.

Mailbox directory layout

The POPular software doesn't force you to use any particular layout for the mailbox directories. The simplest layout would use a /var/spool/mail (or similar) directory with subdirectories for all the mailboxes. This layout has the severe drawback that it doesn't scale well. A directory with thousands of files is slow to access, especially on some filesystems like the Linux Ext2fs. Instead a hierarchical structure is recommended. Depending on the anticipated maximal number of mailboxes in the system the hierarchy has to have more or less levels.

There are several possibilities how to distribute the mailboxes into subdirectories. The obvious way is to use the mailbox name (or part of it) as a name for the subdirectory. This works well if the mailbox names are sufficiently "random", so that the mailboxes are distributed evenly. In most real life situations this is not the case. And even if the mailboxes in your case have a known structure and nice random names this might change over time.

An easy technique is to calculate a hash of the mailbox name and use this to name the subdirectories. The MD5 hash algorithm has proven to be very good for this. Whatever mailbox names you throw at it, the hash is very evenly distributed. And there are lots of implementations for every language available. Using one or two hex numbers from the MD5 hash for every directory level will result in directories with not more than 16 or 256 entries, respectively.

Performance tuning

It is probably a good idea to use the 'noatime' mount option on the mailbox storage disks.

Using virtual servers