Open source and the postmaster

by Elena Blanco on 1 June 2005 , last updated

Archived This page has been archived. Its content will not be updated. Further details of our archive policy.

Introduction

The use of electronic mail (email) has become ubiquitous. With the rise in its popularity, the job of the postmaster, the sysadmin running the email server, has become correspondingly more time consuming and complex. Indeed in large organizations the role of postmaster has grown to demand an extremely specialized person dedicated to that service alone.

The sending of email messages between computers has been happening since the early 1970s but as the Internet has developed it has become a more hostile place. In those early days it was commonplace for email to be relayed between many different servers on route to its final destination but with the growth in the use of email, particularly for commercial purposes, this relaying facility was abused by those who wished to send anonymous or bulk email messages using email servers on high bandwidth connections. As a result the idea of third party relaying, accepting messages for other destinations that are nothing to do with your server and forwarding them on, is no longer encouraged. Happily, now that network connectivity is vastly improved, the requirement to “help out” has also disappeared.

Along with the rise in Unsolicited Commercial Email (UCE), commonly called spam or junk mail, recent years have seen an explosion of computer viruses and worms that propagate themselves via email. Therefore not only does the postmaster’s job entail configuring and maintaining the email server but a large part of the job now involves protecting that email server and its users from UCE and these prevalent viruses and worms.

A program that transfers email messages from one computer to another is called a Mail Transfer Agent (MTA). The MTA normally works behind the scenes. A user does not normally interact with an MTA; the user interacts with a Mail User Agent (MUA), more simply known as an email client, and it is the MUA that will then interact with the MTA. The protocol that MTAs use to communicate with each other and so transfer messages between machines is called the Simple Mail Transfer Protocol (SMTP) and it is really this protocol that makes Internet email possible. It should be noted that whilst MTAs use SMTP to both receive and send messages, the SMTP protocol is also used by MUAs (email clients) to send the messages to an MTA. When an MUA interacts with an MTA to read or download messages it is not done via the SMTP protocol, in this case it will probably be using the IMAP or POP protocol.

In order to understand how MTAs work it is useful to understand a little of the SMTP protocol.

The SMTP protocol

In the 1980s most email was transmitted using the Unix to Unix Copy Protocol (UUCP) which had been designed to support the transfer of email between intermittently connected machines. As more machines began to have permanent network connections rather than dial up connections, SMTP was introduced and was optimized for the situation where the sending and receiving machines were permanently connected to the network. Of course as time progressed SMTP completely took over from UUCP as permanent connectivity became the norm and SMTP has become the de facto standard for Internet email.

SMTP is a fairly simple text based protocol where a connection is established on TCP port 25, then one or more recipients are specified and this is then followed by the text of the message itself. In fact the SMTP protocol is simple enough that a real person can use the telnet program to connect to an SMTP server and then replicate the conversation that usually takes place between two MTAs. Although this may sound like a pointless exercise this technique is often used by postmasters when troubleshooting problems! SMTP is highly reliant on the Domain Name Service (DNS) for its operation as the SMTP server for a given domain is determined by looking up that domain in the DNS and searching for an MX (Mail eXchange) record which identifies the server that handles email for that domain.

Alongside SMTP another important protocol used in the transmission of email is Multipurpose Internet Mail Extensions (MIME). MIME is an Internet standard that describes the format of email and allows information other than the characters used for the English language to be transmitted in email messages. In particular it enables the sending of binary attachments to email messages. Virtually all Internet email is transmitted in MIME format via the underlying SMTP protocol, hence Internet email is sometimes referred to as SMTP/MIME email.

In this article we shall briefly consider some leading open source MTAs that, alongside appropriate UCE and virus detection software, can provide the basis of a robust email service.

Sendmail

Sendmail is an MTA for Unix and Linux platforms. Sendmail was one of the first (if not the first) MTAs to implement SMTP and it is hugely important whenever we consider mail servers. Despite its waning popularity Sendmail still has a significant install base, due in large part to the fact that it was once the standard MTA shipped with most variants of the Unix operating system. The original version of Sendmail was written by Eric Allman in the early 1980s at University of California, Berkeley. For many years Sendmail was distributed under the Berkeley Software Distribution License under which it first shipped. However, Sendmail’s author, Eric Allman, went on to form a company and from Sendmail 8.9 onwards the Sendmail program has been developed and released by this new company, Sendmail, Incorporated under the Sendmail License. Although this is not an OSI certified licence, we would be remiss if we did not include Sendmail in this brief survey.

The Sendmail program has a monolithic design which means that all its activity is controlled by a single binary program. This program is often run as the Unix superuser, root, which clearly represents a severe security threat if it is compromised. However, from Sendmail version 8.12, released in 2001, it is no longer installed by default to run as the superuser.

Sendmail uses a somewhat unique configuration process amongst MTAs; the postmaster configures Sendmail by specifying macros in a special macro file and then using the m4 macro processor program to generate a configuration file, usually called sendmail.cf, which is used by the Sendmail program itself. This file is not intended to be human readable although there are some experienced Sendmail administrators who can in fact make sense of a sendmail.cf file! Sendmail 8.12 introduced support for Milter, an external mail filtering program.

Sendmail has its fans but it also has its detractors who criticize it for being slow, overcomplicated and difficult to maintain especially when compared with recently developed MTAs such as those discussed below. Interestingly, the next generation of Sendmail that is currently under development, MeTA1, is radically different from previous versions of Sendmail and clear parallels can be drawn between this new design and the design of Postfix. Time will tell whether this will silence Sendmail’s detractors or whether this will simply galvanise current Sendmail users to switch to Postfix.

Exim

Exim is a mail transfer agent (MTA) available for Unix and Linux operating systems and is distributed under the GNU General Public License. Exim was written in 1995 by Philip Hazel for use in the University of Cambridge Computing Service’s email systems and since then it has had an impressive take up both in academia and commercial Internet Service Providers (ISPs).

Exim was written as a drop in replacement for Sendmail and so supports the vast majority of Sendmail’s command line arguments. At its inception Exim was loosely based on Smail 3, but its rapid evolution means that it is now far more flexible and user friendly. Like Sendmail, Exim has been designed so that a single main program controls all the facilities of the MTA. This design leaves Exim open to the same security criticisms borne by Sendmail, most notably that if the controlling binary process is compromised then a great deal of control is won, but from the start Exim deprecated running as the superuser and Exim’s actual security record is much better than Sendmail and is comparable with Postfix.

Exim is configured using a configuration file that is easy to read once the postmaster becomes familiar with its concepts of directors, routers and transports. It is extremely flexible and extensible and has some notable features such as support for quotas independent of the operating system, and customizable error and warning messages.

Additionally, Phillip Hazel developed a new library called PCRE (Perl Compatible Regular Expressions) and used it within Exim to replace the old-style POSIX regular expression library. This gave Exim the ability to use the much more powerful Perl regular expressions and has proved popular enough that the PCRE library is now found in many other applications. Exim also provides its own filtering language to provide server side filtering of messages on a per user basis.

Exim was designed for the situation in which most mail is immediately deliverable and in this situation it performs very well. However, if mail is not delivered immediately and is therefore queued then Exim’s perfomance is poorer compared to other MTAs that employ a design model including a central queue manager.

Exim is shipped as the default MTA in Debian GNU/Linux systems since version 3.x and the current version of Exim is 4.69.

Postfix

Postfix is an open source MTA written by Wietsa Venema whilst on a stay at IBM, and has been known in the past as Vmailer and IBM Secure Mailer. Venema, a security expert and author of the well known security program TCP wrappers, wrote Postfix as an alternative to Sendmail that would be fast, secure and easy to administer. It is available for Unix, Linux and Mac OS X platforms and is released under the IBM Public License.

Postfix has a very different design compared with Sendmail and Exim as it does not use a single program to control all the MTA functions. Instead, Postfix divides the mail handling tasks amongst many small programs, each with a limited feature set making security holes easier to locate. Security was always very much at the forefront of Postfix’s design so none of these small programs run as the superuser, they run as a non-privileged postfix user. Postfix handles all mail using four different mail queues; incoming, for holding mail received by the mail server; active, for mail being processed; maildrop, to hold mail sent from the local system; and deferred, for holding mail that cannot be delivered for some reason. It is precisely the fact that mail is segregated in this way that keeps the mail server performing well under heavy load.

In addition to the security that this design offers, it also offers better scalability and improved speed over Sendmail. On the same hardware Postfix handles mail several times faster than Sendmail and is much more efficient at handling high loads. Like Exim, Postfix was designed to be a drop in replacement for Sendmail making the job of moving from Sendmail to Postfix a much less painful task for the postmaster. Fans of Postfix believe that Postfix’s modular design and ease of configuration mean that those who invest the time in moving from Sendmail to Postfix will very quickly reap the benefit of this move in time saved through its comparative ease of administration and sound security.

Postfix is the default MTA in Trustix Secure Linux and Mac OS v10.3 onwards and the current version of Postfix is 2.6.2.

Other MTAs

There are many more MTAs available, and proprietary examples such as Microsoft’s Exchange and IBM’s Domino can be found in widespread use. Another competitor to Sendmail, Exim and Postfix that has seen significant use is qmail. Qmail is not open source licensed either; its licence is simply a statement that redistribution in unaltered form is permitted.

The interoperability of all of these MTAs is built upon a number of open standards called Requests For Comments (RFCs) from the Internet Engineering Task Force (IETF), which cover the formatting, addressing, encoding and many other aspects of email.

Further Reading

Links:

Related information from OSS Watch: