About a year ago I wrote a scathing comment about Ars Technica’s too-slow guide to self-hosted email. I wanted it now-now-now and not over the course of four weeks. Well, I guess it’s time to eat humble pie because my setup didn’t survive transfer to the new machine. Seeing as I hadn’t acquired any deep understanding of the setup apart from a purely abstract grasp of email fundamentals I didn’t have any way to fix it.
In short, I’m going to go in the complete opposite direction and take it slow this time. We are going to setup an MTA that will accept email and that we can use to send email. Only when we thoroughly understand that will we move on to Dovecot and the like. We are going to take it step-by-step, making sure that we understand what we’re doing and why we’re doing it and sometimes even what we’re not doing.
In the course of this first piece in a series, we will get some sort of email up and running – and get that warm feeling that we have accomplished something – but consider it more of a proof-of-concept or a learning experience than anything like the final thing.
Postfix is an MTA and one that is thoroughly grounded in the old school. This means that the ‘default’ concept of email is one in which email is something you, a real system user created by the linux system administrator by useradd with your own home directory and so forth, receive on the shell. You log in to your shell and a line anounces that you have mail. This is what is known as a local account, i.e. an email account that corresponds to an account on the wider system. At it’s most basic you can read mail by simply using cat or less on
/var/mail/[username]. If this is what you want postfix is very easy to setup.
Is this what you want? In all probability not. But it’s what we’re going to do because it’s very easy to setup and it will allow us to quickly see that Postfix works. We send mail to real users, Postfix accepts it. We send it garbled nonsense, it objects. We send mail to non-existant users, it objects. We ask it to pass on excellent offers of penis enlargement to other domains, it tells us to fuck off. It’s nice to see that basically, things work. It’s reassuring to keep in mind once we move into more tricky territory. It will also mean that our learning curve is going to have a nice historical, chronological arc to it: From in-house, local mail to internet-based virtual mail.
A note on ‘local’ vs. ‘virtual’
Getting to know a local setup will also help you avoid using local mail settings, once we start on virtual accounts. Virtual is the opposite of local: It means that email accounts do not correspond to local system accounts. When email was something you only used at work to communicate with colleagues, all of whom were logged into a shell the entire working day, a local system made perfect sense. Today, not so much.
‘Local’ is easily misunderstood to mean, well, local, only using local appellations such as localhost, internal IP addresses or system nicknames like MikesLaptop. As opposed to referencing internet domain names and DNS. This is not so. You can have a Postfix local setup that references the mail server by it’s internet address – as long as the email accounts correspond to a system account. With such a setup, mail to email@example.com should, all other things being equal, go to the user nosuchuser on my server and be concatenated onto the text file /var/mail/nosuchuser where nosuchuser can read it by ssh’ing into the server and cat’ing the file.
Reset your install
If this is not your first attempt at setting up Postfix, how about we clear the table, eh? Start afresh and all that. On a debian based system, there’s apt-get purge for getting rid of the package AND the configuration files. Substitute whatever’s appropriate for your system:
sudo apt-get purge postfix
sudo apt-get install postfix
Installing postfix will in most cases start up an ncurses setup wizard. Selecting ‘Local site’ will set up some basic configuration where local users on the host are automatically recognised as legitimate recipients. After that you will likely be asked for a hostname which may or may not be autofilled in with the value of
$HOSTNAME. Accept and quit.
Be aware that Postfix does sometimes use configuration files that are not part of the postfix package (
/etc/mailname for example) so that some remains of previous setups may be difficult to get rid off. Try to retrace your steps and reset as much as possible.
Okay, now that’s out of the way…
Configuration: The very basics
The main configuration file of Postfix is …
/etc/postfix. That was easy. Here’s the tricky part: Depending on your system, the main.cf file can look very differently. As with all packages, some people prefer huge, commented out configuration files with all possible settings in them and some prefer purged, minimalist ones that only contain what’s needed. As a result of our choice of ‘Local site’ configuration in the install wizard we have inherited a few extra settings (in my setup every line from
#TLS parameters and down has ben added dynamically).
We are going to focus on four settings that are very likely to be in your main.cf. If not they should be. As long as these are properly set up we should be ready to start playing. You can safely use the defaults whether implicit or explicitly written out for all other settings. You should however check that the configuration file does not contain multiple lines for these settings as later ones will override earlier ones.
You might have been warned about some of these in previous guides (at least
mydestination). The reason for this is, as previously stated, some are specific to local setups and will disrupt virtual setups. We are explicitly doing a local setup here, so it’s perfecly okay.
Each setting is set by an equal sign surrounded by spaces. Never use quotation marks. There are a wide variety of ways to provide values for Postfix settings – databases, external files, etc. – but we’ll stick to single values and multiple values, separated by commas.
myhostname and mydomain
myhostname is the server’s identity. It defaults to the name you have given the machine when you installed the OS (i.e. the output of
uname -n), e.g. NEWSERVER. Right now it should be set to what you entered in the install wizard. Later on we will assign it the Fully Qualified Domain Name (FQDN) of the machine on the internet, e.g. mail.brokkr.net.
mydomain is the name of the domain, e.g. brokkr.net.
Mostly these serve as variables to be used in other settings. Postfix uses variables simply by prepending a dollar sign to the variable name and you can reference variables even before they are set.
myhostname however also serves as the identification of the smtp server when we call on it. In other words, when other SMTP servers connect to ours, they will introduce themselves and ours will respond with “Hi, I’m $myhostname”.
As long as we stay on our one machine, we have no need of
mydomain. Postfix has various clever ways of figuring out what the domain name or the host name is if either hasn’t been set but that is of little use and no concern right now. Set
myhostname to what you usually call your machine and leave out
myhostname = STUFFANDSTUFF
myorigin serves to flesh out shorthand addresses. Imagine: I am Alice, you’re Bob. We work for STUFFANDSTUFF and have email on the same server, called STUFFANDSTUFF. Whenever I write to you, Bob, I don’t want to fill out the from field with alice@STUFFANDSTUFF and the to field with bob@STUFFANDSTUFF. So I just write from ‘amy’ and to ‘bob’.
myorigin serves to add the setting to incomplete addresses like these (remember this is before address books and the like). Postfix defaults to using
$myhostname because the scenarios in which
myorigin is likely to be used are such very in-house ones as the one I’ve painted. To keep things transparent, let’s spell it out:
myorigin = $myhostname
As you can see, setting and reusing the
myhostname variable is helping keep things simple already.
mydestination is usually a list written as multiple values, separated by commas. It is a list of “all the domains your Postfix system should accept mail for and deliver to local users” in the words of Postfix The Definitive Guide and should at the moment have ben set by the install wizard to a somewhat overlengthy string.
mydestination is very important: If the domain to which an email is addressed is not in
mydestination, the email is bounced back. We will expand on this later but for now, let’s set Postfix to be the destination for users on localhost and
$myhostname (a small simplification of the default and current setting). It is the same machine and the same users, just two different ways of referring to it. Think about how
ping localhost and
ping $(uname -n) get you the same result.
mydestination = localhost, $myhostname
So now we have a name for our SMTP server, we have a way to fill out the missing bits of in-house addresses and a list of domains that our SMTP server will accept emails for. Let’s start it up and see how things work.
You start Postfix as your start other services, depending on init. My server runs Ubuntu 15.04 so that means we use systemd to fire ‘er up:
systemctl start postfix.service
restart instead of
start if postfix was already running.
I’m going to assume systemd in the rest of this series. A few useful commands for this purpose are as follows.
systemctl -l status postfix.service
This will give you the status of postfix (runnning, stopped, dead for whatever reason) as well as a the latest x lines from the log (about 10) where lines too long are wrapped to the next.
journalctl -r -u postfix.service
If you need more context or older messages, you can consult postfix’s log which is very helpful in understanding what happens when it receives requests and what it does with them.
-r reverses the log so that the newest messages are at the top. That way you don’t have to scroll to the bottom of 10,000 lines.
-u postifx.service singles out messages from the unit
2019 update: On Debian 10 Buster I have noticed that although the
postfix.service file is used to control starting and stopping the master process, the sub-processes that make up postfix report to a different service, labelled
postfix@-.service. So if you wish to listen in, that is the service you should be targetting.
Have you talked to your SMTP server today?
In order to properly understand how Postfix works and responds we’re going to talk to it using telnet and simulate being an external SMTP server that has an email for a user on our SMTP server. That may sound complicated but SMTP isn’t that technical a protocol, it’s practically human readable.
You can do it from the same machine or – unless you have very stringent firewall settings – from within the same network. I’m just going to use the machine itself. Supplant
localhost with the internal IP or name if you’re on the outside, looking in.
telnet localhost 25
Port 25 is the standard SMTP port on which Postfix is listening. This is the response I get:
Trying ::1... Connected to localhost. Escape character is '^]'. 220 THEMINT ESMTP Postfix (Ubuntu)
The first three lines is telnet, the last is Postfix greeting us and telling us it’s
$myhostname (“THEMINT”) and various other information.
The number 220 is a response code, a quick way to ascertain if things went right or wrong. Basically everything in the 200s means that the command was understood and executed successfully, please carry on. 300s means that the server is waiting for more data or instructions from you (see below for an example) and the 400s and 500s are errors of increasing severity.
Convention demands that we introduce ourselves. Seing as we’re connecting from localhost, let’s just tell it that:
HELO localhost 250 THEMINT
250 is the response code that signals that our command was successful and it repeats it’s own name. Next, we tell it we have mail for it and who it’s from. The SMTP server accepts mail from all sorts of places, internally and externally. Postfix doesn’t try to verify our identity. We can easily impersonate anybody – firstname.lastname@example.org if you want – but for now let’s just keep it simple and use real users on our machine for sender and recipient.
MAIL FROM:<userone@localhost> 250 2.1.0 Ok RCPT TO:<usertwo@localhost> 250 2.1.5 Ok
MAIL FROM and
RCPT TO are our commands and the 250 codes are Postfix’ responses. Remember our
$mydestination setting? localhost was one of the two ways we could address users on the machine, the other being THEMINT (i.e. my $myhostname). Substitute THEMINT for localhost and you get exactly the same result. Use a domain name not in mydestination – e.g. microsoft.com – and you get… the same result. At least apparently. Postfix accepts the address but once it tries to deliver the mail, it will discover that the domain name is not in
mydestination and tell you in the log (
systemctl -l status postfix.service) that it has bounced the message:
Oct 15 12:31:13 THEMINT postfix/smtp: 5FBFFD409CC: to=<email@example.com>, relay=none, delay=67, delays=67/0.01/0.03/0, dsn=5.4.4, status=bounced (Host or domain name not found. Name service error for name=nosuchdomainnamehe.re type=AAAA: Host not found)
The first result also obviously depends on
usertwo existing as a system user on the machine. Postfix will not check
userone, but if delivery fails it will inform the sender hereof and so shoot of a mail back to
userone. If the username on the other hand is not a legitimate user on the system, Postfix will inform you immediately of the failure:
550 5.1.1 <userthree@localhost>: Recipient address rejected: User unknown in local recipient table
If we do everything right we now get to tell Postfix the contents of our email. We start this by entering the command
. Postfix responds with code 354 meaning that it is now waiting for us to enter the message. We simply write out the contents, line for line and signal the end of the content with a single period on an otherwise empty line. Postfix responds with a success message and informs us what ID our message has been given (“6B684D4BD2”). Then we enter the command
to break off contact with Postfix, thus also ending our telnet session.
DATA 354 End data with <CR><LF>.<CR><LF> This is an email test. . 250 2.0.0 Ok: queued as 6B684D40BD2 QUIT 221 2.0.0 Bye Connection closed by foreign host.
We have successfully sent an email by pretending to be an SMTP server. If you login as the recipient, i.e.
, you should be greeted with the announcement that you have mail. Use cat to write it out –
– and something like this should appear:
Thu Oct 15 13:02:15 2015
Received: from THEMINT (localhost [IPv6:::1]) by THEMINT (Postfix) with SMTP id 6B684D40BD2 for <usertwo@localhost>; Thu, 15 Oct 2015 13:01:36 +0200 (CEST)
Date: Thu, 15 Oct 2015 13:01:36 +0200 (CEST)
This is an email test.
That’s it for now. I would encourage you to play around with the telnet connection as it gives a lot of feel for what works, what doesn’t and how and why.
In part 2 we will look at some of the more imaginative ways that our local email setup can be twisted and broken, including rewrites and aliases.
As long as your setup is unfinished it’s probably best to turn it off when you’re not working on it. Postfix is by default setup not to relay (send on) any messages not emanating from inside the network, so it is unlikely that somebody could hijack your nascent SMTP server and spam the bejeesus out of the internet (especially since we haven’t done port forwarding yet). However, better safe than sorry.
systemctl stop postfix.service systemctl disable postfix.service