About a year ago I wrote a scathing comment about Ars Technica’s too-slow guide to self-hosted email. I wanted it now-now-now and not over the course of four weeks. Well, I guess it’s time to eat humble pie because my setup didn’t survive transfer to the new machine. Seeing as I hadn’t acquired any deep understanding of the setup apart from a purely abstract grasp of email fundamentals I didn’t have any way to fix it.
In short, I’m going to go in the complete opposite direction and take it slow this time. I am going to setup an MTA that will accept email and that I can use to send email. Only when I thoroughly understand that will I move on to Dovecot and the like. I am going to take it step-by-step, making sure that I understand what I am doing and why I am doing it and sometimes even what I am not doing.
In the course of this first piece in a series, I will get some sort of email up and running – and get that warm feeling that I have accomplished something – but consider it more of a proof-of-concept or a learning experience than anything like the final thing.
Postfix is an MTA and one that is thoroughly grounded in the old school. This means that the ‘default’ concept of email is one in which email is something you, a real system user created by the linux system administrator by
useradd with your own home directory and so forth, receive on the shell. You log in to your shell and a line announces that you have mail. This is what is known as a local account, i.e. an email account that corresponds to an account on the wider system. At it’s most basic you can read mail by using
/var/mail/[username]. If this is what I want postfix is very easy to setup.
Is this what I want? No. But it’s what I am going to do because it’s very easy to setup and it will allow me to quickly see that Postfix works. I send mail to real users, Postfix accepts it. I send it garbled nonsense, it objects. I send mail to non-existant users, it objects. I ask it to pass on excellent offers of penis enlargement to other domains, it tells me no.
Even if this setup will not be useful in a practical sense, it will be useful for gaining a proper understanding of how Postfix works. Because it’s easy it will also mean that I can quickly get something, anything, that works. It will also mean that the learning curve is going to have a nice historical arc to it: From in-house, local mail to internet-based virtual mail.
A note on ‘local’ vs. ‘virtual’
Getting to know a local setup will also help me avoid using local mail settings, once I start on virtual accounts. Virtual is the opposite of local: It means that email accounts do not correspond to local system accounts. When email was something you only used at work to communicate with colleagues, all of whom were logged into a shell the entire working day, a local system made perfect sense. Today, not so much.
‘Local’ is easily misunderstood to mean, well, local, only using local appellations such as localhost, internal IP addresses or system nicknames like MikesLaptop. As opposed to referencing internet domain names and DNS. This is not so. You can have a Postfix local setup that references the mail server by it’s internet address – as long as the email accounts correspond to a system account. With such a setup, mail to firstname.lastname@example.org should, all other things being equal, go to the user
nosuchuser on my server and be concatenated onto the text file /var/mail/nosuchuser where nosuchuser can read it by ssh’ing into the server and cat’ing the file.
Reset your install
To clear any previous attempts at setup, I will start by resetting everything. On a debian based system, there’s
apt-get purge for getting rid of the package AND the configuration files. Substitute whatever’s appropriate for your system:
sudo apt-get purge postfix
sudo apt-get install postfix
Installing postfix will in most cases start up an ncurses setup wizard. Selecting ‘Local site’ will set up some basic configuration where local users on the host are automatically recognised as legitimate recipients. After that you will likely be asked for a hostname which may or may not be autofilled in with the value of
$HOSTNAME. Accept and quit.
Be aware that Postfix does sometimes use configuration files that are not part of the postfix package (
/etc/mailname for example) so that some remains of previous setups may be difficult to get rid off.
Okay, now that’s out of the way…
Configuration: The very basics
The main configuration file of Postfix is …
/etc/postfix. That was easy. Here’s the tricky part: Depending on your system, the main.cf file can look very differently. As with all packages, some people prefer huge, commented out configuration files with all possible settings in them and some prefer purged, minimalist ones that only contain what’s needed. As a result of my choice of ‘Local site’ configuration in the install wizard I have inherited a few extra settings (in my setup every line from
#TLS parameters and down has ben added dynamically).
I am going to focus on four settings that are very likely to be in your main.cf. If not they should be. As long as these are properly set up, I should have something that works. You can safely use the defaults whether implicit or explicitly written out for all other settings. You should however check that the configuration file does not contain multiple lines for these settings as later ones will override earlier ones.
You might have been warned about some of these in previous guides (at least
mydestination). The reason for this is, as previously stated, some are specific to local setups and will disrupt virtual setups. I am explicitly doing a local setup here, so it’s perfecly okay.
Each setting is set by an equal sign surrounded by spaces. Never use quotation marks. There are a wide variety of ways to provide values for Postfix settings – databases, external files, etc. – but I will stick to single values and multiple values, separated by commas.
myhostname and mydomain
myhostname is the server’s identity. It defaults to the name I have given the machine when I installed the OS (i.e. the output of
uname -n), e.g. NEWSERVER. Right now it should be set to what you entered in the install wizard. Later on we will assign it the Fully Qualified Domain Name (FQDN) of the machine on the internet, e.g. mail.brokkr.net.
mydomain is the name of the domain that recieves email, e.g. brokkr.net.
Mostly these serve as variables to be used in other settings. Postfix uses variables simply by prepending a dollar sign to the variable name and you can reference variables even before they are set.
myhostname however also serves as the identification of the smtp server when I call on it. In other words, when other SMTP servers connect to mine, they will introduce themselves and mine will respond with “Hi, I’m
As long as I stay on my one machine, I have no need of
mydomain. Postfix has various clever ways of figuring out what the domain name or the host name is if either hasn’t been set but that is of little use and no concern right now.
myhostname = ACORP
myorigin serves to flesh out shorthand addresses. Imagine: I am Alice, you’re Bob. We work for ACORP and have email on the same server, called ACORP. Whenever I write to you, Bob, I don’t want to fill out the from field with alice@ACORP and the to field with bob@ACORP. So I just write from ‘alice’ and to ‘bob’.
myorigin serves to add the setting to incomplete addresses like these (remember this is before address books and the like). Postfix defaults to using
$myhostname because the scenarios in which
myorigin is likely to be used are such very in-house ones as the one I’ve painted. To keep things transparent, let’s spell it out:
myorigin = $myhostname
Setting and reusing the
myhostname variable is helping keep things simple already.
mydestination is usually a list written as multiple values, separated by commas. It is a list of “all the domains your Postfix system should accept mail for and deliver to local users” in the words of Postfix The Definitive Guide and should at the moment have ben set by the install wizard to a somewhat overlengthy string.
mydestination is very important: If the domain to which an email is addressed is not in
mydestination, the email is bounced back. I will expand on this later but for now, I will set Postfix to be the destination for users on
$myhostname (a small simplification of the default and current setting). It is the same machine and the same users, just two different ways of referring to it.
mydestination = localhost, $myhostname
So now I have a name for my SMTP server, I have a way to fill out the missing bits of in-house addresses and a list of domains that my SMTP server will accept emails for. Let’s start it up and see how things work.
I start Postfix as I start any other service, depending on init. My server runs Ubuntu 15.04 so that means systemd:
systemctl start postfix.service
restart instead of
start if postfix was already running.
I’m going to assume systemd in the rest of this series. A few useful commands for this purpose are as follows.
systemctl -l status postfix.service
This will give you the status of postfix (runnning, stopped, dead for whatever reason) as well as a the latest x lines from the log (about 10) where lines too long are wrapped to the next.
journalctl -r -u postfix.service
If you need more context or older messages, you can consult postfix’s log which is very helpful in understanding what happens when it receives requests and what it does with them.
-r reverses the log so that the newest messages are at the top. That way you don’t have to scroll to the bottom of 10,000 lines.
-u postifx.service singles out messages from the unit
2019 update: On Debian 10 Buster I have noticed that although the
postfix.service file is used to control starting and stopping the master process, the sub-processes that make up postfix report to a different service, labelled
postfix@-.service. So if you wish to listen in, that is the service you should be targetting.
Have you talked to your SMTP server today?
In order to properly understand how Postfix works and responds I am going to talk to it using telnet and simulate being an external SMTP server that has an email for a user on my SMTP server. SMTP isn’t that technical a protocol. It’s practically human readable.
You can do it from the same machine or – unless you have very stringent firewall settings – from within the same network. I’m just going to use the machine itself. Replace
localhost with the internal IP or name if you’re on the outside, looking in.
telnet localhost 25
Port 25 is the standard SMTP port on which Postfix is listening. This is the response I get:
Trying ::1... Connected to localhost. Escape character is '^]'. 220 ACORP ESMTP Postfix (Ubuntu)
The first three lines is telnet, the last is Postfix greeting us and telling me it’s
$myhostname (“ACORP”) and various other information.
The number 220 is a response code, a quick way to ascertain if things went right or wrong. Basically everything in the 200s means that the command was understood and executed successfully, please carry on. 300s means that the server is waiting for more data or instructions from me (see below for an example) and the 400s and 500s are errors of increasing severity.
Convention demands that I introduce myself. Seing as I am connecting from localhost, let’s just tell it that:
HELO localhost 250 ACORP
250 is the response code that signals that my command was successful and it repeats it’s own name. Next, I tell it I have mail for it and who it’s from. The SMTP server accepts mail from all sorts of places, internally and externally. Postfix doesn’t try to verify my identity. At this basic level of setup complexity, I can easily impersonate anybody – email@example.com if I want – but for now I will keep it simple and use real users on my machine for sender and recipient.
MAIL FROM:<userone@localhost> 250 2.1.0 Ok RCPT TO:<usertwo@localhost> 250 2.1.5 Ok
MAIL FROM and
RCPT TO are our commands and the 250 codes are Postfix’ responses. Remember my
localhost was one of the two ways we could address users on the machine, the other being ACORP (i.e. my $myhostname). Substitute ACORP for localhost and you get exactly the same result. Use a domain name not in
$mydestination – e.g. nosuchdomainnamehe.re – and I get… the same result. At least apparently. Postfix accepts the address but once it tries to deliver the mail, it will discover that the domain name is not in
mydestination and tell me in the log (
systemctl -l status postfix.service) that it has bounced the message:
Oct 15 12:31:13 ACORP postfix/smtp: 5FBFFD409CC: to=<firstname.lastname@example.org>, relay=none, delay=67, delays=67/0.01/0.03/0, dsn=5.4.4, status=bounced (Host or domain name not found. Name service error for name=nosuchdomainnamehe.re type=AAAA: Host not found)
The first result also obviously depends on
usertwo existing as a system user on the machine. Postfix will not check
userone, but if delivery fails it will inform the sender hereof and so shoot of a mail back to
userone. If the username on the other hand is not a legitimate user on the system, Postfix will inform me immediately of the failure:
550 5.1.1 <userthree@localhost>: Recipient address rejected: User unknown in local recipient table
If I do everything right I now get to tell Postfix the contents of my email. I start this by entering the command
. Postfix responds with code 354 meaning that it is now waiting for us to enter the message. I simply write out the contents, line for line and signal the end of the content with a single period on an otherwise empty line. Postfix responds with a success message and informs me what ID my message has been given (“6B684D4BD2”). Then I enter the command
to break off contact with Postfix, thus also ending our telnet session.
DATA 354 End data with <CR><LF>.<CR><LF> This is an email test. . 250 2.0.0 Ok: queued as 6B684D40BD2 QUIT 221 2.0.0 Bye Connection closed by foreign host.
I have successfully sent an email by pretending to be an SMTP server. If I login as the recipient, i.e.
, I should be greeted with the announcement that I have mail. Use cat to write it out –
– and something like this should appear:
From userone@localhost Thu Oct 15 13:02:15 2015 Return-Path: <userone@localhost> X-Original-To: usertwo@localhost Delivered-To: usertwo@localhost Received: from ACORP (localhost [IPv6:::1]) by ACORP (Postfix) with SMTP id 6B684D40BD2 for <usertwo@localhost>; Thu, 15 Oct 2015 13:01:36 +0200 (CEST) Message-Id: <20151015110159.6B684D40BD2@ACORP> Date: Thu, 15 Oct 2015 13:01:36 +0200 (CEST) From: userone@localhost This is an email test.
That’s it for now. Playing around with the telnet connection is very instrutive, as it gives a lot of feel for what works, what doesn’t and how and why.
In the next part I will look at some of the more imaginative ways that my local email setup can be twisted and broken, including rewrites and aliases.
As long as my setup is unfinished it’s probably best to turn it off when I’m not working on it. Postfix is by default setup not to relay (send on) any messages not emanating from inside the network, so it is unlikely that somebody could hijack my nascent SMTP server and spam the bejeesus out of the internet (especially since I haven’t done port forwarding yet). However, better safe than sorry.
systemctl stop postfix.service systemctl disable postfix.service
Valentine’s Day Valentine Red Mailbox Pink Background © Rinck Content Studio, Unsplash license.