Let’s do Dovecot slowly and properly – Part 1: PLAIN as day

Part 1 of this new Dovecot series (this one) will cover the very basics, part 2 will cover better and safer authentication and authorization, and after that we will turn our attention back to Postfix, saving more advanced topics for later (parts 3 and 4).

This tutorial presumes knowledge of Postfix and the setup we’re aiming for is one that complements the Postfix one that we’ve set up in previous installments. As with the Postfix series I want to arrive at a working setup from the very first post but knowing full well that it’s not an ideal or final setup. The advantage (over importing somebody else’s full featured setup) is that we’ll actually understand what we have on our hands (and it’s shortcomings). This makes it a lot easier to build and improve upon it and fix it should the need arises.

A note on safety: The setup we’ll end up with today is not going to be confidential in any way, shape, or form. It will expose both the contents of the account’s emails and whatever password you choose to the entire internet. Therefore you should obviously use either a test account or a brand new one that has nothing important on it yet. As for passwords you should pick one for testing that you have not used nor intend to use for any non-testing purposes. That said, un-confidential is not the same as unsafe. Any public facing service is a potential attack vector but this setup is – to the best of my knowledge – no more of one than a more properly confidential setup.

The role of the mail delivery agent

Let’s start with the really-not-stupid ‘stupid question’: What does Dovecot do and why do I need it? Don’t I already have a mail server installed in Postfix?

Quick answer: Postfix is a mail transfer agent that takes care of mail coming into and leaving the server. It can also leave it on the hard drive for system users to find. It does not, however, implement the protocols that enable modern-day email clients to access the mail, i.e. POP3 and IMAP. That’s where Dovecot comes in. The follow-up question I had to that answer was to ask why, when Postfix does so much already, can’t it just go that last mile? Why do I need yet another program just to enable the ‘download’ of my mail? I don’t know the correct answer but I suspect that it’s one part unix philosophy, one part history, and one part free software.

The dictum of do one thing and do it well would argue for a separation of the job of mail transfer agents (MTA) talking to one another and the job of talking to mail user agents (i.e. desktop or phone email clients). As for history, well, originally an MTA was all you needed because you did access mail as a system user on a terminal. The idea that email was something you needed to access independently of that workplace terminal was a later addition. And while some MTAs did incorporate MDA functionality early on, others may have dragged their feet at this novelty long enough for users to get impatient and implementing their own addon solutions. And so the MTA-MDA distinction was born and much later software projects, like Postfix and Dovecot still follow this convention. Probably also because it’s a convenient way to organise an open source division of labour: Smaller teams (or one-man teams) means less need for coordination and more room for the my-way-or-the-highway-take-it-or-leave-it mentality. Which seems to appeal to FOSS tinkerers.

The plan

This is what we’ll be aiming for as the very basic setup in this first post:

• An IMAP server on port 143
• All communications between client (Mail User Agent) and imap server (Mail Delivery Agent) are in plaintext, unencrypted.
• This includes password transmission
• Password is stored in plaintext on the server
• There is one common password for all accounts and no user checking
• All mail on the server is owned by a single system user.

We are pretty much checking as many boxes as we can to make a sysadmin see red here. As I said, this is only intended for testing and the only risk we run is that of no privacy, not of bad guys taking over our machine.

Configuration settings

When writing about Postfix I noted that there were two config file philosophies: The maximalist and the minimalist. The former contains every conceivable setting under the sun with tons of descriptive text, each setting either set to defaults or all commented out. The latter only contains the (few) areas where settings depart from defaults. Dovecot decidedly falls into the maximalist camp. Apart from /etc/dovecot/dovecot.conf there is the subdirectory of /etc/dovecot/conf.d with around 15 sub-config files, all sourced by the main config file. Fortunately, dovecot comes with a parameter that writes out it’s configuration in full without comments:

dovecot -n

The advantage of this approach is that it is less overwhelming than one big file. The disadvantage is that it is going to be tricky remembering which ones you have made changes to. I recommend making a note every time you change a file so that you can retrace your steps. Alternatively, you can backup the original directory and use diff to keep track of changes. Personally, I like creating my own Docker image that COPYs in a few customised .conf files. Whatever works for you.

In the following we will run through the needed configuration options for our first setup. All other options should be left as-is. The dovecot convention seems to be to write out the default setting in the config file, commented out. That way you can know what the default is and whether you need to make changes without having to consult documentation. On the minus side it can feel like you need to uncomment in order for the setting to take effect when in fact you don’t. Note that this does not apply to sourcing extensions (i.e. commented out .ext files are not sourced) or service definitions (i.e. commented out services like service pop3 { ... are not run).

I’m going to assume a) that we already have Postfix setup to virtual mail and b) that we have Dovecot installed with config files laid out as detailed above. If for whatever strange reason, you’re not seeing these config files, they can be extracted from the releases of dovecot source code. Check the core-x.x.xx/doc/example-config/conf.d folder of the zip file.

Mail location

Mail location tells dovecot where to find mail left by the mail transfer agent or MTA, i.e. Postfix. In a later post I will address how and why the MTA should hand over mail to the MDA (Dovecot) and leave the final delivery to it rather than having the MTA itself shove it in your your mailbox (short answer: spam filters and sorting). For now we will rely on the setup from the Postfix posts where Postfix writes mail to a vmail directory of your choosing and just leaves it there for Dovecot to find and advertise to the end user.

In order for Dovecot to find the mail left behind by Postfix, their settings for mail locations have to point to the very same place and order that you have previously set in Postfix. For more on these settings in Postfix, please see part 4 of the Postfix series. As an example the following setups should match each other

In Postfix’s main.cf I have the setting

virtual_mailbox_base = /var/vmail

And in Postfix’s virtual_mailbox_maps hash file I have

alice@brokkr.net         brokkr.net/alice/maildir/

Which means that mail for Alice ends up in a maildir structure under /var/vmail/brokkr.net/alice/maildir/.

Dovecot’s conf.d/10-mail allows us to express the same only with general rules rather than a specific map for each account:

mail_home = /var/vmail/%d/%n
mail_location = maildir:~/maildir

%d is a variable that resolves to the domain name of the recipient and %n resolves to the username. So in the specific instance of Alice, mail_home becomes /var/vmail/brokkr.net/alice/. mail_location references this ‘home’ setting with ~ and prepends it with /maildir so the end result, like above, becomes /var/vmail/brokkr.net/alice/maildir/. The mail_locationalso explicitly tells dovecot to treat it like a maildir setup rather than an mbox one.

Plaintext authorization

Dovecot documentation will invite you to pick an ‘authorization mechanism’ from a long list of such. The authorization mechanism refers to the protocol by which your client will connect to the server and present it’s credentials. It’s important to distinguish this concept from that of the server checking said credentials. That latter part falls under the heading of the password database which we will address shortly.

The authorization mechanism is mostly set in the conf.d file 10-auth.conf. In Ubuntu 18.04 10-auth.conf comes with PLAIN authentication mechanism explicitly enabled and not commented out.

auth_mechanisms = plain

plain sets down the most basic of login mechanisms. I will give a quick demonstration of it in a short while but first there’s a quirk of configuration that we need to discuss first. When trying to run with this and log in on port 143 and no encryption using the Geary email client, I was rejected and found the following in my logs:

dovecot[6795]: imap-login: Disconnected (tried to use disallowed plaintext auth)

Disallowed? Apparently the idea is that dovecot has enabled plain by default but not because they want you to check emails transparently to the world. At the top of 10-auth.conf you should find the following commented out setting:

#disable_plaintext_auth = yes

As mentioned these comments represent the default, so plaintext authentication is disabled despite being added to auth_mechanisms. A reasonable question then is: Why have one setting adding it to a list of allowed authentication mechanisms only to have another ban it? The reason is that you can use plain safely by simply using the mechanism inside an encrypted channel. It is still the same protocol, the same exchanges happening as in ordinary plain, but all communications are encrypted. A somewhat weak analogy might be meeting up with a friend but instead of talking English (or whatever other language people around you speak) you would switch to French language in order to exchange the exact same greetings and messages, i.e. translating “How do you do?” into French, not cheek kissing.

In order to allow ourselves to use unencrypted plain we need to explicitly set disable_plaintext_auth to no

disable_plaintext_auth = no

So what is the plain authorization mechanism? As with many such protocols we can ape it using telnet:

user@host:/# telnet localhost 143

Dovecot (listening on port 143) responds with a greeting:

Trying 86.52.55.217...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot (Ubuntu) ready.

And I tell it that I want to log in with the following username and password:

a1 login alice@brokkr.net ThisIsABadPassword

And dovecot graciously accepts:

a1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE] Logged in

It’s not fancy but it works and once we manage to deliver it over an encrypted channel it will be both safe and easy to use as any client you can think of supports it.

Users and passwords

Even if you ultimately intend to have multiple users, for the purposes of these initial maneouvres, we are going to assume a single user. This allows us some shortcuts when setting up password checking. Dovecot supports a bewildering array of ways to check that the user you’re authenticating as has the right to access the mail you want to access. Fortunately these include the fairly simple static database. You will find all the specific static database settings in the extension file auth-static.conf.ext in the conf.d directory.

Normally, you would look up a user who wants to log in in a database (specifically the passdb database in dovecot’s settings). In this database you see if a) the user exists and if so b) what the user’s password is so that you can check it against whatever the attempted login has offered up. A ‘static’ database always returns the same result regardless of what is put in. In our auth-static.conf.ext we therefore put the following:

passdb {
  driver = static
  args = password=ThisIsABadPassword
}

This database will always respond positively to “ThisIsABadPassword” regardless of which user we tell it we are checking. In fact, it doesn’t even check the user name because the name of the user trying to log in is ignored: It just says yes as the answer to any user/password request that has “ThisIsABadPassword” in the password field. You can even log in as non-existing user and dovecot will allow it…

dovecot[16529]: imap-login: Login: user=<bob@brokkr.net>, method=PLAIN, rip=86.52.55.217, lip=10.0.4.3, mpid=29, session=<3GhQeEJtVKZWNDfZ>

… and even create the necessary maildir directories for the user but they will obviously be empty. Enter a wrong password, however, and dovecot says no:

dovecot[16529]: imap-login: Disconnected (auth failed, 1 attempts in 4 secs): user=<bob@brokkr.net>, method=PLAIN, rip=86.52.55.217, lip=10.0.4.3, session=<Qw9ihkJt6KZWNDfZ>

You may have noticed that the same file also has a userdb setting. Don’t confuse this with username checking: As mentioned above the way to check if a username exists is that passdb responds positively to username/password input. The primary purpose of the user database is linking the user_name_ to a system user, i.e. where is the mail located on the server and who owns the it (user:group). We have already set the location globally and (at the very least at this juncture) have no reason to change it on a per-user basis. As for who owns it, we have already told postfix to assign the mail it deposits to the vmail user of the vmail group via uid and gid respectively (see the virtual_uid_maps and virtual_gid_maps of this post). So all that we need to do here is replicate those settings in words Dovecot understands:

userdb {
  driver = static
  args = uid=vmail gid=vmail
}

Finally we need to make sure that our settings are used. At the very end of conf.d/10-auth.conf you will find lines instructing dovecot to parse various extension files, one for each type of user/password database. Most of them are commented out by default so in order to use our auth-static.conf.ext we need to un-comment the line

#!include auth-static.conf.ext

And while it’s not strictly necessary it will make for cleaner logs to disable the (by default uncommented and enabled) extension that calls on username checking via PAM by commenting out the line:

!include auth-system.conf.ext

You can have multiple user/password databases but we will not use PAM so it might as well not bother and put out warnings about misfiring checks.

Valid users

Above we set the vmail user (of vmail group) as the sole system user owner and proprietor of all email. All well and good assuming that Postfix operates by the same convention. Two things to be aware of though.

The first is that Dovecot has an additional setting that limits who can and cannot own mail and we need to make sure that our vmail user does not fall foul of those rules. In the conf.d file 10-mail.conf you will find the following.

# Valid UID range for users, defaults to 500 and above. This is mostly
# to make sure that users can't log in as daemons or other system users.
# Note that denying root logins is hardcoded to dovecot binary and can't
# be done even if first_valid_uid is set to 0.
#first_valid_uid = 500
#last_valid_uid = 0

Check if your vmail user’s uid is 500 or above (last_valid_uid = 0 means no upper limit) by running

id vmail

… and lower the limit in case it is not. The similar limits on GIDs should be just below.

The other thing to consider is purely for those running Dovecot and Postfix on separate (virtual) machines or containers: Make sure that the vmail username and group correspond to the same uid/gid on both vms/containers. If not you’ll probably run into issues with either Postfix delivering mail or dovecot handling it.

Ports

I’m going to ignore POP3 and focus exclusively on IMAP here which narrows the question of ports down to two: 143 is for non-encypted access and 993 for encrypted (SSL/TLS). Since we’re not (yet) interested in encryption all we need to do is make sure that dovecot listens on 143. Which it does by default in my install. Ports are set as part of a listener inside the definitions of services. You can check the conf.d/10-master.conf for the imap-login service:

service imap-login {

  inet_listener imap {
    port = 143
  }

  inet_listener imaps {
    #port = 993
    #ssl = yes
  }

}

inet_listeners are added to the service on an as-needed basis. While the documentation doesn’t state so explicitly it seems the assigned names to each listener (‘imap’ and ‘imaps’ in the above configuration) are fixed because I got errors when I tried changing them. You can however set the port numbers as you wish in case you believe in security by obscurity (which you shouldn’t of course). You can disable a listener by setting it to port = 0 or just by commenting out all the options as in imaps above (I checked with nmap and there’s nothing listening on port 993 at the moment).

Also, be wary of old tutorials directing you to set listeners in the protocol definitions in conf.d/20-imap.conf as this seems to have changed as far back as ~ 2010-2011. The method described above is the correct one for current (2018) versions of dovecot.

Finally, you should obviously direct routers and/or firewalls to pass on connections to port 143 to the proper machine.

Testing

All that remains is to fire ‘er up. Geary is a nice and simple email client for this purpose. Server is your mailserver, either it’s name on your local network or a fully qualified domain name, Username is the full email address, Password is what you set it to in the passdb and Encryption is ‘None’ (which should automatically set Port to 143). Set SMTP settings to whatever you’re currently running with and go ahead. Geary will warn you about the dangers of not using encryption. Blithely hit ‘Continue’ and whatever mail there is in the account should appear.

So far so good. We have access but obviously not a very good one. The next priority will be to encrypt our communications with dovecot and find a more suitable way to store a password than plaintext on our server.

Let’s do Dovecot slowly and properly: Part 2 – Proper authentication

White bird on person’s hand © Artem Podrez, Pexels license