Let’s do Dovecot slowly and properly – Part 3: LMTP

So far I have had a pretty clear division of labour between Postfix and Dovecot. Postfix accepts mail from other MTAs and puts it into my maildir. Dovecot accepts my client’s inquiries and hands it the mail it finds in same maildir. I muddied the waters a bit when I started asking Dovecot to serve as authenticator for SMTP but those waters are about to get a fair bit murkier. Postfix will now relinquish the job of actually delivering the mail to maildirs and instead hand it over to a Dovecot service.

Why? Postfix has done a fine job of it so far, has it not? Yes, but Postfix is not a very sophisticated mailman. It shoves my mail in the mailbox/maildir that I tell it to based on virtual_mailbox_base and virtual_mailbox_maps but that’s all it can do. If I want to set up filtering rules or other intelligent processing of mail delivery, I am going to need to insert another step, namely using the Local Mail Transfer Protocol or LMTP for short. In essence, LMTP is the protocol that allows Postfix and Dovecot to actually talk to one another, rather than one just leaving messages for the other on the kitchen counter. Insert your own old-married-couple joke here.

The essence of today’s post is simple. I will install, configure and start up Dovecot’s LMTP service. Then I tell Postfix of the new order of things and test that mail is still going where it should. That is it; that is all. I won’t gain anything new and exciting by it this time round. In a later post I’ll use my new LMTP powers to do something a little more interesting, like filtering using Sieve.

Postfix without LMTP

Before I do anything to my current setup I want to take a look at what happens when an SMTP server approaches my MTA, Postfix, with a message. In general I’ve found it helpful to save logs to an archive before making any sort of changes to make it easier to compare what happens now to what happened before the changes.

Sep 25 17:10:49 232d026dd820 postfix/smtpd[179]: connect from unknown[49.88.64.125]
Sep 25 17:10:51 232d026dd820 postfix/smtpd[179]: 6DD181427F6: client=unknown[49.88.64.125]
Sep 25 17:10:54 232d026dd820 postfix/qmgr[107]: 6DD181427F6: from=<ijidfjo@zirq.com>, size=1667, nrcpt=1 (queue active)
Sep 25 17:10:54 232d026dd820 postfix/virtual[185]: 6DD181427F6: to=<alice@brokkr.net>, relay=virtual, delay=3.2, delays=3.2/0.01/0/0, dsn=2.0.0, st
Sep 25 17:10:54 232d026dd820 postfix/qmgr[107]: 6DD181427F6: removed
Sep 25 17:10:54 232d026dd820 postfix/smtpd[179]: disconnect from unknown[49.88.64.125] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5

RedinSkala has a good, detailed breakdown of a similar transaction (with some extra steps and a handover to local rather than virtual). The log follows the SMTP interaction described in the first post in this series which can be helpful to keep in mind. For my immediate purposes the following should suffice:

• 1st line is the initial handshake (“HELO”)
• 2nd line follows after the SMTP server has told Postfix that it has mail, who it’s from and who is the reipient (“MAIL FROM” and “RCPT TO”). The message has now been assigned a unique identifier (“6DD181427F6”) that follows it through the processing.
• 3rd line tells me that the message has been received (in other words, “DATA”, contents, “.”, and “QUIT”) and it is now queued to be processed.
• 4th line indicates that the mail has been handed over to the virtual daemon because the user has been identified as a virtual user.
• Finally, the 5th and 6th lines indicate that the transaction is over: The message is removed from the queue and the connection to the SMTP server ends.

I will take a look later at the same scenario once I have implemented LMTP but – spoiler alert – all that will really change is that the 4th line will disappear, i.e. that the LMTP handover takes the place of the virtual daemon.

Installing the Dovecot LMTP service

There may be some dedicated LMTP servers out there but my searches didn’t really reveal any major projects. Fortunately, Dovecot comes with an LMTP tag-on service. Depending on platform it may require a separate install. On Debian I had to do apt-get install dovecot-lmtpd. The install will plunk down an lmtpd.protocol file in/usr/share/dovecot/protocols.d/ which ensures that the service will be included as an option along with the standard Dovecot imap listeners, thanks to this line in the default dovecot.conf file.

# Enable installed protocols
!include_try /usr/share/dovecot/protocols.d/*.protocol

At least on Debian 10. YMMV.

Dovecot and Postfix LMTP configuration

Very little actually needs to change in my Dovecot setup. As mentioned previously the protocol is added automatically after install (with the default Debian doveot.conf at least) so all I need to do, is to add and configure the service. This is done similar to previous Dovecot services that I have started, like imap and imaps, in 10-master.conf.

service lmtp {
  user = vmail
  # Create inet listener only if you can't use the UNIX socket
  inet_listener lmtp {
    # Avoid making LMTP visible for the entire internet
    #address = 127.0.0.1
    port = 24

  #unix_listener lmtp {
    #mode = 0666
  #}
  }
}

I specify user in order to maintain the order that used to be implemented via a static userdb in Postfix, namely that there is only one user, vmail, that owns mail on all accounts. If user is not set, LMTP is run as root in order for it to take on a new id for each user that receives mail – which only seems relevant in a local setup, not a virtual one.

Next, I use TCP port for communication because I run Postfix and Dovecot in their own separate containers and ports are so much easier than sockets when using containers. Since only Postfix needs to talk to my LMTP service, I am not going to forward the LMTP port (informally, port 24 is common if not canon) so I am not making “LMTP visible for entire internet”. To be perfectly honest, I am not sure if address specifies allowed hosts or interfaces that lmtp should listen on. Not setting it works and doesn’t seem overly dangerous (if someone is in my Docker network, I’m probably done anyway). Because, Dovecot already knows where to look for mail – I told it earlier in 10-mail.conf so it could find the messages left behind by Postfix – I don’t need to tell the LMTP service where to archive it. The LMTP server just uses the same settings, easy peasy.

Now I just need to instruct Postfix to use the service, rather than attempt delivery itself. Entering main.cf I add a single line:

virtual_transport = lmtp:[10.0.4.3]:24

Simple, really. Protocol:[IP]:Port. If Dovecot and Postfix run on the same machine, 127.0.0.1 should do the trick (not so sure about localhost, though, as I have previously been burnt by assuming Postfix does intelligent DNS lookups. Some old settings have now been rendered redudant, so I may as well comment them out or delete them for clarity: virtual_mailbox_base, virtual_uid_maps, and virtual_gid_mapsall tell Postfix’s virtual daemon where to stash messages. But Postfix isn’t going to be writing mails directly to disk any more so it has no need of the path or the permissions for mail storage.

Mail arrival with LMTP

Restarting everything and waiting for spam to arrive (never a long wait), this is what I see in the Postfix logs:

Oct 30 21:39:17 aaac736a83e6 postfix/smtpd[120]: connect from unknown[114.237.109.144]
Oct 30 21:39:17 aaac736a83e6 postfix/smtpd[120]: DC5FF2C15F7: client=unknown[114.237.109.144]
Oct 30 21:39:18 aaac736a83e6 postfix/qmgr[106]: DC5FF2C15F7: from=<hgtbmdenb@bjub.com>, size=1435, nrcpt=1 (queue active)
Oct 30 21:39:18 aaac736a83e6 postfix/qmgr[106]: DC5FF2C15F7: removed
Oct 30 21:39:18 aaac736a83e6 postfix/smtpd[120]: disconnect from unknown[114.237.109.144] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5

As I mentioned previously in the spoiler, the only difference we notice is that the virtual daemon isn’t called anymore. So what happens instead? Let’s go look at the handover from the Dovecot side of things:

Oct 30 21:39:18 lmtp(44): Info: Connect from 10.0.4.2
Oct 30 21:39:18 lmtp(alice@brokkr.net)<44><AqKOK/b0uV0sAAAAuEYLBA>: Debug: auth USER input: alice@brokkr.net uid=1001 gid=1001
Oct 30 21:39:18 lmtp(44, alice@brokkr.net): Debug: Effective uid=1001,gid=1001, home=/volumes/mail/vmail/brokkr.net/alice
Oct 30 21:39:18 lmtp(44, alice@brokkr.net): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/maildir
Oct 30 21:39:18 lmtp(44, alice@brokkr.net): Debug: maildir++:root=/volumes/mail/vmail/brokkr.net/alice/maildir, index=, indexpvt=, control=, inbox=/volumes/mail/vmail/brokkr.net/alice/maildir, alt=
Oct 30 21:39:18 lmtp(alice@brokkr.net)<44><AqKOK/b0uV0sAAAAuEYLBA>: Debug: Mailbox INBOX: Mailbox opened because: lib-lda delivery
Oct 30 21:39:18 lmtp(alice@brokkr.net)<44><AqKOK/b0uV0sAAAAuEYLBA>: Info: msgid=unspecified: saved mail to INBOX
Oct 30 21:39:18 lmtp(44): Info: Disconnect from 10.0.4.2: Client has quit the connection (state=READY)

So apart from the very verbose (thanks to debug setttings) output, repeating all relevant settings and configs, what we see is and incoming connection from Postfix (considered the client in this relationship), a handover, a save to disk and an end to the connection.

So has anything changed on the ground? Well, it shouldn’t have. If I look at the maildir folder, this is what I see:

...
-rw------- 1 vmail vmail 5.0K Oct 29 04:07  1572318464.V802I9815a9M224875.232d026dd820:2,
-rw------- 1 vmail vmail  87K Oct 29 19:30  1572373837.V802I981400M107357.232d026dd820:2,S
-rw------- 1 vmail vmail 5.3K Oct 30 05:45  1572410712.V802I9813e6M419801.232d026dd820:2,S
-rw------- 1 vmail vmail  87K Oct 30 14:11  1572441078.V802I9813e7M481541.232d026dd820:2,S
-rw------- 1 vmail vmail 1.8K Oct 30 21:26 '1572467163.M378997P14.5ecbff28f2dc,S=1813,W=1857:2,S'
-rw------- 1 vmail vmail 1.7K Oct 30 21:39 '1572467958.M732111P44.5ecbff28f2dc,S=1673,W=1707:2,S'

It should be easy to tell when I made the switch because Dovecot clearly uses a different naming scheme but other than that? No, same uid/gid and same permissions. Perfect.

Postfix and user checking

So does Postfix need to know about users at all anymore? It asks Dovecot for user checking when sending mail (SASL) and it leaves mail distribution to users to Dovecot wholesale. There is still one thing, though: I would like Postfix to reject (or silently drop) any mail to non-existent users. But for Postfix to do that, it needs to know which users are real and which aren’t. The easy way to still have this functionality is to keep the otherwise vestigial virtual_mailbox_maps around. As detailed in a previous post, virtual_mailbox_maps maps users to mail locations. I don’t need the mapping anymore because Postfix doesn’t deliver mail to the doorstep any longer but the file can still serve as a simple user check. Here’s how it works when I send a message to nosuchuser on the domain:

Oct 31 10:38:12 aaac736a83e6 postfix/smtpd[645]: connect from unknown[198.54.122.58]
Oct 31 10:38:13 aaac736a83e6 postfix/smtpd[645]: Anonymous TLS connection established from unknown[198.54.122.58]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Oct 31 10:38:13 aaac736a83e6 postfix/smtpd[645]: NOQUEUE: reject: RCPT from unknown[198.54.122.58]: 550 5.1.1 <nosuchuser@brokkr.net>: Recipient address rejected: User unknown in virtual mailbox table; from=<me@anotherdomain.com> to=<nosuchuser@brokkr.net> proto=ESMTP helo=<MTA-08-4.privateemail.com>
Oct 31 10:38:14 aaac736a83e6 postfix/smtpd[645]: disconnect from unknown[198.54.122.58] ehlo=2 starttls=1 mail=1 rcpt=0/1 data=0/1 rset=1 quit=1 commands=6/8

As we can see the system still works: “Recipient address rejected: User unknown in virtual mailbox table”.

If I do remove virtual_mailbox_maps from my main.cf, nothing is going to crash but I will be telling spammers and others that they have found an opening and can expect a lot more traffic to those pseudo accounts. Postfix will accept mail to those accounts because why not, and Dovecots LMTP will follow the mail_location setting and, if necessary, create a new maildir for the ‘account’.

It is possible to reject mail to non-existant users with a Postfix/Dovecot collaboration scheme to avoid having to maintain a list of users with Postfix. However, for my three plus accounts I don’t mind maintaining the virtual_mailbox_maps file even if it is a bit illogical doing it this way now.

What’s next

LMTP is the prerequisite for filtering with Sieve so that is a natural next step. After that we will have a look at how our more sophisticated MDA setup can help us prevent spam from getting into our inbox.

Let’s do Dovecot slowly and properly – Part 4: Filtering with Sieve

Text © Kindel Media, Pexels license