Squeezing the sponge: How to implement a buffering SMTP Handler in Python

…when he needs what you have gleaned, it is but squeezing you,
and, sponge, you shall be dry again.
– Hamlet

One of the first things I implement in most Python projects is some quick stream logging so as not to dot too many print statements across the code. At some point file logging usually  follows as a feature so it can run by itself and I can check up on it from time to time.

Recently I got ambitious and added email logging to poca, my podcast client project. Which isn’t difficult as the logging module has an SMTPHandler built-in that you just stick on your logger and now your logs are airborne. It’s almost xkcd-like. The way it works though is that an email is sent off instantly every time something is logged, just like stream logging only over port 25. This was not what I wanted. To be specific I was looking for something that

  • Would gather up log entries as my script runs it course
  • At the end of the script it would evaluate if there was anything worthwhile sending
  • If so, send it; if not, save, don’t send

The way to do, as suggested by the logging module author himself, is to subclass the BufferingHandler which does a fine job of soaking up the entries and keeping them until its time to release them. He has written an example of how to do this himself and put it on Github.

I had, however, a further requirement. When a podcast feed fails, it’s not necessarily cause for alarm or notifying the user. Maybe the server’s in maintenance mode for a few minutes, maybe builders unplugged the cable for an hour, maybe you did. You only really need email logging to tell you when a podcast feed consistently fails because then it’s probably either a software failure or the feed is offline, either of which requires user intervention. Vinay Sajip’s script doesn’t allow for that. Also, using unicode is easier if we keep a clear distinction between body and headers which Sajip’s approach seems to fudge a bit.

My requirements meant that I would have to not just buffer entries in memory but save them over time and only release them when there was a sufficient number (and of sufficient severity) to worry about.


Let’s do Postfix again but slowly and properly this time – Part 1: A simple local mail receiving server

About a year ago I wrote a scathing comment about Ars Technica’s too-slow guide to self-hosted email. I wanted it now-now-now and not over the course of four weeks. Well, I guess it’s time to eat humble pie because my setup didn’t survive transfer to the new machine. Seeing as I hadn’t acquired any deep understanding of the setup apart from a purely abstract grasp of email fundamentals I didn’t have any way to fix it.

In short, I’m going to go in the complete opposite direction and take it slow this time. We are going to setup an MTA that will accept email and that we can use to send email. Only when we thoroughly understand that will we move on to Dovecot and the like. We are going to take it step-by-step, making sure that we understand what we’re doing and why we’re doing it and sometimes even what we’re not doing.

In the course of this first piece in a series, we will get some sort of email up and running – and get that warm feeling that we have accomplished something –  but consider it more of a proof-of-concept or a learning experience than anything like the final thing.


MUAMSAMTAMDAMAAAAAA! Understanding how email works so you can get on with setting up your own mail server

‘Email infrastructure’ is a word most people never want to hear. And with good reason. The elements and procedures involved in getting email from user A to user B are legion and complex. And that’s just the theory before you even start to get into the practical implementation side of things. However, understanding the theory will make a more practical how-to (like this one for Arch Linux) easier to follow.

This article will offer an introduction to that theory. There are plenty of ressources to draw on, so mostly this is simply a meta article with links to the best of the internet on the various sub-topics with a bit of filler where needed. The aim is not give an in-depth understanding of how, say, SMTP works. The aim is to provide the wanna-be master of his own email domain the minimum of theoretical grounding to accomplish that goal.

Note: This introduction assumes you know what terms like server, client, protocol, and ports mean. At least roughly. Ideally you have some experience setting up other services for network use, such as a web server.