Back in the days when spam was not a thing, and the internet was simpler, if
you wanted to give users an email address under your domain, you’d just add
a forward to your mail server configuration. That took care of the receiving
side, and sending could usually be done with whatever mail server people
already had. Nobody bothered checking the envelope sender or
anyway, and mail servers would happily accept mail from everyone and everywhere
as long as it seemed that it had ended up in the right place. And it was good.
And then along came spam.
SPF and DKIM? You need to run your own SMTP.
Now, of course, this is not a theoretical example. MacPorts has
always provided its project members with an email alias under
However, to fight spam, smart people came up with a multitude of ways to figure
out whether mail received by a mail server was actually sent by who the
envelope claimed to have sent it. There are currently two major mechanisms for
this purpose: Sender Policy Framework (SPF) and
DomainKeys Identified Mail (DKIM).
SPF allows administrators to publish a list of servers that are permitted to
send mail on behalf of a specific domain. Of course, since MacPorts did not
actually provide an SMTP server and expected our developers to use their own
ones, we had no way of gathering such a list and would thus allow the entire
internet to send mail on behalf of
@macports.org, something more and more
mail providers are nowadays treating as an indicator for spam.
DKIM, on the other hand, adds a cryptographic signature to certain selected
fields of an email when it passes through the outgoing server, to be verified
against a public key published in DNS on the receiving end. But again, since
there was no single central SMTP serving
@macports.org, we could not ensure
that all mails had such a signature, and thus could not enable DKIM – which
providers are also using as an indication for spam.
We did know for a while that we would eventually have to setup email submission, but have been delaying the actual setup, since we needed a way to configure the passwords that should be used for SMTP. Since MacPorts' migration to GitHub in October 2016, we only use GitHub’s OAuth2 for authentication. And while mail clients are slowly implementing support for that in SMTP and IMAP, it is not yet widespread enough to be usable in our case.
So, my todo list came down to
- Write a web application that uses GitHub OAuth2 to authenticate users
- Allow setting the SMTP password in a database from that web application. I figured a database would be a good idea, since it’s the most convenient resource to access from different unix users, unlike files and/or sockets where I would have had to configure groups.
- Configure SMTP authentication against the passwords in the database into Postfix.
Sounds simple enough. Boy, was I wrong…
SMTP authentication in Postfix: Cyrus or Dovecot SASL?
I decided to start at the bottom by getting SMTP authentication set up in our Postfix installation. Postfix uses the Simple Authentication and Security Layer (SASL) to authenticate SMTP users. There are two options for SASL daemons, Cyrus SASL, and Dovecot.
I spent a while looking at the two options. Both have options to talk to PostgreSQL, but one point in Cyrus' documentation did surprise me:
The Cyrus SASL library also supports some “shared secret” authentication methods: CRAM-MD5, DIGEST-MD5 and its successor SCRAM. These methods rely on the client and the server sharing a “secret”, usually a password. The server generates a challenge and the client a response proving that it knows the shared secret. This is much more secure than simply sending the secret over the wire proving that the client knows it.
There’s a downside: in order to verify such responses, the server must keep passwords or password equivalents in a database; if this database is compromised, it is the same as if all the passwords for the realm are compromised.
I understand that the secret is required in plain text for these challenge-response authentication mechanisms, but since I was only going to allow login on encrypted connections anyway, I really did not need that functionality, and it sounded like bad practice to store passwords in plain text.
So I kept looking around the documentation and the source code for a way to
store hashed passwords only.
Some documentation seems to suggest that this is possible by
pwcheck_method = auxprop-hashed, but there is no description
whatsoever for that option. I found the
commit that adds this feature, but also could not figure
out which hash function was used here.
I was honestly surprised to find no clear documentation of what I think is a reasonable feature request (or indication of whether this would work at all) in what is probably the most popular open source implementation of SASL.
In comparison, Dovecot’s authentication documentation has a
list of password schemes with up-to-date algorithms such as
Argon2. A clear advantage for dovecot, so that’s what I went with. On a debian
system, switching dovecot’s user lookup to a database is rather simple; in
/etc/dovecot/conf.d/10-auth.conf, I had to comment
auth-system.conf.ext and uncomment
auth-sql.conf.ext as follows:
1 2 3 4 5 6 7 8
This causes password lookups (
passdb) to use the SQL query configured in
/etc/dovecot/dovecot-sql.conf.ext, but does not run a separate query for the
user details such as the user’s home directory, UID, and GID. Since the virtual
users would not have a separate unix user on the machine anyway, there was
little point in setting this to anything but the postfix UID and GID.
dovecot-sql.conf.ext, I only set the
1 2 3 4 5 6
Making Postfix talk to Dovecot SASL
Fortunately, integrating Dovecot SASL into Postfix was surprisingly easy. There
are various articles on the internet that describe the setup, and postfix'
documentation can be used to decide which specific configuration values fit
your use case. I ended up with authentication on the submission service on port
587, by configuring
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Unfortunately, this configuration would allow authenticated users to send with
an abitrary envelope sender address, i.e. once authenticated as
firstname.lastname@example.org, I can send mails on behalf of
smtpd_sender_restrictions = reject_sender_login_mismatch
() to prevent this, but this feature requires
smtpd_sender_login_maps () to be set to a lookup table that
resolves SASL login names to a list of email addresses from which the given
user is authorized to send. And that specific information, is of course,
available in the database. So in addition to dovecot SASL talking to the
database for the password lookup, Postfix must query the same table to get the
list of permitted sender addresses.
sets up Postfix to query this information from the database. Note the added
proxy, which is required because smtpd runs in chroot and could otherwise not
talk to the database server. In
I again configured the appropriate database connection and query:
1 2 3
Of course this query is rather pointless, since it will only ever return its arguments. I couldn’t find a template-based method that would have allowed my to return the SASL login name (which is an email address in my case), and arguably, this method is more flexible in case of future changes to the database schema.
Out of the three steps outlined above, I have only covered the last one, configuring Postfix to use a PostgreSQL database containing hashed passwords with modern hashing algorithms for user authentication. Dovecot SASL provided comprehensive documentation that made this fairly simple, while Cyrus SASL was unfortunately lacking significantly in documentation.
One might think writing the web application to set the password and getting this to work with clients would be easier, to which I can only say: Never underestimate mail clients! But that’s a story for a different post.