Back in the days when spam was not a thing, and the internet was simpler, if you wanted to give users an email address under your domain, you’d just add a forward to your mail server configuration. That took care of the receiving side, and sending could usually be done with whatever mail server people already had. Nobody bothered checking the envelope sender or From header anyway, and mail servers would happily accept mail from everyone and everywhere as long as it seemed that it had ended up in the right place. And it was good. And then along came spam.

SPF and DKIM? You need to run your own SMTP.

Now, of course, this is not a theoretical example. MacPorts has always provided its project members with an email alias under @macports.org. However, to fight spam, smart people came up with a multitude of ways to figure out whether mail received by a mail server was actually sent by who the envelope claimed to have sent it. There are currently two major mechanisms for this purpose: Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM).

SPF allows administrators to publish a list of servers that are permitted to send mail on behalf of a specific domain. Of course, since MacPorts did not actually provide an SMTP server and expected our developers to use their own ones, we had no way of gathering such a list and would thus allow the entire internet to send mail on behalf of @macports.org, something more and more mail providers are nowadays treating as an indicator for spam.

DKIM, on the other hand, adds a cryptographic signature to certain selected fields of an email when it passes through the outgoing server, to be verified against a public key published in DNS on the receiving end. But again, since there was no single central SMTP serving @macports.org, we could not ensure that all mails had such a signature, and thus could not enable DKIM – which providers are also using as an indication for spam.

We did know for a while that we would eventually have to setup email submission, but have been delaying the actual setup, since we needed a way to configure the passwords that should be used for SMTP. Since MacPorts’ migration to GitHub in October 2016, we only use GitHub’s OAuth2 for authentication. And while mail clients are slowly implementing support for that in SMTP and IMAP, it is not yet widespread enough to be usable in our case.

So, my todo list came down to

• Write a web application that uses GitHub OAuth2 to authenticate users
• Allow setting the SMTP password in a database from that web application. I figured a database would be a good idea, since it’s the most convenient resource to access from different unix users, unlike files and/or sockets where I would have had to configure groups.
• Configure SMTP authentication against the passwords in the database into Postfix.

Sounds simple enough. Boy, was I wrong…

SMTP authentication in Postfix: Cyrus or Dovecot SASL?

I decided to start at the bottom by getting SMTP authentication set up in our Postfix installation. Postfix uses the Simple Authentication and Security Layer (SASL) to authenticate SMTP users. There are two options for SASL daemons, Cyrus SASL, and Dovecot.

Cyrus SASL

I spent a while looking at the two options. Both have options to talk to PostgreSQL, but one point in Cyrus’ documentation did surprise me:

The Cyrus SASL library also supports some “shared secret” authentication methods: CRAM-MD5, DIGEST-MD5 and its successor SCRAM. These methods rely on the client and the server sharing a “secret”, usually a password. The server generates a challenge and the client a response proving that it knows the shared secret. This is much more secure than simply sending the secret over the wire proving that the client knows it.

There’s a downside: in order to verify such responses, the server must keep passwords or password equivalents in a database; if this database is compromised, it is the same as if all the passwords for the realm are compromised.

I understand that the secret is required in plain text for these challenge-response authentication mechanisms, but since I was only going to allow login on encrypted connections anyway, I really did not need that functionality, and it sounded like bad practice to store passwords in plain text.

So I kept looking around the documentation and the source code for a way to store hashed passwords only. Some documentation seems to suggest that this is possible by setting pwcheck_method = auxprop-hashed, but there is no description whatsoever for that option. I found the commit that adds this feature, but also could not figure out which hash function was used here.

I was honestly surprised to find no clear documentation of what I think is a reasonable feature request (or indication of whether this would work at all) in what is probably the most popular open source implementation of SASL.

Dovecot SASL

In comparison, Dovecot’s authentication documentation has a list of password schemes with up-to-date algorithms such as Argon2. A clear advantage for dovecot, so that’s what I went with. On a debian system, switching dovecot’s user lookup to a database is rather simple; in /etc/dovecot/conf.d/10-auth.conf, I had to comment !include auth-system.conf.ext and uncomment !include auth-sql.conf.ext.

I configured auth-sql.conf.ext as follows:

passdb {
  driver = sql
  args = /etc/dovecot/dovecot-sql.conf.ext
}
userdb {
  driver = static
  args = uid=postfix gid=postfix home=/var/mail/%u
}

This causes password lookups (passdb) to use the SQL query configured in /etc/dovecot/dovecot-sql.conf.ext, but does not run a separate query for the user details such as the user’s home directory, UID, and GID. Since the virtual users would not have a separate unix user on the machine anyway, there was little point in setting this to anything but the postfix UID and GID.

In dovecot-sql.conf.ext, I only set the driver, connect, default_pass_scheme and password_query fields:

driver = pgsql
connect = postgresql:///smtpselfservice
default_pass_scheme = ARGON2ID
password_query = \
  SELECT username, domain, password \
  FROM users WHERE username = '%n' AND domain = '%d'

Making Postfix talk to Dovecot SASL

Fortunately, integrating Dovecot SASL into Postfix was surprisingly easy. There are various articles on the internet that describe the setup, and postfix’ documentation can be used to decide which specific configuration values fit your use case. I ended up with authentication on the submission service on port 587, by configuring /etc/postfix/master.cf:

submission inet n       -       -       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
   # enable SASL auth
  -o smtpd_sasl_auth_enable=yes
   # support some broken SASL clients
  -o broken_sasl_auth_clients=yes
   # use Dovecot SASL
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
   # No plaintext over unencrypted connections
  -o smtpd_sasl_security_options=noanonymous,noplaintext
   # No anonymous authentication
  -o smtpd_sasl_tls_security_options=noanonymous
   # Add a header with the login name to sent messages
  -o smtpd_sasl_authenticated_header=yes
  -o smtpd_reject_unlisted_recipient=no
   # Only allow authenticated users to send mail
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o smtpd_sender_restrictions=reject_non_fqdn_sender
  -o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

Unfortunately, this configuration would allow authenticated users to send with an abitrary envelope sender address, i.e. once authenticated as cal@macports.org, I can send mails on behalf of raimue@macports.org. Not ideal.

Postfix has smtpd_sender_restrictions = reject_sender_login_mismatch ([1]) to prevent this, but this feature requires smtpd_sender_login_maps ([2]) to be set to a lookup table that resolves SASL login names to a list of email addresses from which the given user is authorized to send. And that specific information, is of course, available in the database. So in addition to dovecot SASL talking to the database for the password lookup, Postfix must query the same table to get the list of permitted sender addresses.

smtpd_sender_login_maps = proxy:pgsql:/etc/postfix/smtpd_sender_login_maps.cf

sets up Postfix to query this information from the database. Note the added proxy, which is required because smtpd runs in chroot and could otherwise not talk to the database server. In /etc/postfix/smtpd_sender_login_maps.cf, I again configured the appropriate database connection and query:

hosts = postgresql:///smtpselfservice
dbname = smtpselfservice
query = SELECT (username || '@' || domain) FROM users WHERE username = '%u' and domain = '%d'

Of course this query is rather pointless, since it will only ever return its arguments. I couldn’t find a template-based method that would have allowed my to return the SASL login name (which is an email address in my case), and arguably, this method is more flexible in case of future changes to the database schema.

Conclusion

Out of the three steps outlined above, I have only covered the last one, configuring Postfix to use a PostgreSQL database containing hashed passwords with modern hashing algorithms for user authentication. Dovecot SASL provided comprehensive documentation that made this fairly simple, while Cyrus SASL was unfortunately lacking significantly in documentation.

One might think writing the web application to set the password and getting this to work with clients would be easier, to which I can only say: Never underestimate mail clients! But that’s a story for a different post.