So Long BMW, and Thanks for All the Fish!

| Comments

Seven years ago, I had just graduated from university and moved to Ulm to take a job at BMW Car IT to develop infotainment software for the (BMW) “cars of the future.” Last October, after four positions across three different teams, I have decided to move on. Time to recap.

Originally, I wanted to cover my entire career at BMW in this blog post. The first two and a half years would already fill a 2500 word essay, though, so I’ve decided to split these posts into a series of shorter posts. This first post covers my day-to-day work for the first two years in the software integration team.

The front of an office building with two BMW flags

A shift in how the automotive industry builds software

I joined right at the beginning of the development cycle for what would be the first infotainment system partly developed by BMW. The hardware part supplier—the so-called “Tier 1”—used to be fully responsible for development of the software of previous head unit systems, except for the BMW-themed user interface. BMW wanted to change this with the software targeting cars released in 2018. We can see the same transition taking place in other auto manufacturers, too: Daimler has Daimler TSS, which are now called “Mercedes-Benz Tech Innovation;” Audi founded; Volkswagen founded CARIAD SE in 2020; Tesla has always been doing their software in-house.

Backing Up Your Android Phone with borgbackup

(updated ) | Comments

Since I last switched phones, I no longer have root access to my Android. On my previous phone, I was using Titanium Backup, which—because it does require root access—is no longer an option. After I bought my current phone, I started using Swift Backup to avoid loosing my apps, messages, call logs and Wi-Fi networks.

That means that I do not have a backup solution in place for data: should my phone break, I would no longer have a copy of the downloads, pictures, and videos. I have been procrastinating a solution for this problem for about 2.5 years now. Time to stop.

What do I use on other systems?

On my workstations, laptops, and servers, I have been using borgbackup. It’s reasonably fast, has deduplication (which means I can run frequent backups), and a nice backup retention system. Contrary to some other tools, such as duplicity, there is no need to periodically create a fresh full backup to get yearly, monthly, and weekly backup retention policies. Also, backups are encrypted and authenticated. This is not super important for me since I run my own borgbackup server and thus do not have a strong trust boundary, but it feels like a good standard precaution to take.

However, borgbackup is not a native Android app, so I had initially discarded the possibility to also use it on my phone. Prematurely, as it turns out.

The Journey to Storing SMTP Passwords in a Database

| Comments

Back in the days when spam was not a thing, and the internet was simpler, if you wanted to give users an email address under your domain, you’d just add a forward to your mail server configuration. That took care of the receiving side, and sending could usually be done with whatever mail server people already had. Nobody bothered checking the envelope sender or From header anyway, and mail servers would happily accept mail from everyone and everywhere as long as it seemed that it had ended up in the right place. And it was good. And then along came spam.

SPF and DKIM? You need to run your own SMTP.

Now, of course, this is not a theoretical example. MacPorts has always provided its project members with an email alias under However, to fight spam, smart people came up with a multitude of ways to figure out whether mail received by a mail server was actually sent by who the envelope claimed to have sent it. There are currently two major mechanisms for this purpose: Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM).

SPF allows administrators to publish a list of servers that are permitted to send mail on behalf of a specific domain. Of course, since MacPorts did not actually provide an SMTP server and expected our developers to use their own ones, we had no way of gathering such a list and would thus allow the entire internet to send mail on behalf of, something more and more mail providers are nowadays treating as an indicator for spam.

DKIM, on the other hand, adds a cryptographic signature to certain selected fields of an email when it passes through the outgoing server, to be verified against a public key published in DNS on the receiving end. But again, since there was no single central SMTP serving, we could not ensure that all mails had such a signature, and thus could not enable DKIM – which providers are also using as an indication for spam.

We did know for a while that we would eventually have to setup email submission, but have been delaying the actual setup, since we needed a way to configure the passwords that should be used for SMTP. Since MacPorts’ migration to GitHub in October 2016, we only use GitHub’s OAuth2 for authentication. And while mail clients are slowly implementing support for that in SMTP and IMAP, it is not yet widespread enough to be usable in our case.

So, my todo list came down to

  • Write a web application that uses GitHub OAuth2 to authenticate users
  • Allow setting the SMTP password in a database from that web application. I figured a database would be a good idea, since it’s the most convenient resource to access from different unix users, unlike files and/or sockets where I would have had to configure groups.
  • Configure SMTP authentication against the passwords in the database into Postfix.

Sounds simple enough. Boy, was I wrong…

Attending the Google Summer of Code Mentor Summit

| Comments

From Friday, October 13th to Sunday, October 15th 2017 I had the opportunity to attend the Google Summer of Code Mentor Summit in Sunnyvale, CA. This is a summary of my experiences.

If you are not familiar with Google Summer of Code, it is a program where university students spend the summer working for an open source project. Google pays the students as a way to give back to open source, which is heavily used in their products. Students are mentored by experienced developers from the projects and Google invites two mentors per project into the US in autumn each year for an unconference-style summit.

Together with Mojca Miklavec I mentored Zero King, who did a great job at implementing better usability for GitHub pull requests for MacPorts by setting up Travis CI and a PR helper bot. Our original plan was to attend the Mentor Summit with 2015 student and this year organization administrator Jackson Isaac, but unfortunately his visa to the US was denied and Mojca stepped in instead.

Baidu Spider Caused More Than 80% of Our Trac's HTTP Traffic

| Comments

Since MacPorts’ move off Apple’s MacOSForge in October, we have been running MacPorts’ Trac installation on our own infrastructure. We used to rely on server and admin time generously donated by Apple. Now that we no longer enjoy this luxury, we are on our own when it comes to keeping our infrastructure running.

For a few months, we were bedeviled by high server load apparently caused by our Trac installation and had a hard time figuring out the cause. Our monitoring showed a large number of HTTP requests and Trac’s response time would regularly take a nose-dive as soon as the backup started.

After a few attempts of tuning various knobs without too much success, I finally decided to grab the Apache access logs and run awstats on them. Since we rotate our access logs biweekly I only had 10 days of February for analysis, but even those 10 days revealed some pretty interesting data.