neverpanic.deZola2023-03-25T11:16:59+00:00https://neverpanic.de/atom.xmlS/PDIF keepalive using mpv and ffmpeg2023-03-25T11:16:59+00:002023-03-25T11:16:59+00:00https://neverpanic.de/blog/2023/03/25/spdif-keepalive-using-mpv-and-ffmpeg/<p>For sound output at my desk, I connect <a href="https://www.caldigit.com/ts3-plus/">my dock</a> to a <a href="https://teufel.de/decoderstation-5-100161000">digital audio receiver</a> with a Toslink
optical audio cable using S/PDIF. In this setup, the connection between my laptop and the reciever stops if there is no
audio output for a while. This power saving measure is the road to hell that’s paved with good intentions: if audio
output resumes for a notification sound, it takes the receiver about half a second before it starts playing. This delay
cuts off the start of the sound.</p>
<div>
<img
class="center"
src="/processed_images/e2b7cb443d0a30ab00.jpg"
title="Teufel decoderstation 5 showing 'No Signal' on optical input 2"
alt="Teufel decoderstation 5 showing 'No Signal' on optical input 2"
width="790"
height="296"
/>
</div>
<p>To make matters worse, the <a href="https://teufel.de/concept-e-400-digital-51-set-100167000">amplifier in my subwoofer</a> also has a power saving feature and turns off if it
detects no output for a while.</p>
<p>To prevent this, I needed continuous sound output that would keep both the S/PDIF channel between my dock and my
receiver open as well as my amplifier convinced that it’s still playing audio. At the same time, that audio stream
should be as imperceptible as possible to me.</p>
<p>The open source <a href="https://ffmpeg.org/">ffmpeg library</a> contains a configurable noise generator that can solve this problem. The
<a href="https://ffmpeg.org/ffmpeg-filters.html#anoisesrc">anoisesrc</a> audio source provides different “colors” of noise, and the <a href="https://mpv.io/"><code>mpv(1)</code></a> player can play these
sources. A small script called <code>silence</code> in my <code>$PATH</code> now prevents my receiver and amplifier from going into power
saving:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;">!/bin/sh
</span><span style="color:#fffd87;">exec</span><span> mpv av://lavfi:anoisesrc=r=44100:a=0.01:c=violet
</span></code></pre>
<p>I settled on the “violet” noise color (<code>c=violet</code>), because I found it to be the one I would notice the least. The
<code>a=0.01</code> parameter reduces the volume of the noise signal, while <code>r=44100</code> sets the sample rate to 44.1 kHz.</p>
<p>For now, I start this script manually when I need it. I should probably automate starting it when my audio output is via
S/PDIF and I’m not trying to use it for encoded output such as Dolby Digital or DTS, but I won’t let good enough be the
enemy of perfect here.</p>
<div>
<img
class="center"
src="/processed_images/f6d5cf778c0983ec00.jpg"
title="Teufel decoderstation 5 showing '5.1 Stereo' on optical input 2"
alt="Teufel decoderstation 5 showing '5.1 Stereo' on optical input 2"
width="790"
height="292"
/>
</div>
Open Source in a Headunit2023-03-13T21:36:52+00:002023-03-13T21:36:52+00:00https://neverpanic.de/blog/2023/03/13/open-source-in-a-headunit/<h3 id="preamble">Preamble</h3>
<p>This is the third post in my <a href="/categories/so-long-bmw-series/">series on my experiences at BMW Car IT GmbH</a>. If you’ve missed the
previous posts, you can <a href="/categories/so-long-bmw-series/">find them in my archive</a>.</p>
<p>In this post, I’m covering the use of open source software in the MGU head unit project at BMW.</p>
<div>
<img
class="center"
src="/processed_images/981774a6695e76e600.jpg"
title="Open Source Software in cars—a model for success"
alt="An old BMW with a fully rusted body parked on a street."
width="790"
height="280"
/>
</div>
<h2 id="without-open-source-software-there-is-no-project">Without open source software, there is no project</h2>
<span class="pullquote-right" data-pullquote="Open source software helps companies to build more competetive products.">
<p>Modern car infotainment systems are no longer possible without <a href="https://en.wikipedia.org/wiki/Open-source_software">open source software</a>. Not because a company could
not build them as closed source at all, but because they could not do so in a cost effective way: using open source
software just saves money. The limited number of developers can then focus on features that are differentiating in the
market rather than reimplementing basic features. In short, open source software helps companies to build more
competitive products.</p>
<!-- Open source software helps companies to build more competetive products. -->
</span>
<p>As I had already mentioned in <a href="/blog/2022/05/04/so-long-bmw-and-thanks-for-all-the-fish/">my first post on the work I did at BMW Car IT GmbH</a> between
<a href="/blog/2014/08/07/goodbye-university/">2014</a> and 2021, BMW had gathered some 250 open source enthusiasts in Ulm and Munich to build an infotainment
system for its cars released after 2018. This system would be a Linux system, and most of the userland resembled
a modern Linux desktop distribution.</p>
<h2 id="to-distribute-or-not-to-distribute-that-is-the-question">To distribute, or not to distribute, that is the question</h2>
<p>This choice implied that BMW would respect the licenses of the various open source projects used in a typical Linux
distribution. Almost all open source licenses have restrictions on what users must do when distributing the code in
compiled or source code form. As a consequence, we had to figure out where BMW was “distributing.” The sale of a car
that contains this code is obviously a distribution, but there were other cases to consider: BMW Car IT GmbH is
a separate entity, so does passing a finished build to BMW AG constitute a distribution, even if BMW AG fully owns BMW
Car IT GmbH? Furthermore, a series of companies were developing software for this platform, and developer tools—such as
software development kits—were available in compiled form for download from our servers. That probably is
“distribution,” and we would have to obey the license conditions. But would the companies BMW was paying to develop
software for them sue over a license violation? Probably not.</p>
<span id="continue-reading"></span>
<p>With the question of where distribution happens out of the way, the obligations of the tens of different licenses were
up next. These are reasonably well understood for most common licenses, and generally fall into the two categories
“permissive” and “copyleft.”</p>
<h2 id="what-do-i-have-to-do-to-distribute-this">What do I have to do to distribute this?</h2>
<p>Permissive licenses are licenses that do not require distribution of the source code along compiled binaries. The BSD
<a href="https://opensource.org/license/bsd-2-clause/">[1]</a>, <a href="https://opensource.org/license/bsd-3-clause/">[2]</a> and <a href="https://opensource.org/license/mit/">MIT</a> licenses and <a href="https://opensource.org/license/apache-2-0/">Apache 2.0</a> are popular members of this
group. For those, it’s typically enough to include a copyright notice and the license text in the product
documentation<sup class="footnote-reference"><a href="#1">1</a></sup>.</p>
<p>Copyleft licenses instead require providing the source code for distributed binaries. Common representatives of this
group are the GNU licenses (<a href="https://opensource.org/license/gpl-2-0/">GPL 2.0</a>, <a href="https://opensource.org/license/gpl-3-0/">GPL 3.0</a>, <a href="https://opensource.org/license/lgpl-2-1/">LGPL 2.1</a>), the <a href="https://opensource.org/license/mpl-2-0/">Mozilla Public License
MPL 2.0</a>, or the <a href="https://opensource.org/license/cddl-1-0/">Common Development and Distribution License CDDL 1.0</a>. They form two subgroups
called “weak copyleft” and “strong copyleft” depending on whether the obligation to include source code also expands to
derivative works of a copyleft component. The GNU GPL licenses are strong copyleft, while the rest use weak copyleft.</p>
<p>Now, <em>I’m not a lawyer, and neither am I here to give legal advice</em>, so take the following lines with a grain of salt:
the common interpretation of the GPL is that linking against a library licensed under the GNU GPL means that you are
creating a derivative work. As a consequence, you should not link code you want to keep proprietary against anything
that is GPL-licensed.</p>
<h2 id="license-compliance-at-bmw">License compliance at BMW</h2>
<p>What does this open source license 101 mean for BMW and what risks does using open source software introduce? There were
two main areas of concern with the use of open source software:</p>
<ul>
<li>Unidentified uses of GPL-licensed (that is strong copyleft) software that would retroactively force the company to
publish proprietary source code.</li>
<li>Unintentional non-compliance with license terms, because a license was not or incorrectly identified, followed by
a court-granted injunction that would stop production and sale of new cars. This scenario (commonly named after
Patrick McHardy, a Linux kernel developer who gained fame with GPL copyright enforcement court cases
<a href="https://lwn.net/Articles/694890/">[1]</a>, <a href="https://lwn.net/Articles/752485/">[2]</a>, <a href="https://opensource.com/article/17/8/patrick-mchardy-and-copyright-profiteering">[3]</a>) was a particular nightmare because stopping
production is expensive.</li>
</ul>
<p>As a result, somebody needed to keep a close eye on what obligations the use of open source software brings with it.
This was already well-established at BMW. A huge handbook with a couple of hundred pages documented every known license,
the company lawyer’s interpretation, obligations, and rules for use in products and development environments. Every
project did have to produce a document showing its architecture, used open source libraries and tools and their license.
Other developers with experience in license compliance would review this for any of the risks outlined earlier.</p>
<p>Licenses that did not yet have an entry in the handbook, or did not get general approval from the lawyers would require
separate review in a council consisting of both developers and lawyers. This also is one of the first problems I saw:
a modern GNU/Linux system uses <strong>a lot of</strong> different licenses.</p>
<h3 id="you-get-a-license-you-get-a-license-everybody-gets-a-license">You get a license, you get a license, everybody gets a license!</h3>
<span class="pullquote-right" data-pullquote="One Monday morning in Munich, two corporate lawyers and three developers discuss whether "choosing to make
a Bunny unhappy" is a restriction.">
<p>The MIT license has a surprising number of variants. The Fedora Linux distribution has a <a href="https://fedoraproject.org/wiki/Licensing:MIT">wiki page</a>, that
lists no less than 35 of them, all with minor differences. Picture a group of lawyers and developers sitting in
a meeting room, discussing the minutiae of different choices of words and their impact on the meaning of the license,
and you’ll get a good impression of the pain this license proliferation inflicts on users. On top of this proliferation,
developers sometimes create their own variants of licenses without consulting a lawyer. For example, the <a href="https://github.com/g-truc/glm/blob/master/copying.txt">Happy Bunny
License</a> used by the OpenGL Mathematics project GLM is yet another MIT license where the author has added the
following paragraph:</p>
<blockquote>
<p>Restrictions:
By making use of the Software for military purposes, you choose to make a
Bunny unhappy.</p>
</blockquote>
<!-- One Monday morning in Munich, two corporate lawyers and three developers discuss whether "choosing to make
a Bunny unhappy" is a restriction. -->
<p>I do recall one Monday morning in a meeting room in Munich, where two corporate lawyers and three developers discuss for
30 minutes whether consumer cars sold to governments or the military are “use of the Software for military purposes,”
and if they were, whether “choos[ing] to make a Bunny unhappy” is a restriction. If you ever considered modifying an
existing license, I have to beg you for one thing: don’t.</p>
</span>
<h3 id="due-diligence">Due diligence</h3>
<p>The other big looming problem companies face when reusing open source software is metadata quality. For its MGU head
units, BMW used the <a href="https://www.yoctoproject.org/">Yocto project</a>. Yocto uses recipes to describe how to compile and package a software
component. These recipes contain a field for the license of said software component, but—as with all open source
software—there are no correctness guarantees on this data. This problem is not entirely theoretical either: in 2014,
I found an <a href="https://mailman.alsa-project.org/hyperkitty/list/alsa-devel@alsa-project.org/thread/EJQ2SU2J6WVOXPO6OHIQ5X65EA53DEWD/#EJQ2SU2J6WVOXPO6OHIQ5X65EA53DEWD">example of header file copied from the kernel (licensed under GPL 2.0) in the otherwise LGPL 2.1-licensed
alsa-libs</a>. A <a href="https://dl.acm.org/doi/pdf/10.1145/3571852">2022 publication</a> of the <a href="https://oss.cs.fau.de/">professorship for open source software in
Erlangen</a> shows that this is still a problem today and found copyleft licensed material in
a significant number of projects on GitHub that are claim they are under a permissive license:</p>
<blockquote>
<p>Observation 3</p>
<p>There is a significant number of in-code licenses with a different level of restrictiveness than the declared license.</p>
</blockquote>
<p>Various projects are making progress to tackle this issue: the Linux Foundation has developed the <a href="https://spdx.org/licenses/">SPDX standard</a>
to make licensing information machine-readable and adding it to project’s source code using the
<a href="https://spdx.dev/ids/"><code>SPDX-License-Identifier</code> header</a>. nexB publish <a href="https://github.com/nexB/scancode-toolkit">scancode-toolkit</a>, which is
a quick and easy way to verify license metadata.</p>
<p>To at least partially address some of these risks, BMW includes not just copyleft source code in its open source
archives, but also permissively licensed open source code, including any potential modifications made. This mitigates
the risk of misidentification of code licensed under a copyleft license as permissive. As a nice side effect, publishing
all open source code regardless of whether it’s required or not, is also more in the spirit of open source.</p>
<h2 id="where-are-those-license-texts-in-cars-though">Where are those license texts in cars, though?</h2>
<p>Now, nothing from that lengthy discussion matters, if your customers <a href="https://shkspr.mobi/blog/2016/02/bmw-are-sending-their-software-updates-unencrypted/">disect a software update</a>,
notice that it contains open source software, but cannot actually find the license texts and the written offer to
request the corresponding source code. Back in 2016, BMW Australia customer support did not know what to do with
a request for open source code and refused, which <a href="https://shkspr.mobi/blog/2016/03/bmw-and-the-gpl/">then ended up on the</a> <a href="https://news.ycombinator.com/item?id=11217019">Hacker News front
page</a>.</p>
<p>I was working on gathering the license texts for the MGU infotainment around that time, and since BMW’s PR department
did not seem interested in getting involved in a Hacker News discussion, I did something that probably blatantly
violated the company policy on dealing with social media. I sent an email to the author explaining where to find the
license texts and where to request the DVD with the source code.</p>
<p>Two weeks later, <a href="https://shkspr.mobi/blog/2016/03/bmw-are-complying-with-the-gpl/">BMW actually sent out the DVD for this car</a> and this blog post did receive
<a href="https://news.ycombinator.com/item?id=11384968">some attention on Hacker News</a> again, so while I may have been violating policy, I’d like to think
getting involved did at least generate <em>some</em> good PR.</p>
<p>Note that the DVD (pictured in the linked blog post) is a bog standard DVD+R without any print or label on it. If you’re
wondering whether BMW couldn’t have at least made an effort to make look a bit nicer: that’s a conscious decision to
prevent the OSS DVDs from becoming a collector’s item ;-)</p>
<div class="footnote-definition" id="1"><sup class="footnote-definition-label">1</sup>
<p>This is an oversimplification, but this is a blog post, not a seminal paper on the differences between various
specific open source licenses.</p>
</div>
When Your Inter-Process Communication Framework Meets the ABI2022-10-08T12:02:36+00:002022-10-08T12:02:36+00:00https://neverpanic.de/blog/2022/10/08/when-your-ipc-framework-meets-the-abi/<p>This is the second post in my <a href="/categories/so-long-bmw-series/">series on my experiences at BMW Car IT GmbH</a>. If you’ve missed the
first post, you can <a href="/blog/2022/05/04/so-long-bmw-and-thanks-for-all-the-fish/">go back and read it here</a>.</p>
<p>In this post, I’m covering missing binary compatibility and its effects on development processes.</p>
<h2 id="the-road-to-a-recompile-is-paved-with-abi-incompatibilities">The road to a recompile is paved with ABI incompatibilities</h2>
<p>To exchange data between processes in its infotainment systems, BMW uses <a href="https://covesa.github.io/capicxx-core-tools/">CommonAPI</a>. CommonAPI was
originally developed by the <a href="https://en.wikipedia.org/wiki/GENIVI_Alliance">GENIVI alliance</a>, which <a href="https://www.telematicswire.net/genivi-alliance-rebrands-as-connected-vehicle-systems-alliance/">rebranded itself</a> as the <a href="https://www.covesa.global/">Connected
Vehicle Systems Alliance COVESA</a> in 2021.</p>
<p>CommonAPI allows specification of interfaces in the <a href="https://franca.github.io/franca/">Franca Interface Definition Language FIDL</a> independent of the
message bus used to send and receive messages. It achieves this by generating C++ bindings from the interface
description. These bindings then use a transport backend (CommonAPI supports <a href="https://covesa.github.io/capicxx-dbus-tools/">DBus</a> and
<a href="https://covesa.github.io/capicxx-someip-tools/">Some/IP</a>) that’s also compiled from generated code.</p>
<span class="pullquote-right" data-pullquote="for every change in CommonAPI, recompilation of *all*
custom software was necessary">
<p>The CommonAPI project does not provide any guarantees on the stability of the <a href="https://en.wikipedia.org/wiki/Application_binary_interface">application binary interface</a> of its
generated code. That means that programs compiled against header files generated with an older version of CommonAPI’s
generators can not use bindings and transport backend libraries generated by a newer version. Because CommonAPI’s
generated code heavily uses C++ templates, this meant that for every change in CommonAPI, recompilation of <em>all</em>
custom software was necessary.</p>
</span>
<span id="continue-reading"></span>
<p>Recompiling on its own would already be a four to six hour ordeal on the 16 core 40 GiB RAM virtual machines used by the
CI system. Caching reduced these build times for normal builds, but when a component as low-level as the inter-process
communication framework changes, the cache would not contain any of the required results. To make matters worse, for
intellectual property and build time reasons, some development teams delivered precompiled binaries, which would require
a rebuild by somebody with access to the source code against the new CommonAPI version. This ended up being a major
coordination issue across development teams in the entire project every four to six weeks.</p>
<p>Contrary to the ABI stability guarantees, which CommonAPI does not give, it did guarantee that the wire protocol, that
is, the messages sent via DBus or Some/IP, would not change. As a consequence, old binaries would theoretically keep
working if they still had access to the old bindings and transport libraries. In practice, we could not use this,
because the filenames of the bindings and transport libraries would be the same when generated with the old and the new
version. Statically linking could have solved this, but disk space was precious and static binaries tend to be larger
than their dynamically linked counterparts, so we discarded this approach.</p>
<h2 id="how-does-the-open-source-world-do-it">How does the open source world do it?</h2>
<p>This problem is not new, and our solution of synchronizing a merge point across the entire project is only an option if
there is a central entity to coordinate that. My colleagues and I looked to the open source world for a solution, where
such a central entity does not exist.</p>
<p>Whenever the ABI of a library changes, open source projects increase a version number in the <a href="https://en.wikipedia.org/wiki/Soname">SONAME field</a> of
the library. By convention, the filename of a library contains the SONAME, so a changed SONAME also resolves the
filename conflict we were facing. This gave maintainers of binary components the chance to recompile against the new
updated CommonAPI libraries in their own time as long as we continued shipping the old libraries.</p>
<p>With symbolic links to pkg-config files and a bit of magic in CMake files which we generated for the CommonAPI interface
libraries, this process became almost transparent for developers: they’d get the current version from the software
development kit without changing their source code.</p>
<h2 id="hey-this-may-actually-be-nice-for-other-reasons-too">Hey, this may actually be nice for other reasons, too!</h2>
<p>As a positive side effect, changes in interface definitions now also caused the library SONAME to change, so teams could
write code that worked with both the old and the new versions of interfaces during a transition period, which allowed
them to decouple their development process from the large “big bang” synchronization points that everybody else had to
respect when CommonAPI interface updates arrived. Some teams even linked against both old and new versions of the
interface libraries to provide both services at the same time to allow their users to update to the newer API when they
were ready.</p>
<p>We implemented large parts of this mechanism in CMake in the build system for the CommonAPI binding and transport
libraries. This build system also generated the required pkg-config <code>.pc</code> files and CMake config files for the libraries
it built, with the necessary version information and the magic to make this transparent for users. This work has not
been open sourced yet—a recurring theme for some of the work I did. I don’t think there were any business concerns
against doing so. It’s rather a matter of putting the work in to do it, and I regret not prioritizing that.</p>
So Long BMW, and Thanks for All the Fish!2022-05-04T01:50:35+00:002022-05-04T01:50:35+00:00https://neverpanic.de/blog/2022/05/04/so-long-bmw-and-thanks-for-all-the-fish/<p><a href="/blog/2014/08/07/goodbye-university/">Seven years ago</a>, I had just graduated from university and moved to Ulm to take a job at BMW Car IT to develop
infotainment software for the (BMW) “cars of the future.” Last October, after four positions across three different
teams, I have decided to move on. Time to recap.</p>
<p>Originally, I wanted to cover my entire career at BMW in this blog post. The first two and a half years would already
fill a 2500 word essay, though, so I’ve decided to split these posts into a series of shorter posts. This first post
covers my day-to-day work for the first two years in the software integration team.</p>
<div>
<img
class="center"
src="/processed_images/fade5ac8139a54b300.jpg"
title="The Energon building, BMW Car IT's location in Ulm"
alt="The front of an office building with two BMW flags"
width="790"
height="398"
/>
</div>
<h2 id="a-shift-in-how-the-automotive-industry-builds-software">A shift in how the automotive industry builds software</h2>
<p>I joined right at the beginning of the development cycle for what would be the first infotainment system partly
developed by BMW. The hardware part supplier—the so-called <a href="https://www.amatechinc.com/resources/blog/returnable-packaging/tier-1-2-3-automotive-industry-supply-chain-explained">“Tier 1”</a>—used to be fully responsible for
development of the software of previous head unit systems, except for the BMW-themed user interface. BMW wanted to
change this with the software targeting cars released in 2018. We can see the same transition taking place in other auto
manufacturers, too: Daimler has <a href="https://www.mercedes-benz-techinnovation.com/en/">Daimler TSS</a>, which are now called “Mercedes-Benz Tech Innovation;” Audi founded
<a href="https://www.esolutions.de/">e.solutions</a>; Volkswagen founded <a href="https://cariad.technology/">CARIAD SE</a> in 2020; Tesla has always been doing their software
in-house.</p>
<span id="continue-reading"></span>
<p>Two major reasons motivated this change. First, developing the software from scratch is becoming both too complicated
and too expensive due to the size and feature set of modern infotainment systems. Second, the auto manufacturers now
consider their infotainment and user interface part of their value proposition and an avenue to new revenue streams.
Everybody wants to be the best in connectivity, offer the smoothest integrations with music and video streaming
services, or just ship the <a href="https://electrek.co/2021/01/19/how-to-make-your-tesla-fart/">funniest easter-eggs</a>. Most companies in this space are concerned that ignoring
the software is going to lose them customer experience to the tech giants—considering Android Auto’s success, not an
unreasonable assumption.</p>
<h2 id="take-some-250-linux-people-put-them-together-get-infotainment-in-return">Take some 250 Linux people, put them together, get infotainment in return</h2>
<span class="pullquote-right" data-pullquote="If you've used a recent desktop Linux, you'd probably find the root shell
of a BMW MGU infotainment system pretty familiar.">
<p>BMW needed its own infotainment system. To do this, they had hired a sizable team of young and eager Linux enthusiasts,
a significant chunk of which with prior experience from Nokia before they closed their site in Ulm. About 200 people in
Ulm plus another 60-70 in Munich worked for BMW Car IT GmbH, the Munich alternative to Daimler TSS, CARIAD, and
e.solutions. Most of my colleagues were Linux and open source enthusiasts, and that did show both in the pleasant
atmosphere as well as in the final product. If you’ve used a recent desktop Linux, you’d probably find the root shell
of a BMW MGU infotainment system pretty familiar. You’d encounter systemd, systemd journal, wayland, glibc, and
a pretty normal GNU userland – that is, except for bash and coreutils, of course, because it wouldn’t be an embedded
system with those, would it? I’ll leave the discussion of whether a quadcore CPU, 6 GiB RAM system would ever reasonably
constitute “embedded” as an exercise to the reader.</p>
</span>
<p>Learning to develop a car infotainment system based on Linux was an interesting challenge. There was a bit of a rivalry
between the Tier 1 and its developers and the revolutionary open source group in Ulm: they were often trying to do
things the way they had been done before, while we were trying to question everything and modernize both the product and
developer experience plus apply best practices—running various services as root, for example, was just no longer
acceptable for a product in 2018.</p>
<h2 id="the-build-failed-again">“The build failed again”</h2>
<p>Right from the start, we wanted compiling and automated testing handled by a continuous integration system. Fast
iterations, between two and three potentially shippable releases a day, and a system defined from a single git commit
hash were best practice in 2014, and we wouldn’t settle for less.</p>
<p>The balance between making sure the current development state is well tested enough to deliver and development speed was
not easy to find. Possibly the biggest lesson I learned is that test stability matters, a <strong>lot</strong>. A single bug that
fails the build or a test that toggles just 5 % of the time can ruin an otherwise flawless multi-hour build and test
run, and unless you have a strategy to deal with test instability and recover (such as re-running these tests, manually
waiving them after inspection, or disabling them), your developers are going to be both annoyed, and waste a lot of
time. Working hours spent on identifying and fixing such instabilities reach a positive return on investment fast.</p>
<p>One thing I think we did get right back then was serializing merge requests, testing them in-order and merging them when
they pass testing. This technique is now becoming more widespread among continuous integration systems. The Zuul CI
system, for example, calls this approach <a href="https://zuul-ci.org/docs/zuul/latest/gating.html">project gating</a>, and GitLab CI started supporting similar concepts.
Using gating did prevent cases where two conflicting changes would work separately, but together would have broken the
mainline, which in turn would have wasted hours of developers waiting for fixes.</p>
<h2 id="all-mail-clien-w-wbuild-tools-suck-this-one-just-sucks-less">All <a href="https://en.wikipedia.org/wiki/Mutt_(email_client)">mail clien</a>^W^Wbuild tools suck, this one just sucks less</h2>
<p>To build the rougly 3550 components of the infotainment system, we needed a tool that would allow us to compile and
assemble a Linux distribution. The system of choice for us was the <a href="https://www.yoctoproject.org/">yocto build-your-own-distribution project with its
build system bitbake</a>.</p>
<span class="pullquote-left" data-pullquote="the system whose limitations you know is probably a better choice over a system where you don't know how
it's going to fail">
<p>It worked reasonably well, but I’d be lying if I said there weren’t a significant number of pitfalls to be aware of and
knobs we had to adjust to keep performance from plummeting. Most of us developed a love-hate relationship with its
mixture of shell scripting and Python code in the same file, and its wonderfully simple to debug caching system. In
hindsight, the system whose limitations you know is probably a better choice over a system where you don’t know how
it’s going to fail. Or, as a colleague used to put it, when he switched to a team that uses Android: bitbake is the
worst build system, except for all the others.</p>
</span>
<h3 id="the-build-failed-again-again">“The build failed again” again</h3>
<p>As with tests, one of the major lessons learned was that non-determinism in the build process can end up costing your
developers a lot of time. Packagers were notoriously bad at declaring all their dependencies manually. They would often
forgot to declare dependencies, but the build would pass anyway. One of the builds bitbake ran in parallel added the
dependency to the build sysroot shared among these parallel builds. This worked until a commit changed the order of
parallel builds, the dependency vanished, and the build failed due to the missing dependency.</p>
<p>This particular problem is now fixed in recent releases, as <a href="https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#migration-2.3-recipe-specific-sysroots">recipe-specifc sysroots</a> replaced the shared build
sysroot. Undeclared dependencies are just no longer present in a component’s build environment. This problem was so
widespread that I <a href="https://github.com/neverpanic/poky/commit/323c05e2dad6ce244ca180847dc2e1d591bb20e4">developed an initial implementation of the solution</a> and invited upstream to weigh in on it.</p>
<h3 id="the-build-is-so-slow">“The build is so slow”</h3>
<span class="pullquote-right" data-pullquote="The
1808 virtual cores and five TB of RAM split into 119 virtual machines built around 800 system builds per week, up
to 25 of which became release candidates.">
<p>Build performance was another big area of concern. One of my former bosses has published numbers on the scale of our
system and continuous integration setup at a <a href="https://wiki.covesa.global/download/attachments/364316/GENIVI_AMM_2018_Achim_Demelt.pdf?version=1&modificationDate=1524099162000&api=v2">meeting of the GENIVI alliance in 2018</a>: we compiled 3568
components—most of them open source—and bundled them into 30 filesystem images for four hardware variants. The
1808 virtual cores and five TB of RAM split into 119 virtual machines built around 800 system builds per week, up
to 25 of which became release candidates. For each build, we ran 2426 tests, 1661 of them on hardware, the rest
virtualized.</p>
</span>
<h2 id="we-did-it">We did it!</h2>
<p>Despite the challenges and bugs we encountered, and stressful final weeks, I’m proud to say that in 2018, we delivered
a working product that does not have to hide when compared to the competition. It shipped with privilege separation,
secure boot, a good customer experience, and the second over-the-air software update for the whole car after Tesla. It
was an honor to be part of this journey.</p>
<div>
<img
class="center"
src="/processed_images/9e1a45360490db7500.jpg"
title="An over-the-air software upgrade in progress"
alt="Screen 'Remote Software Upgrade', warning that the vehicle is going to be unusable for 20 minutes during the upgrade."
width="790"
height="363"
/>
</div>
<p>In future posts in this series, I’m planning to cover various topics, such as:</p>
<ul>
<li><a href="/blog/2022/10/08/when-your-ipc-framework-meets-the-abi/">ABI compatibility and how the lack of it can ruin your development process</a></li>
<li><a href="/blog/2023/03/13/open-source-in-a-headunit/">Open source license compliance, convincing the lawyers, and some sneaky behind-the-scenes PR work for which I probably
would have gotten in trouble</a></li>
<li>Simplifying cross-compilation for developers using Linux namespaces</li>
<li>Over-the-air software updates and their pitfalls, and why retrying things until they succeed is sometimes a good
approach</li>
<li>My journey into the security team</li>
</ul>
<p>If you’re interested, <a href="/atom.xml">watch this space</a> or <a href="https://chaos.social/@neverpanic">follow me on Mastodon</a>.</p>
Backing Up Your Android Phone with borgbackup2022-01-25T20:18:35+00:002022-02-08T01:10:43+00:00https://neverpanic.de/blog/2022/01/25/backing-up-your-android-phone-with-borgbackup/<p>Since I last switched phones, I no longer have root access to my Android. On my previous phone, I was using <a href="https://play.google.com/store/apps/details?id=com.keramidas.TitaniumBackup">Titanium
Backup</a>, which—because it does require root access—is no longer an option. After I bought my <a href="https://www.oneplus.com/7pro">current
phone</a>, I started using <a href="https://play.google.com/store/apps/details?id=org.swiftapps.swiftbackup">Swift Backup</a> to avoid loosing my apps, messages, call logs and Wi-Fi
networks.</p>
<p>That means that I do not have a backup solution in place for data: should my phone break, I would no longer have a copy
of the downloads, pictures, and videos. I have been procrastinating a solution for this problem for about 2.5 years now.
Time to stop.</p>
<h2 id="what-do-i-use-on-other-systems">What do I use on other systems?</h2>
<p>On my workstations, laptops, and servers, I have been using <a href="https://www.borgbackup.org/">borgbackup</a>. It’s reasonably fast, has
deduplication (which means I can run frequent backups), and a nice backup retention system. Contrary to some other
tools, such as duplicity, there is no need to periodically create a fresh full backup to get yearly, monthly, and weekly
backup retention policies. Also, backups are encrypted and authenticated. This is not super important for me
since I run my own borgbackup server and thus do not have a strong trust boundary, but it feels like a good standard
precaution to take.</p>
<span class="pullquote-right" data-pullquote="borgbackup is not a native Android app, so I had initially discarded the possibility to also use it on my
phone.">
<p>However, borgbackup is not a native Android app, so I had initially discarded the possibility to also use it on my
phone. Prematurely, as it turns out.</p>
</span>
<span id="continue-reading"></span>
<p>A <a href="https://github.com/borgbackup/borg/issues/1155">ticket in the borgbackup repository on GitHuub</a> requests a port to Android. In the four years that the
ticket was open, somebody ended up packaging borgbackup for <a href="https://termux.com/">Termux</a>, an Android terminal emulator. This means
that, theoretically, you can run <code>pkg install borgbackup</code> in Termux and then run <code>borg</code> in Termux to create your backup.
Due to some of the features and limitations of the Android operating system, things are not quite as simple as that.</p>
<h2 id="borgbackuping-android">Borgbackuping Android</h2>
<p>Andreas Erhard has written a <a href="https://www.technik-zeilen.de/blog/android-geraete-mit-borgbackup-sichern">German blog post</a> on running borgbackup on Android in December that
I initially followed. When I tried to replicate it, there were some setup steps missing, without which the backup does
not work. I am writing my post to include those and document the changes I made.</p>
<h3 id="setup">Setup</h3>
<p>I was about to run commands and write scripts on my Android phone, and I did not want to type them on the phone’s
keyboard. The <a href="https://github.com/Genymobile/scrcpy">scrcpy</a> project can forward the phone’s screen to a workstation if you have access using the
<a href="https://developer.android.com/studio/command-line/adb">Android Debug Bridge adb</a>.</p>
<h4 id="installing-packages">Installing packages</h4>
<p>I started by installing the required packages. First, the Termux Android app (which I used <a href="https://f-droid.org/en/packages/com.termux/">from F-Droid</a>
because the release in the Google Play store comes with an outdated package repository configuration and thus does not
work out of the box). I also installed <a href="https://f-droid.org/en/packages/com.termux.api/">Termux:API</a> which contains <a href="https://wiki.termux.com/wiki/Termux:API">helper utilities</a> to
allow scripts to determine, for example, whether the phone is charging.</p>
<p>Then I opened Termux and installed the rest of the required packages:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>pkg upgrade
</span><span>pkg install openssh </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> for ssh-keygen
</span><span>pkg install termux-api </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> for termux-battery-status and other termux-* commands
</span><span>pkg install vim </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> because I used vim to write the backup script
</span><span>pkg install borgbackup
</span></code></pre>
<h4 id="generating-an-ssh-key">Generating an SSH key</h4>
<p>I use SSH to access my borgbackup server, so I created an SSH key pair on my phone. Ed25519 is a good choice as
cryptosystem for SSH keys, so I used that:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>ssh-keygen -t ed25519
</span></code></pre>
<div>
<img
class="center"
src="/processed_images/4dde72a46a239e4900.png"
title="Generating an SSH key on my phone using scrcpy"
alt="A Termux terminal screen showing output of ssh-keygen -t ed25519, mirrored to a workstation using scrcpy"
width="790"
height="413"
/>
</div>
<p>The generated public key then goes into <code>~/.ssh/authorized_keys</code> on my backup server.</p>
<h4 id="getting-the-required-android-permissions">Getting the required Android permissions</h4>
<p>Android’s permission system does not give applications access to storage by default. Termux comes with a helper script
to request that. Running <code>termux-setup-storage</code> requests this permission—which you need to allow in the resulting
pop-up.</p>
<div>
<img
class="center"
src="/processed_images/be9789417a70eebf00.png"
title="Requesting the Files and Media permission"
alt="An Android Files and Media permission prompt"
width="790"
height="413"
/>
</div>
<p>I am using <code>termux-wake-lock</code> and <code>termux-wake-unlock</code> to prevent the phone from sleeping while the backup is running.
On the first run, this also prompts for approval, so you should run the pair manually once:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>termux-wake-lock </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> prompts for permission
</span><span>termux-wake-unlock </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> release the lock again, we only need it while the backup is running
</span></code></pre>
<h4 id="choosing-a-passphrase-and-creating-the-backup-repository">Choosing a passphrase and creating the backup repository</h4>
<p>I encrypt my backups using a passphrase. In this case, I used <a href="https://github.com/redacted/XKCD-password-generator">xkcdpass</a> to generate one. You must select the
encryption type when creating the borg repository. The creation operation prompts for the passphrase.</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>HOST</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup.location.example.com</span><span style="color:#d6d6d680;">"
</span><span>HOSTNAME</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">cplus7pro</span><span style="color:#d6d6d680;">" </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> the gethostname(3) function returns localhost, we cannot rely on it
</span><span>TARGET</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">user@</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOST}:/mnt/backup/</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}</span><span style="color:#d6d6d680;">"
</span><span>
</span><span>borg init --encryption repokey </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">TARGET</span><span style="color:#d6d6d680;">"
</span></code></pre>
<p>If you want an append-only repository, you should pass <code>--append-only</code> here. Depending on your phone’s architecture,
<code>--encryption repokey-blake2</code> may be faster than the default of SHA256. See the output of <code>borg init --help</code> for more
guidance.</p>
<h3 id="writing-the-backup-script">Writing the backup script</h3>
<p>Once the required setup steps are complete, you can write the backup script. I am including the script I use below with
comments. Find the entire script at the end of this section.</p>
<h4 id="shebang">Shebang</h4>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;">!/data/data/com.termux/files/usr/bin/bash
</span></code></pre>
<p>The shebang line needs adjusting to work with Termux. You can use the <code>termux-fix-shebang</code> utility to do this after you
wrote a normal <code>#!/bin/bash</code> shebang. I use bash, rather than sh, because I am using arrays.</p>
<h4 id="declarations">Declarations</h4>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>HOST</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup.location.example.com</span><span style="color:#d6d6d680;">"
</span><span>HOSTNAME</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">cplus7pro</span><span style="color:#d6d6d680;">"
</span><span>TARGET</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">user@</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOST}:/mnt/backup/</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}</span><span style="color:#d6d6d680;">"
</span><span>
</span><span style="color:#fffb9d;">export </span><span>BORG_PASSPHRASE</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;"><your-chosen-passphrase></span><span style="color:#d6d6d680;">"
</span><span>
</span><span style="color:#fffb9d;">declare </span><span>-a TERMUX_NOTIFICATIONS</span><span style="color:#ececec;">=</span><span>()
</span><span>TERMUX_NOTIFICATION_ID</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup-</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}</span><span style="color:#d6d6d680;">"
</span><span>
</span><span style="color:#fffd87;">set </span><span>-o pipefail
</span></code></pre>
<p>First, I declare the backup target. I keep the <code>$HOST</code> variable separately because an invocation of <code>ping</code> uses it
before attempting the backup to determine whether the server is available. The <code>gethostname(3)</code> function returns
“localhost” in Termux, so rather than using the <code>{hostname}</code> format pattern when creating new backups (as I do on other
systems), I have to define the name of my system manually.</p>
<p>Exporting the <code>BORG_PASSPHRASE</code> environment variable causes borg to no longer ask for the passphrase when performing
backup operations.</p>
<p>The <code>TERMUX_NOTIFICATIONS</code> and <code>TERMUX_NOTIFICATION_ID</code> variables group and provide tracking for clean up of
notifications that my backup script sends using the <code>termux-notification</code> utility.</p>
<p>Finally, <code>set -o pipefail</code> makes the entire command fail if one of the commands in a pipe fails.</p>
<h4 id="notification-functions-cleanup-notify">Notification functions: <code>cleanup</code> & <code>notify</code></h4>
<p>Next, I define two functions: one to send Android notifications and another one for cleanup at the end of the script.</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#fffd87;">cleanup</span><span>() {
</span><span> </span><span style="color:#fed6af;">for</span><span> notification </span><span style="color:#fed6af;">in </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{TERMUX_NOTIFICATIONS[@]}</span><span style="color:#d6d6d680;">"</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">do
</span><span> termux-notification-remove </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">notification</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fed6af;">done
</span><span> termux-wake-unlock
</span><span>}
</span></code></pre>
<p>The script appends IDs for notifications that I do not want to keep after the end of the script to
<code>TERMUX_NOTIFICATIONS</code>. When <code>cleanup</code> runs, it removes any notifications in this variable. Any wakelocks that the
script might have held are also released. This function is going to be registered as trap exit handler using <code>trap "cleanup" EXIT</code>.</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Send a notification to the user.
</span><span style="color:#a0cfa1;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Usage: echo "message" | notify persist identifier [options...]
</span><span style="color:#a0cfa1;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> If persist is 0, the notification will be removed when the script exits.
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Otherwise, it will be kept (e.g. for warning or error messages).
</span><span style="color:#a0cfa1;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> The identifier can be used to overwrite a previous notification with the same
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> identifier. This can be useful for progress messages.
</span><span style="color:#a0cfa1;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Further options are those supported by termux-notification. The message must
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> be passed on stdin.
</span><span style="color:#fffd87;">notify</span><span>() {
</span><span> </span><span style="color:#fffb9d;">local </span><span>persist</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">1
</span><span> </span><span style="color:#fffd87;">shift
</span><span> </span><span style="color:#fffb9d;">local </span><span>id</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">1</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">shift
</span><span> </span><span style="color:#fffb9d;">local </span><span>-a args</span><span style="color:#ececec;">=</span><span>(</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">--group</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{TERMUX_NOTIFICATION_ID}</span><span style="color:#d6d6d680;">" "</span><span style="color:#d68686;">--id</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">id</span><span style="color:#d6d6d680;">"</span><span>)
</span><span>
</span><span> </span><span style="color:#fed6af;">if </span><span>termux-notification </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{args[@]}</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">persist</span><span style="color:#d6d6d680;">" </span><span>-eq 0 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> TERMUX_NOTIFICATIONS</span><span style="color:#ececec;">+=</span><span>(</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">id</span><span style="color:#d6d6d680;">"</span><span>)
</span><span> </span><span style="color:#fed6af;">fi
</span><span> </span><span style="color:#fed6af;">fi
</span><span>}
</span></code></pre>
<p>A helper function that sends Android notifications using <a href="https://wiki.termux.com/wiki/Termux-notification">termux-notification</a>. Depending on the
<code>persist</code> parameter, the notifications are automatically removed when <code>cleanup</code> runs.</p>
<h4 id="logging-helpers">Logging helpers</h4>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#fffd87;">msg</span><span>() {
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">***</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"
</span><span>}
</span><span>
</span><span style="color:#fffd87;">info</span><span>() {
</span><span> msg </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">INFO:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"
</span><span> termux-toast -s </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">*</span><span style="color:#d6d6d680;">"
</span><span>}
</span><span>
</span><span style="color:#fffd87;">warn</span><span>() {
</span><span> msg </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">WARN:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Warning:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">| </span><span>\
</span><span> notify 1 failure \
</span><span> --title </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> --alert-once \
</span><span> --priority low
</span><span>}
</span><span>
</span><span style="color:#fffd87;">err</span><span>() {
</span><span> msg </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">ERROR:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Error:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">| </span><span>\
</span><span> notify 1 failure \
</span><span> --title </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> --alert-once \
</span><span> --priority high
</span><span> </span><span style="color:#fffd87;">exit</span><span> 1
</span><span>}
</span></code></pre>
<p>More helper functions deal with status messages by log level. A toast (emitted using <code>termux-toast</code>) displays
informational messages. Warnings and errors become persistent notifications and are available after the exit of the
script.</p>
<h4 id="checking-whether-the-backup-can-be-performed-prepare">Checking whether the backup can be performed: <code>prepare</code></h4>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#fffd87;">prepare</span><span>() {
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>termux-battery-status </span><span style="color:#ececec;">| </span><span>grep </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">status</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">| </span><span>grep -qE </span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">"(CHARGING|FULL)"</span><span style="color:#d6d6d680;">'</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> warn </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Not charging, not performing backup</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fed6af;">return</span><span> 1
</span><span> </span><span style="color:#fed6af;">fi
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>termux-wifi-connectioninfo </span><span style="color:#ececec;">| </span><span>grep </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">supplicant_state</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">| </span><span>grep -q </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">COMPLETED</span><span style="color:#d6d6d680;">"</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> warn </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">WiFi not connected, not performing backup</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fed6af;">return</span><span> 1
</span><span> </span><span style="color:#fed6af;">fi
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>ping -w 10 -c 3 </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">HOST</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">></span><span>/dev/null</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> warn </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Failed to ping target </span><span style="color:#fed6af;">$</span><span style="color:#d68686;">HOST</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fed6af;">return</span><span> 1
</span><span> </span><span style="color:#fed6af;">fi
</span><span>}
</span></code></pre>
<p>This function checks whether a backup should run and is called before the backup. Using the <code>termux-battery-status</code> and
<code>termux-wifi-connectioninfo</code> utilities it verifies that the phone is either charging or full, and that it’s connected to
Wi-Fi. The output of <code>termux-wifi-connectioninfo</code> has fields for the Wi-Fi SSID, but those end up hidden for me, probably
because Android now requires permissions (quite possibly the location permissions) before it tells you the name of the
Wi-Fi to which you are connected.</p>
<p>At the end of this function, I run a ping test to my backup host. If it does not answer, it’s either down and the backup
would fail anyway, or the phone is not in the correct network.</p>
<h4 id="backing-up-files">Backing up files</h4>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#fffd87;">backup</span><span>() {
</span><span> </span><span style="color:#fffb9d;">local </span><span>-a flags</span><span style="color:#ececec;">=</span><span>()
</span><span>
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> enable interactive output
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span>-t 0 </span><span style="color:#fffd87;">] </span><span style="color:#ececec;">&& </span><span style="color:#fffd87;">[ </span><span>-t 1 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> flags</span><span style="color:#ececec;">+=</span><span>(</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">--stats</span><span style="color:#d6d6d680;">' '</span><span style="color:#d68686;">--progress</span><span style="color:#d6d6d680;">' '</span><span style="color:#d68686;">--list</span><span style="color:#d6d6d680;">'</span><span>)
</span><span> </span><span style="color:#fed6af;">fi
</span><span>
</span><span> info </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Starting backup</span><span style="color:#d6d6d680;">"
</span><span> ionice -c 3 \
</span><span> nice -n20 \
</span><span> borg create \
</span><span> --noatime \
</span><span> --compression</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">lz4</span><span style="color:#d6d6d680;">' </span><span>\
</span><span> --exclude-caches \
</span><span> --exclude</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">fm:/storage/emulated/0/*/.thumbnails</span><span style="color:#d6d6d680;">' </span><span>\
</span><span> --exclude</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">pp:/storage/emulated/0/Android/data</span><span style="color:#d6d6d680;">' </span><span>\
</span><span> --exclude</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">pp:/storage/emulated/0/Android/obb</span><span style="color:#d6d6d680;">' </span><span>\
</span><span> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{flags[@]}</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{TARGET}::</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}-{utcnow:%Y-%m-%dT%H:%M:%S}</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> /storage/emulated/0/ \
</span><span> /data/data/com.termux/files/home
</span><span>}
</span></code></pre>
<p>This function creates a new backup snapshot. I used the <code>-t</code> test for the stdin and stdout file descriptors to detect
whether a user ran the script interactively to increase debug output. If you do not want massive amounts of output while
testing, you may want to remove this, or run the script with stdin connected to <code>/dev/null</code>.</p>
<p>I backup the entire SD card, which Android mounts at <code>/storage/emulated/0</code>. <code>termux-setup-storage</code> creates symbolic
links in <code>~/storage</code> that point to specific subfolders of this path, but I want all data to be part of the backup, not
just these subfolders. I added the location of the script itself and all its configuration in
<code>/data/data/com.termux/files/home</code>.</p>
<p>My photo gallery stores generated thumbnails in folders named <code>.thumbnails</code>, which I do not want in the backups. The
<code>Android/data</code> and <code>Android/obb</code> subfolders are not accessible for Termux, so I exclude them as well.</p>
<h4 id="pruning-old-backups">Pruning old backups</h4>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#fffd87;">prune</span><span>() {
</span><span> </span><span style="color:#fffb9d;">local </span><span>-a flags</span><span style="color:#ececec;">=</span><span>()
</span><span>
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> enable interactive output
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span>-t 0 </span><span style="color:#fffd87;">] </span><span style="color:#ececec;">&& </span><span style="color:#fffd87;">[ </span><span>-t 1 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> flags</span><span style="color:#ececec;">+=</span><span>(</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">--stats</span><span style="color:#d6d6d680;">' '</span><span style="color:#d68686;">--list</span><span style="color:#d6d6d680;">'</span><span>)
</span><span> </span><span style="color:#fed6af;">fi
</span><span>
</span><span> info </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Pruning old backups...</span><span style="color:#d6d6d680;">"
</span><span> borg prune \
</span><span> --prefix</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}-</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> --keep-within</span><span style="color:#ececec;">=</span><span>14d \
</span><span> --keep-daily</span><span style="color:#ececec;">=</span><span>31 \
</span><span> --keep-weekly</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span>((</span><span style="font-weight:bold;color:#87d6d5;">6 </span><span style="color:#ececec;">* </span><span style="font-weight:bold;color:#87d6d5;">4</span><span>)) \
</span><span> --keep-monthly</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span>(( </span><span style="font-weight:bold;color:#87d6d5;">2 </span><span style="color:#ececec;">* </span><span style="font-weight:bold;color:#87d6d5;">12 </span><span>)) \
</span><span> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{flags[@]}</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{TARGET}</span><span style="color:#d6d6d680;">"
</span><span>}
</span></code></pre>
<p>As the last of the functions, this removes old backups that are no longer required. This is also where you could
configure your personal preferences for retention. In my case, I keep all backups in the last two weeks, one daily
backup for 31 days, weekly backups for about 6 months, and monthly backups for two years.</p>
<h4 id="getting-the-scheduling-right">Getting the scheduling right</h4>
<p>On Android, there is no cron daemon that you could use to invoke this script automatically. You could install a cron
daemon in Termux (and I have read articles online where people have done that), but I remain sceptical whether Android’s
power management features would keep the cron daemon in the Termux running.</p>
<p>Instead, Termux comes with <code>termux-job-scheduler</code>, which can run periodic jobs. <code>termux-job-scheduler</code>’s help output
tells us about its features:</p>
<pre style="background-color:#393939;color:#dedede;"><code><span>Usage: termux-job-scheduler [options]
</span><span>Schedule a script to run at specified intervals.
</span><span> -p/--pending list pending jobs and exit
</span><span> --cancel-all cancel all pending jobs and exit
</span><span> --cancel cancel given job-id and exit
</span><span>Options for scheduling:
</span><span> -s/--script path path to the script to be called
</span><span> --job-id int job id (will overwrite any previous job with the same id)
</span><span> --period-ms int schedule job approximately every period-ms milliseconds (default 0 means once).
</span><span> Note that since Android N, the minimum period is 900,000ms (15 minutes).
</span><span> --network text run only when this type of network available (default any): any|unmetered|cellular|not_roaming|none
</span><span> --battery-not-low boolean run only when battery is not low, default true (at least Android O)
</span><span> --storage-not-low boolean run only when storage is not low, default false (at least Android O)
</span><span> --charging boolean run only when charging, default false
</span><span> --persisted boolean should the job survive reboots, default false
</span><span> --trigger-content-uri text (at least Android N)
</span><span> --trigger-content-flag int default 1, (at least Android N)
</span></code></pre>
<p>Note that while it can theoretically run a job daily, it can not run it at a specific time of day, for example, each
night at 4 AM, where my phone is probably charging and has Wi-Fi.</p>
<p>As a workaround, I run the script every 30 minutes, but add checks to the script itself so that (a) it does not run
before 4 AM, and (b) it runs once per day. As a nice side effect, the phone re-attempts a backup every 30 minutes in
case the conditions were not fulfilled at 4 AM:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Run once per day, unless BORGBACKUP_FORCE=1
</span><span>MARKER_FILE</span><span style="color:#ececec;">=</span><span style="color:#d68686;">~/.borgbackup-</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}-</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">(date +</span><span style="color:#fed6af;">%</span><span style="color:#d68686;">Y-</span><span style="color:#fed6af;">%</span><span style="color:#d68686;">m-</span><span style="color:#fed6af;">%</span><span style="color:#d68686;">d)</span><span style="color:#d6d6d680;">"
</span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{BORGBACKUP_FORCE</span><span style="color:#ececec;">:-</span><span style="color:#d68686;">0}</span><span style="color:#d6d6d680;">" </span><span>-eq 0 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">(date +</span><span style="color:#fed6af;">%</span><span style="color:#d68686;">H)</span><span style="color:#d6d6d680;">" </span><span>-lt 4 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Backup not yet due, waiting...</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">exit</span><span> 0
</span><span> </span><span style="color:#fed6af;">elif </span><span style="color:#fffd87;">[ </span><span>-f </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">MARKER_FILE</span><span style="color:#d6d6d680;">" </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Backup already ran today</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">exit</span><span> 0
</span><span> </span><span style="color:#fed6af;">fi
</span><span style="color:#fed6af;">fi
</span><span>
</span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>prepare</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> info </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Server connectivity or charging status does not meet expectations, skipping backup.</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">exit</span><span> 1
</span><span style="color:#fed6af;">fi
</span><span>rm -f ~/.borgbackup-</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}</span><span style="color:#d6d6d680;">"</span><span>-</span><span style="color:#ececec;">*
</span><span>touch </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">MARKER_FILE</span><span style="color:#d6d6d680;">"
</span></code></pre>
<h4 id="grabbing-the-wakelock-and-running-the-backup">Grabbing the wakelock and running the backup</h4>
<p>What remains is invoking the defined functions in the right order. Registering the <code>cleanup</code> function as an exit handler
ensures that any non-persistent notifications sent get cleaned up, and any held wakelocks are released when the script
exits, whether it succeeded or failed.</p>
<p>Then, I grab a wakelock using <code>termux-wake-lock</code> to prevent Android from killing my process while it’s performing
a backup, display a notification that the backup is running (which is not persistent and is going to be removed when the
script exits), and then run backup and prune.</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#fffd87;">trap </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">cleanup</span><span style="color:#d6d6d680;">"</span><span> EXIT
</span><span>termux-wake-lock
</span><span>notify 0 progress \
</span><span> --alert-once \
</span><span> --ongoing \
</span><span> --priority low \
</span><span> --title </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> --content </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Running backup for </span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}</span><span style="color:#d6d6d680;">"
</span><span>
</span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>backup</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> err </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Backup failed, aborting!</span><span style="color:#d6d6d680;">"
</span><span style="color:#fed6af;">fi
</span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>prune</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> warn </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Pruning failed. Continuing anyway.</span><span style="color:#d6d6d680;">"
</span><span style="color:#fed6af;">fi
</span><span>
</span><span>info </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Backup finished successfully</span><span style="color:#d6d6d680;">"
</span></code></pre>
<h4 id="the-full-script">The full script</h4>
<p>As promised, this section contains the full script without interruptions:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;">!/data/data/com.termux/files/usr/bin/bash
</span><span>
</span><span>HOST</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup.location.example.com</span><span style="color:#d6d6d680;">"
</span><span>HOSTNAME</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">cplus7pro</span><span style="color:#d6d6d680;">"
</span><span>TARGET</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">user@</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOST}:/mnt/backup/</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}</span><span style="color:#d6d6d680;">"
</span><span>
</span><span style="color:#fffb9d;">export </span><span>BORG_PASSPHRASE</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;"><your-chosen-passphrase></span><span style="color:#d6d6d680;">"
</span><span>
</span><span style="color:#fffb9d;">declare </span><span>-a TERMUX_NOTIFICATIONS</span><span style="color:#ececec;">=</span><span>()
</span><span>TERMUX_NOTIFICATION_ID</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup-</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}</span><span style="color:#d6d6d680;">"
</span><span>
</span><span style="color:#fffd87;">set </span><span>-o pipefail
</span><span>
</span><span style="color:#fffd87;">cleanup</span><span>() {
</span><span> </span><span style="color:#fed6af;">for</span><span> notification </span><span style="color:#fed6af;">in </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{TERMUX_NOTIFICATIONS[@]}</span><span style="color:#d6d6d680;">"</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">do
</span><span> termux-notification-remove </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">notification</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fed6af;">done
</span><span> termux-wake-unlock
</span><span>}
</span><span>
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Send a notification to the user.
</span><span style="color:#a0cfa1;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Usage: echo "message" | notify persist identifier [options...]
</span><span style="color:#a0cfa1;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> If persist is 0, the notification will be removed when the script exits.
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Otherwise, it will be kept (e.g. for warning or error messages).
</span><span style="color:#a0cfa1;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> The identifier can be used to overwrite a previous notification with the same
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> identifier. This can be useful for progress messages.
</span><span style="color:#a0cfa1;">#
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Further options are those supported by termux-notification. The message must
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> be passed on stdin.
</span><span style="color:#fffd87;">notify</span><span>() {
</span><span> </span><span style="color:#fffb9d;">local </span><span>persist</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">1
</span><span> </span><span style="color:#fffd87;">shift
</span><span> </span><span style="color:#fffb9d;">local </span><span>id</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">1</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">shift
</span><span> </span><span style="color:#fffb9d;">local </span><span>-a args</span><span style="color:#ececec;">=</span><span>(</span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">--group</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{TERMUX_NOTIFICATION_ID}</span><span style="color:#d6d6d680;">" "</span><span style="color:#d68686;">--id</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">id</span><span style="color:#d6d6d680;">"</span><span>)
</span><span>
</span><span> </span><span style="color:#fed6af;">if </span><span>termux-notification </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{args[@]}</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">persist</span><span style="color:#d6d6d680;">" </span><span>-eq 0 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> TERMUX_NOTIFICATIONS</span><span style="color:#ececec;">+=</span><span>(</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">id</span><span style="color:#d6d6d680;">"</span><span>)
</span><span> </span><span style="color:#fed6af;">fi
</span><span> </span><span style="color:#fed6af;">fi
</span><span>}
</span><span>
</span><span style="color:#fffd87;">msg</span><span>() {
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">***</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"
</span><span>}
</span><span>
</span><span style="color:#fffd87;">info</span><span>() {
</span><span> msg </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">INFO:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"
</span><span> termux-toast -s </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">*</span><span style="color:#d6d6d680;">"
</span><span>}
</span><span>
</span><span style="color:#fffd87;">warn</span><span>() {
</span><span> msg </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">WARN:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Warning:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">| </span><span>\
</span><span> notify 1 failure \
</span><span> --title </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> --alert-once \
</span><span> --priority low
</span><span>}
</span><span>
</span><span style="color:#fffd87;">err</span><span>() {
</span><span> msg </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">ERROR:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Error:</span><span style="color:#d6d6d680;">" "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">@</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">| </span><span>\
</span><span> notify 1 failure \
</span><span> --title </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> --alert-once \
</span><span> --priority high
</span><span> </span><span style="color:#fffd87;">exit</span><span> 1
</span><span>}
</span><span>
</span><span style="color:#fffd87;">prepare</span><span>() {
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>termux-battery-status </span><span style="color:#ececec;">| </span><span>grep </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">status</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">| </span><span>grep -qE </span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">"(CHARGING|FULL)"</span><span style="color:#d6d6d680;">'</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> warn </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Not charging, not performing backup</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fed6af;">return</span><span> 1
</span><span> </span><span style="color:#fed6af;">fi
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>termux-wifi-connectioninfo </span><span style="color:#ececec;">| </span><span>grep </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">supplicant_state</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">| </span><span>grep -q </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">COMPLETED</span><span style="color:#d6d6d680;">"</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> warn </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">WiFi not connected, not performing backup</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fed6af;">return</span><span> 1
</span><span> </span><span style="color:#fed6af;">fi
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>ping -w 10 -c 3 </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">HOST</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">></span><span>/dev/null</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> warn </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Failed to ping target </span><span style="color:#fed6af;">$</span><span style="color:#d68686;">HOST</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fed6af;">return</span><span> 1
</span><span> </span><span style="color:#fed6af;">fi
</span><span>}
</span><span>
</span><span style="color:#fffd87;">backup</span><span>() {
</span><span> </span><span style="color:#fffb9d;">local </span><span>-a flags</span><span style="color:#ececec;">=</span><span>()
</span><span>
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> enable interactive output
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span>-t 0 </span><span style="color:#fffd87;">] </span><span style="color:#ececec;">&& </span><span style="color:#fffd87;">[ </span><span>-t 1 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> flags</span><span style="color:#ececec;">+=</span><span>(</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">--stats</span><span style="color:#d6d6d680;">' '</span><span style="color:#d68686;">--progress</span><span style="color:#d6d6d680;">' '</span><span style="color:#d68686;">--list</span><span style="color:#d6d6d680;">'</span><span>)
</span><span> </span><span style="color:#fed6af;">fi
</span><span>
</span><span> info </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Starting backup</span><span style="color:#d6d6d680;">"
</span><span> ionice -c 3 \
</span><span> nice -n20 \
</span><span> borg create \
</span><span> --noatime \
</span><span> --compression</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">lz4</span><span style="color:#d6d6d680;">' </span><span>\
</span><span> --exclude-caches \
</span><span> --exclude</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">fm:/storage/emulated/0/*/.thumbnails</span><span style="color:#d6d6d680;">' </span><span>\
</span><span> --exclude</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">pp:/storage/emulated/0/Android/data</span><span style="color:#d6d6d680;">' </span><span>\
</span><span> --exclude</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">pp:/storage/emulated/0/Android/obb</span><span style="color:#d6d6d680;">' </span><span>\
</span><span> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{flags[@]}</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{TARGET}::</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}-{utcnow:%Y-%m-%dT%H:%M:%S}</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> /storage/emulated/0/ \
</span><span> /data/data/com.termux/files/home
</span><span>}
</span><span>
</span><span style="color:#fffd87;">prune</span><span>() {
</span><span> </span><span style="color:#fffb9d;">local </span><span>-a flags</span><span style="color:#ececec;">=</span><span>()
</span><span>
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> enable interactive output
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span>-t 0 </span><span style="color:#fffd87;">] </span><span style="color:#ececec;">&& </span><span style="color:#fffd87;">[ </span><span>-t 1 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> flags</span><span style="color:#ececec;">+=</span><span>(</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">--stats</span><span style="color:#d6d6d680;">' '</span><span style="color:#d68686;">--list</span><span style="color:#d6d6d680;">'</span><span>)
</span><span> </span><span style="color:#fed6af;">fi
</span><span>
</span><span> info </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Pruning old backups...</span><span style="color:#d6d6d680;">"
</span><span> borg prune \
</span><span> --prefix</span><span style="color:#ececec;">=</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}-</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> --keep-within</span><span style="color:#ececec;">=</span><span>14d \
</span><span> --keep-daily</span><span style="color:#ececec;">=</span><span>31 \
</span><span> --keep-weekly</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span>((</span><span style="font-weight:bold;color:#87d6d5;">6 </span><span style="color:#ececec;">* </span><span style="font-weight:bold;color:#87d6d5;">4</span><span>)) \
</span><span> --keep-monthly</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span>(( </span><span style="font-weight:bold;color:#87d6d5;">2 </span><span style="color:#ececec;">* </span><span style="font-weight:bold;color:#87d6d5;">12 </span><span>)) \
</span><span> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{flags[@]}</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{TARGET}</span><span style="color:#d6d6d680;">"
</span><span>}
</span><span>
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Run once per day, unless BORGBACKUP_FORCE=1
</span><span>MARKER_FILE</span><span style="color:#ececec;">=</span><span style="color:#d68686;">~/.borgbackup-</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}-</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">(date +</span><span style="color:#fed6af;">%</span><span style="color:#d68686;">Y-</span><span style="color:#fed6af;">%</span><span style="color:#d68686;">m-</span><span style="color:#fed6af;">%</span><span style="color:#d68686;">d)</span><span style="color:#d6d6d680;">"
</span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{BORGBACKUP_FORCE</span><span style="color:#ececec;">:-</span><span style="color:#d68686;">0}</span><span style="color:#d6d6d680;">" </span><span>-eq 0 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> </span><span style="color:#fed6af;">if </span><span style="color:#fffd87;">[ </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">(date +</span><span style="color:#fed6af;">%</span><span style="color:#d68686;">H)</span><span style="color:#d6d6d680;">" </span><span>-lt 4 </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Backup not yet due, waiting...</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">exit</span><span> 0
</span><span> </span><span style="color:#fed6af;">elif </span><span style="color:#fffd87;">[ </span><span>-f </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">MARKER_FILE</span><span style="color:#d6d6d680;">" </span><span style="color:#fffd87;">]</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Backup already ran today</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">exit</span><span> 0
</span><span> </span><span style="color:#fed6af;">fi
</span><span style="color:#fed6af;">fi
</span><span>
</span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>prepare</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> info </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Server connectivity or charging status does not meet expectations, skipping backup.</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">exit</span><span> 1
</span><span style="color:#fed6af;">fi
</span><span>rm -f ~/.borgbackup-</span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}</span><span style="color:#d6d6d680;">"</span><span>-</span><span style="color:#ececec;">*
</span><span>touch </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">MARKER_FILE</span><span style="color:#d6d6d680;">"
</span><span>
</span><span style="color:#fffd87;">trap </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">cleanup</span><span style="color:#d6d6d680;">"</span><span> EXIT
</span><span>termux-wake-lock
</span><span>notify 0 progress \
</span><span> --alert-once \
</span><span> --ongoing \
</span><span> --priority low \
</span><span> --title </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">borgbackup</span><span style="color:#d6d6d680;">" </span><span>\
</span><span> --content </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Running backup for </span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{HOSTNAME}</span><span style="color:#d6d6d680;">"
</span><span>
</span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>backup</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> err </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Backup failed, aborting!</span><span style="color:#d6d6d680;">"
</span><span style="color:#fed6af;">fi
</span><span style="color:#fed6af;">if </span><span style="color:#ececec;">! </span><span>prune</span><span style="color:#ececec;">; </span><span style="color:#fed6af;">then
</span><span> warn </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Pruning failed. Continuing anyway.</span><span style="color:#d6d6d680;">"
</span><span style="color:#fed6af;">fi
</span><span>
</span><span>info </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">Backup finished successfully</span><span style="color:#d6d6d680;">"
</span></code></pre>
<h3 id="scheduling-the-backup-script">Scheduling the backup script</h3>
<p>We can now run the backup script manually to test whether it works, but it’s not yet run automatically. To do that, we
must invoke <code>termux-job-scheduler</code>:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>termux-job-scheduler \
</span><span> --script ~/bin/borg-backup-to-nas \
</span><span> --period-ms 1800000 \
</span><span> --network unmetered \
</span><span> --persisted true
</span></code></pre>
<p>Here, I have chosen</p>
<ul>
<li>a frequency of 1.8 million milliseconds, that is 1800 seconds or 30 minutes</li>
<li>unmetered network (the script would not perform a backup anyway otherwise)</li>
<li>keeping the job across a phone reboot (<code>--persisted true</code>)</li>
</ul>
<p>You can verify that the scheduler has accepted the job using <code>termux-job-scheduler -p</code>.</p>
<h2 id="seeing-it-in-action">Seeing it in action</h2>
<p>Seeing is believing, so I recoded the backup run. See also the impressive deduplication statistics at the end that show
why I like doing by backups with borg:</p>
<video preload="metadata" controls="" width="780" height="360">
<source src="/blog/2022/01/25/backing-up-your-android-phone-with-borgbackup//backup.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
</video>
<span class="pullquote-right" data-pullquote="A new incremental backup takes about one and a half minutes for the 26 GB of data on my phone's SD card">
<p>A new incremental backup takes about one and a half minutes for the 26 GB of data on my phone’s SD card. With
minimal changes, a new backup uses just 140 kB of disk space on the backup server. This makes new backups
extraordinarily cheap, so that there is no good reason not to do them often.</p>
</span>
<h2 id="where-to-go-from-here">Where to go from here</h2>
<p>This solution works for me. If you want to replicate this setup, keep in mind that this backup does not include call
logs, messages, Wi-Fi networks, or apps. You may want to use an separate app (such as <a href="https://play.google.com/store/apps/details?id=org.swiftapps.swiftbackup">Swift Backup</a>) to write
this data to a folder on your SD card, which would include them in the borgbackup run.</p>
<p>I also do not yet have long term test data on how stable the solution is, and how often Android’s power saving features
interrupt my backup. I may update this post later with results.</p>
<p><strong>Update:</strong> I have been running this setup for just shy of two weeks now, and have yet to encounter issues with it. When
not at home, the backup is re-attempted every 30 minutes. If there was an issue, I would receive a notification, which
means the process is silent and low on maintenance.</p>
<p>If you try this on your own, I would welcome feedback—feel free to use the comments below to leave your experience or
any suggestions for improvement you might have.</p>
The Journey to Storing SMTP Passwords in a Database2020-11-18T16:40:48+00:002020-11-18T16:40:48+00:00https://neverpanic.de/blog/2020/11/18/the-journey-to-storing-smtp-passwords-in-a-database/<p>Back in the days when spam was not a thing, and the internet was simpler, if
you wanted to give users an email address under your domain, you’d just add
a forward to your mail server configuration. That took care of the receiving
side, and sending could usually be done with whatever mail server people
already had. Nobody bothered checking the envelope sender or <code>From</code> header
anyway, and mail servers would happily accept mail from everyone and everywhere
as long as it seemed that it had ended up in the right place. And it was good.
And then along came spam.</p>
<h2 id="spf-and-dkim-you-need-to-run-your-own-smtp">SPF and DKIM? You need to run your own SMTP.</h2>
<p>Now, of course, this is not a theoretical example. <a href="https://www.macports.org/">MacPorts</a> has
always provided its project members with an email alias under <code>@macports.org</code>.
However, to fight spam, smart people came up with a multitude of ways to figure
out whether mail received by a mail server was actually sent by who the
envelope claimed to have sent it. There are currently two major mechanisms for
this purpose: <a href="https://en.wikipedia.org/wiki/Sender_Policy_Framework">Sender Policy Framework (SPF)</a> and
<a href="https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail">DomainKeys Identified Mail (DKIM)</a>.</p>
<p>SPF allows administrators to publish a list of servers that are permitted to
send mail on behalf of a specific domain. Of course, since MacPorts did not
actually provide an SMTP server and expected our developers to use their own
ones, we had no way of gathering such a list and would thus allow the entire
internet to send mail on behalf of <code>@macports.org</code>, something more and more
mail providers are nowadays treating as an indicator for spam.</p>
<p>DKIM, on the other hand, adds a cryptographic signature to certain selected
fields of an email when it passes through the outgoing server, to be verified
against a public key published in DNS on the receiving end. But again, since
there was no single central SMTP serving <code>@macports.org</code>, we could not ensure
that all mails had such a signature, and thus could not enable DKIM – which
providers are also using as an indication for spam.</p>
<p>We did know for a while that we would eventually have to setup email
submission, but have been delaying the actual setup, since we needed a way to
configure the passwords that should be used for SMTP. Since MacPorts’ migration
to GitHub in <a href="https://lists.macports.org/pipermail/macports-dev/2016-October/034232.html">October</a> <a href="https://lists.macports.org/pipermail/macports-dev/2016-October/034399.html">2016</a>, we only use GitHub’s OAuth2 for
authentication. And while mail clients are slowly implementing support for
that in SMTP and IMAP, it is not yet widespread enough to be usable in our
case.</p>
<p>So, my todo list came down to</p>
<ul>
<li>Write a web application that uses GitHub OAuth2 to authenticate users</li>
<li>Allow setting the SMTP password in a database from that web application.
I figured a database would be a good idea, since it’s the most convenient
resource to access from different unix users, unlike files and/or sockets
where I would have had to configure groups.</li>
<li>Configure SMTP authentication against the passwords in the database into
Postfix.</li>
</ul>
<p>Sounds simple enough. Boy, was I wrong…</p>
<span id="continue-reading"></span><h2 id="smtp-authentication-in-postfix-cyrus-or-dovecot-sasl">SMTP authentication in Postfix: Cyrus or Dovecot SASL?</h2>
<p>I decided to start at the bottom by getting SMTP authentication set up in our
Postfix installation. Postfix uses the
<a href="https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer">Simple Authentication and Security Layer (SASL)</a> to authenticate SMTP
users. There are two options for SASL daemons, <a href="https://www.cyrusimap.org/sasl/">Cyrus SASL</a>, and
<a href="https://wiki.dovecot.org/HowTo/PostfixAndDovecotSASL">Dovecot</a>.</p>
<h3 id="cyrus-sasl">Cyrus SASL</h3>
<p>I spent a while looking at the two options. Both have options to talk to
PostgreSQL, but one point in Cyrus’ documentation did surprise me:</p>
<blockquote>
<p>The Cyrus SASL library also supports some “shared secret” authentication
methods: CRAM-MD5, DIGEST-MD5 and its successor SCRAM. These methods rely on
the client and the server sharing a “secret”, usually a password. The server
generates a challenge and the client a response proving that it knows the
shared secret. This is much more secure than simply sending the secret over
the wire proving that the client knows it.</p>
</blockquote>
<blockquote>
<p>There’s a downside: in order to verify such responses, the server must keep
passwords or password equivalents in a database; if this database is
compromised, it is the same as if all the passwords for the realm are
compromised.</p>
</blockquote>
<p>I understand that the secret is required in plain text for these
challenge-response authentication mechanisms, but since I was only going to
allow login on encrypted connections anyway, I really did not need that
functionality, and it sounded like bad practice to store passwords in plain
text.</p>
<p>So I kept looking around the documentation and the source code for a way to
store hashed passwords only.
<a href="https://www.cyrusimap.org/sasl/sasl/pwcheck.html#auxprop">Some documentation seems to suggest</a> that this is possible by
setting <code>pwcheck_method = auxprop-hashed</code>, but there is no description
whatsoever for that option. I found the
<a href="https://github.com/cyrusimap/cyrus-sasl/commit/62ce0768aa375cf0d16102570970b232dcb1cb28">commit that adds this feature</a>, but also could not figure
out which hash function was used here.</p>
<p>I was honestly surprised to find no clear documentation of what I think is
a reasonable feature request (or indication of whether this would work at all)
in what is probably the most popular open source implementation of SASL.</p>
<h3 id="dovecot-sasl">Dovecot SASL</h3>
<p>In comparison, Dovecot’s authentication documentation has a
<a href="https://doc.dovecot.org/configuration_manual/authentication/password_schemes/">list of password schemes</a> with up-to-date algorithms such as
Argon2. A clear advantage for dovecot, so that’s what I went with. On a debian
system, switching dovecot’s user lookup to a database is rather simple; in
<code>/etc/dovecot/conf.d/10-auth.conf</code>, I had to comment <code>!include auth-system.conf.ext</code> and uncomment <code>!include auth-sql.conf.ext</code>.</p>
<p>I configured <code>auth-sql.conf.ext</code> as follows:</p>
<pre data-lang="conf" style="background-color:#393939;color:#dedede;" class="language-conf "><code class="language-conf" data-lang="conf"><span style="color:#fffb9d;">passdb </span><span>{
</span><span> driver </span><span style="color:#ececec;">=</span><span> sql
</span><span> args </span><span style="color:#ececec;">= /</span><span>etc</span><span style="color:#ececec;">/</span><span>dovecot</span><span style="color:#ececec;">/</span><span>dovecot</span><span style="color:#ececec;">-</span><span>sql</span><span style="color:#ececec;">.</span><span>conf</span><span style="color:#ececec;">.</span><span>ext
</span><span>}
</span><span style="color:#fffb9d;">userdb </span><span>{
</span><span> driver </span><span style="color:#ececec;">=</span><span> static
</span><span> args </span><span style="color:#ececec;">=</span><span> uid</span><span style="color:#ececec;">=</span><span>postfix gid</span><span style="color:#ececec;">=</span><span>postfix home</span><span style="color:#ececec;">=/</span><span>var</span><span style="color:#ececec;">/</span><span>mail</span><span style="color:#ececec;">/</span><span>%u
</span><span>}
</span></code></pre>
<p>This causes password lookups (<code>passdb</code>) to use the SQL query configured in
<code>/etc/dovecot/dovecot-sql.conf.ext</code>, but does not run a separate query for the
user details such as the user’s home directory, UID, and GID. Since the virtual
users would not have a separate unix user on the machine anyway, there was
little point in setting this to anything but the postfix UID and GID.</p>
<p>In <code>dovecot-sql.conf.ext</code>, I only set the <code>driver</code>, <code>connect</code>,
<code>default_pass_scheme</code> and <code>password_query</code> fields:</p>
<pre data-lang="conf" style="background-color:#393939;color:#dedede;" class="language-conf "><code class="language-conf" data-lang="conf"><span>driver </span><span style="color:#ececec;">=</span><span> pgsql
</span><span>connect </span><span style="color:#ececec;">= </span><span style="font-weight:bold;color:#cc9495;">postgresql:///smtpselfservice
</span><span>default_pass_scheme </span><span style="color:#ececec;">= </span><span style="font-weight:bold;color:#d6d6ae;">ARGON2ID
</span><span>password_query </span><span style="color:#ececec;">= \
</span><span> SELECT username</span><span style="color:#ececec;">,</span><span> domain</span><span style="color:#ececec;">,</span><span> password </span><span style="color:#ececec;">\
</span><span> FROM users </span><span style="font-weight:bold;color:#d6d6ae;">WHERE</span><span> username </span><span style="color:#ececec;">= </span><span style="color:#d68686;">'%n' </span><span style="font-weight:bold;color:#d6d6ae;">AND</span><span> domain </span><span style="color:#ececec;">= </span><span style="color:#d68686;">'%d'
</span></code></pre>
<h2 id="making-postfix-talk-to-dovecot-sasl">Making Postfix talk to Dovecot SASL</h2>
<p>Fortunately, integrating Dovecot SASL into Postfix was surprisingly easy. There
are various articles on the internet that describe the setup, and postfix’
documentation can be used to decide which specific configuration values fit
your use case. I ended up with authentication on the submission service on port
587, by configuring <code>/etc/postfix/master.cf</code>:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>submission inet n - - - - smtpd
</span><span> -o syslog_name=postfix/submission
</span><span> -o smtpd_tls_security_level=encrypt
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> enable SASL auth
</span><span> -o smtpd_sasl_auth_enable=yes
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> support some broken SASL clients
</span><span> -o broken_sasl_auth_clients=yes
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> use Dovecot SASL
</span><span> -o smtpd_sasl_type=dovecot
</span><span> -o smtpd_sasl_path=private/auth
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> No plaintext over unencrypted connections
</span><span> -o smtpd_sasl_security_options=noanonymous,noplaintext
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> No anonymous authentication
</span><span> -o smtpd_sasl_tls_security_options=noanonymous
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Add a header with the login name to sent messages
</span><span> -o smtpd_sasl_authenticated_header=yes
</span><span> -o smtpd_reject_unlisted_recipient=no
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Only allow authenticated users to send mail
</span><span> -o smtpd_client_restrictions=permit_sasl_authenticated,reject
</span><span> -o smtpd_sender_restrictions=reject_non_fqdn_sender
</span><span> -o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject
</span><span> -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
</span><span> -o milter_macro_daemon_name=ORIGINATING
</span></code></pre>
<p>Unfortunately, this configuration would allow authenticated users to send with
an abitrary envelope sender address, i.e. once authenticated as
<code>cal@macports.org</code>, I can send mails on behalf of <code>raimue@macports.org</code>. Not
ideal.</p>
<p>Postfix has <code>smtpd_sender_restrictions = reject_sender_login_mismatch</code>
(<a href="http://www.postfix.org/postconf.5.html#smtpd_sender_restrictions">[1]</a>) to prevent this, but this feature requires
<code>smtpd_sender_login_maps</code> (<a href="http://www.postfix.org/postconf.5.html#smtpd_sender_login_maps">[2]</a>) to be set to a lookup table that
resolves SASL login names to a list of email addresses from which the given
user is authorized to send. And that specific information, is of course,
available in the database. So in addition to dovecot SASL talking to the
database for the password lookup, Postfix must query the same table to get the
list of permitted sender addresses.</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>smtpd_sender_login_maps = proxy:pgsql:/etc/postfix/smtpd_sender_login_maps.cf
</span></code></pre>
<p>sets up Postfix to query this information from the database. Note the added
<code>proxy</code>, which is required because smtpd runs in chroot and could otherwise not
talk to the database server. In <code>/etc/postfix/smtpd_sender_login_maps.cf</code>,
I again configured the appropriate database connection and query:</p>
<pre data-lang="conf" style="background-color:#393939;color:#dedede;" class="language-conf "><code class="language-conf" data-lang="conf"><span>hosts </span><span style="color:#ececec;">= </span><span style="font-weight:bold;color:#cc9495;">postgresql:///smtpselfservice
</span><span>dbname </span><span style="color:#ececec;">=</span><span> smtpselfservice
</span><span>query </span><span style="color:#ececec;">= </span><span style="font-weight:bold;color:#d6d6ae;">SELECT</span><span> (username </span><span style="color:#ececec;">|| </span><span style="color:#d68686;">'@' </span><span style="color:#ececec;">||</span><span> domain) </span><span style="font-weight:bold;color:#d6d6ae;">FROM</span><span> users </span><span style="font-weight:bold;color:#d6d6ae;">WHERE</span><span> username </span><span style="color:#ececec;">= </span><span style="color:#d68686;">'%u'</span><span> and domain </span><span style="color:#ececec;">= </span><span style="color:#d68686;">'%d'
</span></code></pre>
<p>Of course this query is rather pointless, since it will only ever return its
arguments. I couldn’t find a template-based method that would have allowed my
to return the SASL login name (which is an email address in my case), and
arguably, this method is more flexible in case of future changes to the
database schema.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Out of the three steps outlined above, I have only covered the last one,
configuring Postfix to use a PostgreSQL database containing hashed passwords
with modern hashing algorithms for user authentication. Dovecot SASL provided
comprehensive documentation that made this fairly simple, while Cyrus SASL was
unfortunately lacking significantly in documentation.</p>
<p>One might think writing the web application to set the password and getting
this to work with clients would be easier, to which I can only say: Never
underestimate mail clients! But that’s a story for a different post.</p>
Attending the Google Summer of Code Mentor Summit2017-10-29T20:24:20+00:002017-10-29T20:24:20+00:00https://neverpanic.de/blog/2017/10/29/attending-the-google-summer-of-code-mentor-summit/<p>From Friday, October 13th to Sunday, October 15th 2017 I had the opportunity to attend the Google Summer of Code Mentor
Summit in <a href="http://www.openstreetmap.org/relation/112145">Sunnyvale, CA</a>. This is a summary of my experiences.</p>
<p>If you are not familiar with <a href="https://en.wikipedia.org/wiki/Google_Summer_of_Code">Google Summer of Code</a>, it is a program where university students spend the summer
working for an open source project. Google pays the students as a way to give back to open source, which is heavily used
in their products. Students are mentored by experienced developers from the projects and Google invites two mentors per
project into the US in autumn each year for an <a href="https://en.wikipedia.org/wiki/Unconference">unconference</a>-style summit.</p>
<p>Together with <a href="https://github.com/mojca">Mojca Miklavec</a> I mentored <a href="https://github.com/l2dy">Zero King</a>, who did a great job at implementing better usability
for GitHub pull requests for MacPorts by setting up Travis CI and a PR helper bot. Our original plan was to attend the
Mentor Summit with 2015 student and this year organization administrator <a href="https://github.com/JacksonIsaac">Jackson Isaac</a>, but unfortunately
his visa to the US was denied and Mojca stepped in instead.</p>
<span id="continue-reading"></span><h2 id="friday-13th-travelling">Friday 13th Travelling</h2>
<p>My journey to Sunnyvale started in their early morning of Friday, the 13th. I took a train to Frankfurt for an
eleven-hour non-stop flight to San José. I had not noticed the date when booking the flight, but when my airline of
choice decided to do a two-for-one special on takeoffs I was beginning to think I may not have chosen the best date to
fly ;-).</p>
<p>Due to a cockpit window that flew open on the roll, our first takeoff was aborted. With hot brakes and a potential
technical problem in the cockpit, we saw the fire brigade roll up to the aircraft and later on had to deboard as the
problem could not be fixed immediately. While waiting at the gate for further information on the flight, I met other
open source project representatives going to the Mentor Summit.</p>
<span class="pullquote-left" data-pullquote=""I'm going to GSoC Mentor Summit", "We're getting quite a few of you guys lately!"">
<p>About two and half hours later, our flight eventually left with a repaired cockpit window, but without two containers of
luggage that were unloaded on the apron in Frankfurt, presumably because somebody decided not to reboard the flight and
their luggage had to be removed. After an otherwise uneventful flight to SJC and a breeze with the immigration process
(“I’m going to GSoC Mentor Summit”, “We’re getting quite a few of you guys lately!”) I unfortunately had to learn
that my luggage was among those unloaded in Frankfurt and I was going to have to get by without it until Sunday
afternoon. This meant that I had to do some shopping at the expense of my breakfast on Saturday.</p>
</span>
<h2 id="friday-evening-opening-event">Friday Evening: Opening Event</h2>
<div>
<img
class="left"
src="/processed_images/97ea90a982d2b0df00.jpg"
title="Obligatory business when attending a Google conference"
alt="Person standing next to the Google Tech Corners logo"
width="395"
height="246"
/>
</div>
<p>The weekend started off on Friday night with dinner and an opening talk by the conference organizers at the summit
location Google Tech Corners. About 5 miles from Google’s main campus, Google Tech Corners are a series of office
buildings that host both Google employees as well as meeting rooms for events such as the Mentor Summit.</p>
<h2 id="saturday-sunday-unconference-sessions">Saturday & Sunday: Unconference Sessions</h2>
<p>The two remaining days were almost entirely reserved for unconference sessions on various topics. I attended a number of
different sessions but also extensively used the “hallway track” to get in touch with others in open source. I will list
some notable sessions below.</p>
<ul>
<li><strong>Grading Criteria for Proposals</strong><br />
A discussion on how to avoid personal preferences and bias when choosing proposals. There were a lot of good ideas and
we ended up summarizing that there should be a list of criteria published before GSoC and available for students to
grade technical and social skills as well as the project and its planning.<sup class="footnote-reference"><a href="#1">1</a></sup><br />
My personal takeaway from this session was that GSoC consists of a lot of small projects that only have less than five
students, and big projects are surprisingly uncommon.</li>
<li><strong>Open Source Autonomous Vehicles - Are They Possible?</strong><br />
Sort of a brainstorming on what is needed for autonomous driving, what is already there, who is currently
participating in automotive open source and how to convince more companies to contribute. An interesting topic in
general, especially due to the legal questions around autonomous driving.</li>
<li><strong>Google Track: What More?</strong><br />
A brainstorming discussion on what Google could do besides Google Summer of Code to help open source projects. There
were a lot of different ideas, but from what I recall none of them were an obvious no-brainer. I asked whether Google
could provide build capacity, which was backed by the Homebrew representative and some other audience members. I would
be very surprised if this was feasible for Google, though.</li>
<li><strong>Organizational Homes for FOSS Projects</strong><br />
This loose discussion on different umbrella projects was hosted by <a href="https://sfconservancy.org/about/staff/#brett">Software Freedom Conservancy’s Brett Smith</a>.
I learned that openSUSE is not only the open source version of the SUSE distribution, but also an umbrella for various
other projects.</li>
<li><strong>Convincing Large Companies to Use Open Source</strong><br />
Largely a discussion on which licenses are company-friendly and which ones are not. Mentioned some of the problems
companies face and where open source could help, such as making license compliance easy by providing machine-readable
license specifications (e.g. using <a href="https://spdx.org/">SPDX</a>).<br />
From a MacPorts point of view, we should probably consider adapting the standardized identifiers from the
<a href="https://spdx.org/licenses/">SPDX license list</a> for our license field. Unfortunately that would be some additional effort for
licenses we did not distinguish so far, such as the various BSD-style licenses.</li>
<li><strong>Fail Your Students!</strong><br />
Facilitated by the GSoC organization team, this discussion allowed mentors to align on when and how to fail students.
In quite a number of cases it seems that if you are asking yourself the question whether to fail the student the answer
is likely “yes”. Google has an interest in keeping the bar for successful GSoC participation high and does not want
mentors to waste their time. I learned that mentor availability is the biggest scalability problem with Google Summer
of Code, not student availability or financial reasons. Additionally, suggesting the student withdraw before failing
him may make the decision easier for mentors and less harsh for students.</li>
</ul>
<div>
<img
class="right"
src="/processed_images/ecc4ac6592a6dce700.jpg"
title="Attendees could mention topics to talk about on their badges"
alt="Mentor Summit Badge for Clemens Lang from the MacPorts Project, mentioning talking points Reproducibility, Licenses, Automotive, Security, Packaging and Licensing, and displaying flags Former Student, First Mentor Summit, and Org Admin"
width="395"
height="726"
/>
</div>
<h2 id="evening-activities-the-hallway-track">Evening Activities & the Hallway Track</h2>
<p>In addition to the various sessions, meeting other people in the hallways or over lunch was an interesting and very
rewarding experience. I met (among many others) <a href="https://github.com/woodruffw">William Woodruff</a> from the Homebrew project, and not only
did we get along very well<sup class="footnote-reference"><a href="#2">2</a></sup>, but we also noticed that both Homebrew and MacPorts face a number of similar problems
such as build CPU cycle availability in their endeavor to bring open source software to the Mac.</p>
<div>
<img
class="right"
src="/processed_images/d0319c17617f51b900.jpg"
title="Roasting marshmallows by the fire"
alt="Close-up shot of a fireplace"
width="395"
height="289"
/>
</div>
<p>On Saturday night, Google organized a social event with live music, marshmallow roasting, drinks and two satirical
cartoonists outside the Google Tech Corner buildings. While I did not anticipate how chilly it would actually get in
California in October, I thoroughly enjoyed the evening and the discussions with other open source projects.</p>
<p>I learned, for example, that there is no central federal institution that keeps a copy of DMCA takedown notices sent in
the US, and that a university stepped in with a research project based on open source, which eventually lead to a number
of fake DMCA notices being discovered. In the same spirit, we discussed (but unfortunately could not answer) whether the
European Union’s “right to be forgotten” at least provides record of who attempted to remove information.</p>
<p>On Sunday evening after the official end of the summit, I joined a few other attendees for a private barbecue in San
Francisco. In addition to some great food, I learned a lot about <a href="https://www.qubes-os.org/">Qubes OS</a>, an operating system that reduces
your attack surface by isolating many attack vectors in separate virtual machines on top of Xen.</p>
<h2 id="leaving-california">Leaving California</h2>
<p>Overall, the Google Summer of Code Mentor Summit was a fascinating experience and very well organized. I had a blast at
the conference, learned a lot of new things and met many interesting people. I would definitely attend again.</p>
<p>On Monday afternoon I left for the airport and had a great view over the bay area after takeoff for an uneventful flight
back home.</p>
<div>
<img
class="center"
src="/processed_images/b2ded35478bed88700.jpg"
title="The bay area from above after takeoff"
alt="The bay area from above after takeoff"
width="790"
height="501"
/>
</div>
<div class="footnote-definition" id="1"><sup class="footnote-definition-label">1</sup>
<p>Detailed meeting minutes are available. Contact me if you are interested.
<sup class="footnote-reference"><a href="#2">2</a></sup>: There seems to be a pattern; I met Mike McQuaid in <a href="https://lists.macports.org/pipermail/macports-dev/2015-December/032016.html">Dec 2015 in Athens</a> and had a similar experience.</p>
</div>
Baidu Spider Caused More Than 80% of Our Trac's HTTP Traffic2017-02-14T08:00:19+00:002017-02-14T08:00:19+00:00https://neverpanic.de/blog/2017/02/14/baidu-spider-caused-more-than-80-percent-of-our-tracs-http-traffic/<p>Since <a href="https://lists.macports.org/pipermail/macports-dev/2016-August/033405.html">MacPorts’ move off</a> <a href="https://lists.macports.org/pipermail/macports-dev/2016-October/034232.html">Apple’s MacOSForge</a>
in October, we have been running <a href="https://trac.macports.org/">MacPorts’ Trac installation</a> on our own
infrastructure. We used to rely on server and admin time generously donated by
Apple. Now that we no longer enjoy this luxury, we are on our own when it comes
to keeping our infrastructure running.</p>
<p>For a few months, we were bedeviled by high server load apparently caused by
our Trac installation and had a hard time figuring out the cause. Our
monitoring showed a large number of HTTP requests and Trac’s response time
would regularly take a nose-dive as soon as the backup started.</p>
<p>After a few attempts of tuning various knobs without too much success,
I finally decided to grab the Apache access logs and run <a href="http://www.awstats.org/">awstats</a> on
them. Since we rotate our access logs biweekly<sup class="footnote-reference"><a href="#1">1</a></sup> I only had 10 days of
February for analysis, but even those 10 days revealed some pretty
interesting data.</p>
<span id="continue-reading"></span>
<p>Between February 1<sup>st</sup> and February 10<sup>th</sup>, we had
648,004 hits classified as “viewed traffic” by awstats, as opposed to 7,788,704
hits of “not viewed traffic”. Viewers transferred 3.31 GB of data, while
an astonishing 351.92 GB of bandwidth were used by bots.</p>
<p>Awstats’ “Robots/Spiders visitors” analysis also told us that our top three
spider visitors were</p>
<ul>
<li>Baiduspider with 3,206,576 hits (+37 for <code>robots.txt</code>) using <strong>288.44 GB</strong> of bandwidth</li>
<li>YandexBot with 375,010 hits (+60), with 1.75 GB of bandwidth</li>
<li>AhrefsBot with 219,847 hits (+17), which used 2.83 GB</li>
</ul>
<p>For comparison, in the same time frame, Googlebot visited our Trac installation
40,777 (+21) times, causing 512 MB of traffic.</p>
<span class="pullquote-right" data-pullquote="Baidu did not only cause 81 % of our traffic,
they also sent us about 3.7 requests per second on average.">
<p>As these statistics show, Baidu did not only cause 81 % of our traffic,
they also sent us about 3.7 requests per second on average. Using
<a href="https://httpd.apache.org/docs/2.4/mod/mod_status.html">Apache mod_status</a> spot checks I saw that requests coming from
Baidu’s IP ranges would often crawl URLs in Trac’s repository browser
(<code>/browser</code>) and history (<code>/log</code>). Since our move to GitHub, we only maintain
these Trac modules to avoid breaking old links, but do not actively advertise
them any longer. It seemed like a bad idea to spend that amount of traffic for
a subsection of our ticket system that we only kept for compatibility.</p>
</span>
<p>As a consequence, we added an exclusion to our <code>robots.txt</code><sup class="footnote-reference"><a href="#2">2</a></sup> for Baidu and
continued watching our statistics. As an image is worth a thousand words, here
are some excerpts from our monitoring that visualize the effect. I added the
block around midnight on February 11<sup>th</sup>.</p>
<p><div>
<img
class="center"
src="/processed_images/6401ee89d8ed59f500.png"
alt="CPU usage - by week"
width="790"
height="597"
/>
</div>
<div>
<img
class="center"
src="/processed_images/13bdac92efb8fe1c00.png"
alt="Apache volume - by week"
width="790"
height="445"
/>
</div>
<div>
<img
class="center"
src="/processed_images/32701fbb80611e9300.png"
alt="PostgreSQL transactions - by week"
width="790"
height="464"
/>
</div>
<div>
<img
class="center"
src="/processed_images/41f3c9f881d31e6900.png"
alt="Temperatures - by week"
width="790"
height="559"
/>
</div>
</p>
<p>This leaves me wondering: How is that kind of crawling behavior acceptable and
why has Baidu not improved their detection of similar content under dynamic
URLs?</p>
<div class="footnote-definition" id="1"><sup class="footnote-definition-label">1</sup>
<p>This is a conscious decision; we neither need, nor want more data.
<sup class="footnote-reference"><a href="#2">2</a></sup>: There is <a href="http://webmasters.stackexchange.com/questions/31837/how-to-block-baidu-spiders#comment30961_31838">some</a> <a href="https://perishablepress.com/block-baidu-bot/">doubt</a> <a href="https://www.webmasterworld.com/search_engine_spiders/4348357.htm">online</a> on whether Baidu respects <code>robots.txt</code> and I wanted to see quick results, so I also reconfigured our webserver to deny access to the <code>/browser</code> and <code>/log</code> URLs for Baidu’s IP ranges. Spot checks in mod_status seem to indicate that Baidu in fact respects the blocks in <code>robots.txt</code> after a few days, just as their <a href="http://www.baidu.com/search/robots_english.html">documentation</a> states.</p>
</div>
CVE-2015-0842, CVE-2015-0843 in yubiserver2017-02-13T00:17:17+00:002017-02-13T00:17:17+00:00https://neverpanic.de/blog/2017/02/13/cve-2015-0842-cve-2015-0843/<p>Back in March 2015, I reported a security issue in <a href="http://www.include.gr/debian/yubiserver/">Yubiserver</a>,
a small specialized HTTP server to verify HOTP/OATH tokens generated by
Yubico’s <a href="https://www.yubico.com/products/yubikey-hardware/">Yubikeys</a>. I’m publishing the details for reference.</p>
<p>I was looking for a new Yubikey validation server and, given its small size,
decided to code review any candidates due to their small size. While looking at
yubiserve, I found security issues in the code.</p>
<span id="continue-reading"></span><h2 id="cve-2015-0842">CVE-2015-0842</h2>
<p>CVE-2015-0842 is an SQL injection issue. Yubiserver’s database backend is
SQLite. User input is not added to prepared statements using
<a href="https://www.sqlite.org/c3ref/bind_blob.html"><code>sqlite3_bind_text</code></a>, but instead uses <code>sprintf(3)</code> without
any escaping. This affects the first 12 bytes of a Yubico OTP in OTP mode in
<a href="https://github.com/Yubico/yubico-c/blob/master/modhex.1.txt">modhex encoding</a> (although the encoding will be skipped if non-modhex
characters are encountered). Because the attacker is limited to 12 characters
inside the <code>WHERE</code> clause of <code>SELECT</code> query the attack potential is small.</p>
<p>A similar issue affects the <code>id</code> parameter that is used to select a key for the
HMAC that protects the response, with the exception that only a length
restriction on the whole request exists. Again, this injection happens in
a <code>SELECT</code> query. While an attacker can escape the quoting and add arbitrary SQL
in the <code>WHERE</code> clause, the possibilities for exploitation are limited: <code>INSERT</code>
and <code>DELETE</code> statements cannot be used in subqueries and separating an
additional query using a semicolon does not work due to the use of
<a href="https://sqlite.org/c3ref/prepare.html"><code>sqlite3_prepare_v2</code></a> and <a href="https://www.sqlite.org/c3ref/step.html"><code>sqlite3_step</code></a>
over <a href="https://www.sqlite.org/c3ref/exec.html"><code>sqlite3_exec</code></a>.</p>
<p>The vulnerability can possibly be used to extract secret keys from the database
using subqueries. This may be limited by the length limit on queries that can be
executed. A proof of concept that tells us whether a secret key of a yubikey
starts with “ab” would be:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>curl \
</span><span> </span><span style="color:#d6d6d680;">$'</span><span style="color:#d68686;">http://localhost:port/wsapi/2.0/verify?otp=a&id=1</span><span style="font-weight:bold;color:#ff8080;">\'\t</span><span style="color:#d68686;">AND</span><span style="font-weight:bold;color:#ff8080;">\t</span><span style="color:#d68686;">(SELECT</span><span style="font-weight:bold;color:#ff8080;">\t</span><span style="color:#d68686;">aeskey</span><span style="font-weight:bold;color:#ff8080;">\t</span><span style="color:#d68686;">FROM</span><span style="font-weight:bold;color:#ff8080;">\t</span><span style="color:#d68686;">yubikeys)</span><span style="font-weight:bold;color:#ff8080;">\t</span><span style="color:#d68686;">LIKE</span><span style="font-weight:bold;color:#ff8080;">\t</span><span style="color:#d68686;">"ab%";--</span><span style="color:#d6d6d680;">'
</span></code></pre>
<h3 id="links">Links</h3>
<ul>
<li><a href="https://security-tracker.debian.org/tracker/CVE-2015-0842">CVE-2015-0842 on Debian’s security tracker</a></li>
<li><a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=796495">Debian Bug report #796495</a></li>
</ul>
<h2 id="cve-2015-0843">CVE-2015-0843</h2>
<p>CVE-2015-0843 covers multiple buffer overflows due to extensive use of
<code>sprintf(3)</code> without range checks. All inputs that end up in queries, log
messages or are sent back in the response are affected.</p>
<p>Denial of service is trivial by sending long inputs:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>curl \
</span><span> </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">http://localhost:port/wsapi/2.0/verify?otp=x&id=</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">(openssl rand -hex 512)</span><span style="color:#d6d6d680;">"
</span></code></pre>
<p>On most Linux distributions, this triggers the stack canary and kills the
process. Further exploitation may be possible, but has not been attempted.</p>
<h3 id="links-1">Links</h3>
<ul>
<li><a href="https://security-tracker.debian.org/tracker/CVE-2015-0843">CVE-2015-0843 on Debian’s security tracker</a></li>
<li><a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=796495">Debian Bug report #796495</a></li>
</ul>
<h2 id="timeline">Timeline</h2>
<ul>
<li><em>2015-03-31</em>: Issue reported to the Debian security team, who agree to forward it upstream.</li>
<li><em>2015-05-28</em>: Debian security team confirms they are still waiting for patches and agree to follow-up upstream.</li>
<li><em>2015-05-29</em>: Upstream releases 0.6 with patches for the vulnerabilities.</li>
<li><em>2015-10-02</em>: Debian security team confirms CVE-2015-0842 and CVE-2015-0843 are assigned to these issues and patches have not been backported to wheezy and jessie.</li>
<li><em>2017-02-13</em>: Public release.</li>
</ul>
<h2 id="mitigations">Mitigations</h2>
<p>Updating to 0.6 fixes both vulnerabilities. The SQL injections no longer occur
because the code uses prepared statements and parameter binding. The buffer
overflows have been mitigated by switching to <code>snprintf(3)</code> and providing
appropriate sizes, truncating the data if it would overflow the buffer.</p>
<p>The update to 0.6 does not include functional changes, so upgrading should be
safe.</p>
<h2 id="recommendations">Recommendations</h2>
<p>Switch to a server implementation that does not suffer from buffer overflows by
language design. For example:</p>
<ul>
<li><a href="https://github.com/stumpyfr/yubikey-server">yubikey-server</a> in Go</li>
<li><a href="https://github.com/scumjr/yubikeyedup">yubikeyedup</a> in Python</li>
</ul>
Secure Erase on OS X El Capitan2016-04-04T19:09:36+00:002016-04-04T19:09:36+00:00https://neverpanic.de/blog/2016/04/04/secure-erase-on-os-x-el-capitan/<p>With the update to OS X El Capitan, Apple has rewritten Disk Utility. The pre-10.11 Disk Utility
used to have an option to securely erase a disk – a feature I needed because I plan to throw
a faulty disk away.</p>
<p><div>
<img
class="left"
src="/processed_images/283ac142ac544e8d00.png"
title="security options should be in the lower right"
alt="El Capitan Disk Utility erase dialog without security options button"
width="395"
height="202"
/>
</div>
Now, Apple still documents the option in <a href="https://support.apple.com/kb/PH22241?locale=en_US" title="Apple Knowledge Base PH22241: Disk Utility (El Capitan): Erase a volume">KB article PH22241</a>, but has implemented code
that hides the “Security Options” button in certain situations. Unfortunately, they did not document
which conditions need to be fulfilled for the button to be shown, which leads to the situation that
I do not see it on any of my disks. My guess would be that the option is not available for encrypted
disks, but since I do not have any unencrypted drives I cannot verify that assumption.</p>
<h2 id="why-would-you-wipe-an-encrypted-disk">Why would you wipe an encrypted disk?</h2>
<p>For an encrypted volume, wiping the header that contains the master encryption key should be enough
to ensure that no data can be recovered. Conveniently, Apple does not provide an option to wipe the
volume’s encryption header and documentation on Apple’s CoreStorage format it scarce, which means
I don’t know where the header actually is. So a full wipe it is.</p>
<p>Luckily just because the GUI does not support the feature anymore does not mean that it cannot be
done. The <code>diskutil</code> command line tool still has a <code>secureErase</code> option that supports overwriting
entire volumes. Because I was doing this with CoreStorage volumes, I first had to delete that volume
before <code>secureErase</code> would unmount the physical disk below:</p>
<pre style="background-color:#393939;color:#dedede;"><code><span>:) clemens@cBookPro:~$ diskutil cs deleteVolume CD3D75E0-F317-42B6-B44F-FDCB1A9448CD
</span><span>The Core Storage Logical Volume UUID is CD3D75E0-F317-42B6-B44F-FDCB1A9448CD
</span><span>Started CoreStorage operation on disk7 cTM
</span><span>Unmounting disk7
</span><span>Removing Logical Volume from Logical Volume Group
</span><span>Finished CoreStorage operation on disk7 cTM
</span></code></pre>
<p>Once the logical volume was gone, I was able to start the wipe with <code>diskutil secureErase</code>:</p>
<pre style="background-color:#393939;color:#dedede;"><code><span>:) clemens@cBookPro:~$ diskutil secureErase
</span><span>Usage: diskutil secureErase [freespace] level MountPoint|DiskIdentifier|DeviceNode
</span><span>Securely erases either a whole disk or a volume's freespace.
</span><span>Level should be one of the following:
</span><span> 0 - Single-pass zeros.
</span><span> 1 - Single-pass random numbers.
</span><span> 2 - US DoD 7-pass secure erase.
</span><span> 3 - Gutmann algorithm 35-pass secure erase.
</span><span> 4 - US DoE 3-pass secure erase.
</span><span>Ownership of the affected disk is required.
</span><span>Note: Level 2, 3, or 4 secure erases can take an extremely long time.
</span><span>:( clemens@cBookPro:~$ diskutil secureErase 2 disk4
</span><span>Started erase on disk4
</span><span>Pass: 1
</span><span>Pass: 2
</span><span>Pass: 3
</span><span>Pass: 4
</span><span>[ - 0%..10%..20%..30%..40%..50%.......................... ] 52% 25:03:07
</span></code></pre>
<p>I did a little research that suggests that a single wipe is sufficient to prevent data recovery on
modern disks, so the DoD 7-pass I used might seem like overdoing it, but since I’m throwing the disk
out because it was causing write errors I am also using this as a last benchmark to see if it would
trash the disk completely.</p>
OnePlus One Review2015-01-22T21:17:45+00:002015-01-22T21:17:45+00:00https://neverpanic.de/blog/2015/01/22/oneplus-one-review/<div>
<img
class="center"
src="/processed_images/9a65b5cdd1959eef00.jpg"
alt="OnePlus logo on the OnePlus Oneʼs packaging"
width="790"
height="228"
/>
</div>
<p>The One by OnePlus is a flagship phone designed and produced by the Chinese startup OnePlus founded
in December 2013. Only a few months later, the company announced the phone in April 2014. The
astonishing speed can be explained if you know that the company’s founder, Pete Lau, previously was
Vice President at <a href="https://en.wikipedia.org/wiki/Oppo_Electronics" title="Wikipedia for Oppo Electronics">Oppo Electronics</a> and is no newcomer to the smartphone business.</p>
<p>The phone’s specs are clearly targeted at the high-end market. For example, it features a 2.5 GHz
quad-core CPU, 3 gigabytes of DDR3 RAM, a 1080p IPS display and a 3100 mAh battery. The official
website has the <a href="https://oneplus.net/one#specifications" title="OnePlus One Specifications">details</a> – there really is no point in repeating all of them here.</p>
<h2 id="i-swear-it-s-that-large">I swear, it’s <em>that</em> large</h2>
<span class="pullquote-right" data-pullquote="My biggest concern when ordering the phone was its size.">
<p>There obviously already is a myriad of reviews on the OnePlus One (for example on YouTube), so I’ll
just skip ahead to the points that are relevant to me as computer scientist and the features that
surprised me. My biggest concern when ordering the phone was its size. At 5.5 inch, the screen
is huge, after all. I was pleasantly surprised to see the 15.3 x 7.6 cm phone fitting in my front
pocket comfortably. It does get a little cumbersome at times while driving, but that’s entirely
manageable and only manifests itself during long drives. On the contrary, it was interesting to see
how quickly I adjusted to the available screen real estate. Even before I actually switched my SIM
over to the new phone, I was asking myself why I bothered for so long with the vile 4.3 inch,
480x800 screen of my old HTC Desire HD.</p>
</span>
<span id="continue-reading"></span><h2 id="and-it-also-lasts-that-long">And it also lasts <em>that</em> long</h2>
<p>A definitive plus in my opinion is the One’s battery. The 3100 milliamp hours easily last for two
days, even when you’re bored and watch three hours of YouTube videos in Full HD before going to bed.
While I’m writing this, my phone is on battery for a little more than 13.5 hours and has still 81
% of its charge left. I made a couple of screenshots after a two-day stint to help you estimate your
range with the battery:</p>
<div>
<img
class="right"
src="/processed_images/e2859526b19f1cc200.png"
alt="Battery burndown after two days: mobile network and wifi always on, significant awake time"
width="395"
height="702"
/>
</div>
<p>There is one downside though: The battery is not customer-replaceable. I’m not racking my brain on
this at the moment. If a replacement becomes necessary in a few years I might just go ahead and do
the replacement on my own. It might not be as easy as plugging in a connector, but I refuse to
believe that it can’t be done with a little effort.</p>
<h2 id="it-s-going-to-satisfy-you">It’s going to satisfy you</h2>
<p>The most important part of the OnePlus One is obviously the software. The model of the OPO sold in
Europe ships with Cyanogenmod 11.0.0 M8 which is based on Android 4.4.4. It comes with all the
features you expect from Cyanogenmod like a plethora of customizable settings and Privacy Guard. The
One offers a choice between hardware buttons at the bottom of the screen and software-painted
on-screen buttons if you prefer those. The phone does not come with a music player, but features
a pretty nice camera app. It supports theming and comes with a theme store, but to be honest the
themes I’ve tried did not look very good. Even the one you can choose to enable during the phone’s
initial setup does weird things to your app icons, and to make things worse, I couldn’t get it
switched back to the normal icon set. Google apps are pre-flashed and cannot be deleted from the
phone, so if that’s an issue for you, you will have to flash a custom ROM or delete them using ROM
modification software.</p>
<p>Speaking of custom ROMs, getting root on the One was actually pretty easy. Besides the usual
<code>fastboot oem unlock</code> that will trigger a factory reset of your phone and allow you to flash
arbitrary ROMs there also is a way to <a href="http://forum.xda-developers.com/oneplus-one/development/mod-reset-unlock-tamper-bit-t2820912" title="Script to set the unlock and tamper bits of the One">set the tamper and unlock bits directly</a>.
<span class="pullquote-left" data-pullquote="The only thing I really needed to root for is ad
blocking.">
<p>This allegedly allows you to unlock the boot loader without data loss (I have not verified this).
While I’m usually cautious with stuff from xda developers, this script is actually really easy to
review, and I encourage you to do so. If you mess up, there is documentation on how to
<a href="http://forum.xda-developers.com/oneplus-one/general/guide-return-opo-to-100-stock-t2826541" title="Guide to restoring your One to the factory image">reinstall the OEM image</a>. Since the original software is already very good I even
considered not rooting the phone at all. The only thing I really needed to root for is ad
blocking.</p>
</span>
</p>
<p>One of the selling points of the phone for me definitely was the Cyanogenmod-based software. I hope
this will prevent the <a href="http://en.wikipedia.org/wiki/HTC_Desire_HD#Software" title="Wikipedia for the Desire HD; read the last few paragraphs on the Android 4.0 update">update disaster I had with HTC’s Desire HD</a>, which was never
updated to Android 4 despite continued promises by HTC to do so. This experience will keep me off
HTC phones forever and left me wary of anything but Google’s Nexus phones until now.</p>
<p>Looking at the responsiveness leaves nothing to desire. The Oneplus One is blazingly fast at tasks
that took my previous phone up to twenty seconds, such as getting a list of all sharing options
offered by the installed apps. Here’s a short video to give you an idea. Note that I have encryption
enabled on my phone – it might be even faster without that.</p>
<video preload="metadata" controls="" width="360" height="640">
<source src="/blog/2015/01/22/oneplus-one-review//2015-01-22-oneplus-one-review-responsiveness.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
</video>
<h2 id="was-it-good-for-you-too">Was it good for you, too?</h2>
<p>There is a lot more to say about the OnePlus One than I can cover here. Please check other reviews
or ask specific questions in the comments if you want to know more about any of the following
topics:</p>
<ul>
<li>camera image and video quality, slow motion filming</li>
<li>look and feel, especially of the back side</li>
<li>storage and the lack of an SD card slot</li>
<li>radio and call quality</li>
<li>audio quality</li>
<li>display quality</li>
<li>gesture shortcuts</li>
<li>hardware button location and impression</li>
<li>weight</li>
</ul>
<p>Overall, I can definitely recommend the OnePlus One and would buy it again. Please drop me a message
if you’re interested in an invite and I’ll try to think of you when I get some.</p>
Goodbye University, Hello Professional Life2014-08-07T00:06:58+00:002014-08-07T00:06:58+00:00https://neverpanic.de/blog/2014/08/07/goodbye-university/<div>
<img
class="right"
src="/processed_images/84295e941b37dbd800.jpg"
alt="Part of my university diploma."
width="395"
height="296"
/>
</div>
<p>A period of my life is coming to an end. Yesterday’s mail made that all the more obvious to me, since it contained my
university diploma. I have now officially graduated <a href="https://www.fau.de/" title="FAU Erlangen-Nuremberg">Friedrich-Alexander-University of Erlangen-Nuremberg</a> with
a Master’s degree in computer science. This is reason for celebration, especially since I managed to pass with
distinction, but it is also an opportunity to look back. Since I will not stay at university or in Erlangen, graduation
comes with a farewell.</p>
<p>I have enjoyed the last few years in Erlangen, especially at the <a href="https://www4.cs.fau.de/" title="Distributed Systems and Operating Systems">System Software Group</a> and its
<a href="https://www4.cs.fau.de/Research/KESO/" title="KESO: A Multi-JVM for Deeply Embedded Real-Time Systems">KESO Research Project</a> where I wrote my Master’s thesis on
“<a href="/downloads/documents/lang-14-mt.pdf" title="PDF, 723K">Compiler-Assisted Memory Management Using Escape Analysis in the KESO JVM</a>”.
However, in the last few months in Erlangen I’ve realized that it was time to move on and seek new challenges. And
I have.</p>
<p>On September 1<sup>st</sup> I will take up a job as “software integrator Linux” at <a href="http://www.bmw-carit.de/">BMW Car IT</a> in the city of
<a href="https://www.ulm.de/ulm/" title="Website of Ulm">Ulm</a>. I’m hoping my experience with continuous integration from KESO, package management, and build systems from
MacPorts may be helpful at my position. I’m really looking forward to working for BMW and moving to Ulm, and what I’ve
seen so far has been fantastic! :-)</p>
<p>Off to pastures new!</p>
What's New in MacPorts 2.3.02014-05-24T13:52:01+00:002014-05-24T13:52:01+00:00https://neverpanic.de/blog/2014/05/24/whats-new-in-macports-2-3-0/<p>MacPorts 2.3.0 has been <a href="https://lists.macosforge.org/pipermail/macports-announce/2014-May/000029.html">released</a>. But what’s new for users, and why should they use the new features? </p>
<p>This release contains a lot of changes under the hood that users probably won’t notice. For example, MacPorts no longer
uses the system-provided version of Tcl, but ships its own copy. That might seem like a step backward at the first
glance, but simplifies compatibility with older systems such as Tiger or Leopard (hello, PPC users), allows us to clean
up some of the cruft in the codebase and fix some long-standing issues like signal handling in future releases.</p>
<p>Another change most users won’t notice is the use of HTTP pipelining (I know, I know, what took us so long?), which
should be beneficial especially when downloading a lot of binary packages from our mirrors. Also related to downloads,
but very much noticeable are the new progress bars. Previously download progress information was only available when run
in verbose mode, but 2.3.0 comes with a nice progress indicator for downloads taking longer than a few seconds. You’ll
also see the same progress bar in rev-upgrade, which previously indicated its progress using a simple percentage number.</p>
<p>One of the changes I’ve been waiting for (and working on) is “trace mode”. Trace mode is a poor man’s sandbox initially
<a href="https://darwinbuild.macosforge.org/trac/browser/trunk/darwintrace/darwintrace.c">developed for the darwinbuild project</a> at Apple. It is based on library preloading, a technique known from
Linux systems using the environment variable <code>LD_PRELOAD</code>. That makes it inherently insecure, but since
security (i.e. protection against malicious attackers) has never been a goal for this sandbox, that’s not critical.
Trace mode adjusts the environment of a build in MacPorts by hiding all files that shouldn’t be there in a vanilla
installation of OS X and files in the MacPorts prefix that aren’t installed by a dependency of the current port. Trace
mode is a great tool for both port authors and users: Missing dependencies are easily identified and files in
<code>/usr/local</code> can no longer interfere with a MacPorts build with trace mode enabled. This last point is
especially important since lots of third party installers and other package managers (looking at you, homebrew) install
files in <code>/usr/local</code>. The next time a port fails to build for you, clean and re-try with <code>port
-t</code> instead.</p>
<p>Other minor, but helpful new features include a check for the presence of the Xcode Command Line Tools and Xcode license
agreement acceptance and a helpful new overview for the select feature at <code>port select --summary</code>.</p>
Constant Resyncs with Windows 7 Software RAID2014-04-19T21:00:20+00:002014-04-19T21:00:20+00:00https://neverpanic.de/blog/2014/04/19/constant-resyncs-with-windows-7-software-raid/<p>Despite my <a href="/blog/2008/11/30/university-and-macbook-pro/">switch to a MacBook Pro</a> almost six years ago I still have a Windows box I occasionally
use, mostly when I forget to bring my MacBook’s PSU (which happens surprisingly often, despite having two of them for
exactly that reason).</p>
<p>Since disk space on rotational drives is cheap these days I switched to a RAID 1 configuration when I last upgraded the
hardware in said computer. I went with two Western Digital WD20EARX 2 TB drives and first tried my mainboard’s fake RAID
(AMD RAIDXpert on an AMD SB710 chipset). Long story short, I was unsatisfied with the performance and afraid of data
loss in case my mainboard dies and I couldn’t get one with the same chipset.</p>
<p>I’ve seen software RAID on Linux and it was working much better for me than my mainboard’s attempt at it, so I figured
I’d try the software RAID implemented in Windows 7 (Professional, Enterprise and Ultimate only). Simple to setup and
with acceptable sync speeds I thought I had found what I was looking for – but then it seemed every time I rebooted, the
disk array would be inconsistent and resync from scratch. Needless to say, performance was plummeting. The machine would
hang for seconds waiting for I/O and every reboot would make it all start over. Even worse, the resync couldn’t be
aborted and the disk array couldn’t be disbanded either (who at Microsoft thought that was a good idea?).</p>
<p>Turns out the culprit was a known one. <a href="http://support.microsoft.com/kb/2913050/en-us">KB 2913050</a> says “Mirrored RAID volumes report Resynching status after you
restart Windows 7 […]”, and it would happen after each hotfix package installation, so for infrequently used computers
basically every (re-)boot. I especially liked the “resolution”:</p>
<blockquote>
<p>Microsoft is aware of this issue and intends to address it in a future release of Windows.</p>
</blockquote>
<p>Translation:</p>
<blockquote>
<p>It’s broken, but we’re not going to bother fixing it in Windows 7. Give us some money, if you want RAID support.</p>
<p>Or you could set the following magic registry keys, but we’re not going to tell you what they do, how they affect the
Volume Shadow Copy Service or why we’re not setting them with a hotfix for everybody.</p>
</blockquote>
<p>Please, somebody remind me why I thought Microsoft had a good reputation for their support of business-grade software…</p>
Autoconf: AC_CONFIG_SUBDIRS with Custom Flags for Subprojects2014-04-10T20:07:25+00:002014-04-10T20:07:25+00:00https://neverpanic.de/blog/2014/04/10/autoconf-ac-config-subdirs-with-custom-flags-for-subprojects/<p>Starting with version 2.3.0, <a href="https://www.macports.org/">MacPorts</a> will use its own copy of <a href="https://www.tcl.tk/">Tcl</a> rather than relying on the Tcl shipped by
Apple with OS X. Since MacPorts still works on versions of OS X down to Tiger that <a href="https://trac.macports.org/wiki/TclVersionInfo">only have Tcl 8.4</a>, all
features introduced in Tcl 8.5 have been off limits and workarounds had to be used. That turned out to be
unsatisfactory, especially avoiding <code>{*}</code> <a href="https://www.tcl.tk/cgi-bin/tct/tip/293">argument expansion</a> (with the ugly workaround of using eval).</p>
<p>The idea of bundling a private copy of Tcl <a href="https://lists.macosforge.org/pipermail/macports-dev/2013-July/023224.html">first came up</a> in July 2013 on the macports-dev mailing list,
originally in the context of the Apple distribution of Tcl changing in OS X Mavericks in a way that would no longer
allow MacPorts to build from source if the optional Command Line Tools package wasn’t installed.</p>
<h2 id="the-problem">The Problem</h2>
<p>MacPorts uses GNU autoconf in its build system. GNU autoconf supports bundling dependencies in subdirectories using the
<code>AC_CONFIG_SUBDIRS</code> macro – but it wasn’t sufficient for two reasons:</p>
<ul>
<li>The Tcl configure script creates a file that is needed by MacPorts’ configure to find the correct Tcl interpreter and
build setup. <code>AC_CONFIG_SUBDIRS</code> will delay configuring subpackages to the very end, but we needed it to be done
earlier.</li>
<li><code>AC_CONFIG_SUBDIRS</code> will always pass the same arguments given to the main configure script, which includes the prefix
setting. That would install our local copy of Tcl to a location that’s being used by the MacPorts tcl port (which is
version 8.6 and at the moment incompatible with some of the MacPorts code).</li>
</ul>
<p>There have been a few attempts at solving similar problems, <a href="http://lists.gnu.org/archive/html/autoconf/2011-04/msg00006.html">one of which</a> is a patch against
autoconf and has been sent to the autoconf mailinglist in April 2011, but was apparently never applied. I didn’t want to
require a patched autoconf to generate the MacPorts configure script either, so applying the patch was not an option.</p>
<h2 id="the-solution">The Solution</h2>
<p>An unsolved problem in a technology I haven’t used a lot yet? That sounded like a great opportunity to learn something
new – and so I wrote the missing macro, mostly by reading and copying the source of <code>AC_CONFIG_SUBDIRS</code> and adjusting it
where needed. I added extracting from a tarball so I didn’t have to commit the extracted Tcl sources.
<code>MP_CONFIG_TARBALL</code> (<a href="https://trac.macports.org/browser/trunk/base/aclocal.m4?rev=118736&marks=99-307#L95">source</a>) takes the path to a tarball, the directory that’s created by extracting the
tarball that contains the configure script and a list of configure parameters to pass to the subproject. Each given
configure parameter overrides those given on the main project’s command line and preserves those that have not been
overwritten like <code>AC_CONFIG_SUBDIRS</code> would.</p>
Downloading Google Web Fonts for Local Hosting2014-03-19T19:28:14+00:002014-03-19T19:28:14+00:00https://neverpanic.de/blog/2014/03/19/downloading-google-web-fonts-for-local-hosting/<p>I am trying to make this blog completely tracker-free. To achieve that, I need to avoid fetching resources from any
third-party servers, such as Google Web Fonts. There are <a href="http://ijotted.blogspot.de/2012/05/self-hosting-web-fonts-for-use-on-your.html">some</a> <a href="http://ijotted.blogspot.de/2012/05/download-eot-ttf-woff-formats-of-font.html">articles</a>
discussing how to download the different formats required for cross-browser compatibility, but manually downloading
a series of files using different browsers isn’t what I had in mind – I’m a computer science student after all. So
I toolchained it. Meet the Google Web Font download script (requires Bash 4.x), available from
<a href="https://github.com/neverpanic/google-font-download">its new home on GitHub</a>.</p>
<p>The script provides a command line interface to specify the fonts to be downloaded as given in the <code>?family=</code> parameter
of the URL generated by Google’s Web Font service. It uses <code>curl</code> to emulate different user agents (since the CSS file
returned by Google’s setup depends on it), downloads the font files into the current directory and generates a CSS file
that provides the font definition using <a href="http://www.fontspring.com/blog/the-new-bulletproof-font-face-syntax">Fontspring’s Bulletproof @Font-face Syntax</a>. The generated file is
a drop-in replacement for the file previously fetched from Google’s servers. Since it includes all font files and
doesn’t depend on the user agent it might not be as efficient as the original, though (I have not actually verified
this). If efficiency is key for you, you may want to limit the number of formats you provide, because
<a href="http://caniuse.com/#search=woff">WOFF support</a> is pretty widespread in modern browsers.</p>
<p>Don’t forget to <a href="http://stackoverflow.com/questions/7415640/correct-apache-addtype-directives-for-font-mime-types">configure the MIME types for the font files</a> on your web server.</p>
<h2 id="changelog">Changelog</h2>
<ul>
<li>Version 1.2, 2015-06-13
<ul>
<li>Add getopt interface</li>
<li>Support font subsets, <a href="https://gist.github.com/pointergr/d4cb1609ff03469af011">modifications by Thomas Papamichail</a></li>
</ul>
</li>
<li>Version 1.1.1, 2015-04-09
<ul>
<li>Switch user agent to IE 8 for WOFF to fix problems</li>
</ul>
</li>
<li>Version 1.1, 2014-06-21
<ul>
<li>Remove colons and spaces from file names for Windows compatibility, <a href="https://twitter.com/campino2k/status/449274390916923392">patch from campino2k</a></li>
<li>Add check for Bash version, 4.x is required</li>
<li>Correctly handle fonts without a local PostScript name</li>
<li>Change <code>format('ttf')</code> to <code>format('truetype')</code> in CSS output</li>
<li>Add license header and comments</li>
<li>Added sed extended regex flag detection</li>
</ul>
</li>
<li>Version 1.0, 2014-03-19</li>
</ul>
Shiny New Blog2014-03-19T16:01:43+00:002014-03-19T16:01:43+00:00https://neverpanic.de/blog/2014/03/19/shiny-new-blog/<p>This blog has been overhauled completely. My old website had not been updated in three years and was beginning to fall
apart in some places. Note that I have not completely moved all content from the old site, which made this move easier
and quicker.</p>
<p>First things first: If you were subscribed to the old blog’s RSS feed, please check and update, if necessary, your feed
URL from <code>http://www.neverpanic.de/blog/atom/</code> or <code>http://www.neverpanic.de/blog/rss2/</code> to
<code>https://neverpanic.de/atom.xml</code>. The old URLs now are permanent redirects to the correct new URL, so if you see this
post in your feed reader you’re probably fine and don’t have to touch anything.</p>
<p>This blog was previously using the free 1.9.x version of <a href="http://en.wikipedia.org/wiki/ExpressionEngine" title="ExpressionEngine on Wikipedia">ExpressionEngine</a> from EllisLabs called ExpressionEngine
Core. The software had been giving me update notices for quite a while, but when I wanted to update it to the latest
version I noticed the free version had been pulled from the downloads page. It’s back in version 2.x now, but I’m not
going to make the same mistake of relying on closed source software again. I switched to <a href="http://octopress.org/">Octopress</a>, a static site
generator based on <a href="http://jekyllrb.com/">Jekyll</a>. I don’t particularly like software written in Ruby because most Ruby projects aren’t even
trying to get into the standard Linux/BSD/OS X package managers, and this one is no exception, but it works reasonably
well for me. I didn’t follow the <a href="http://octopress.org/docs/setup/">installation instructions</a> where it says to use rbenv or RVM but
just installed ruby 1.9 and bundler from MacPorts and ran</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>bundle install --binstubs --path</span><span style="color:#ececec;">=</span><span>vendor/bundle
</span></code></pre>
<p>That puts all the dependencies self-contained into <code>vendor/bundle</code> (no, I don’t want to install gems as root into system
locations or install yet another package manager) and creates a series of wrapper scripts for binaries in <code>./bin</code>. Now
I just have to remember to use <code>bin/rake $command</code> instead of <code>rake $command</code> to work with Octopress.</p>
<p>I chose to selectively pick the posts I moved from the previous blog software and completely removed the separate “About
me” and “Portfolio” sections. I felt most of the stuff I removed didn’t represent very well what I’m currently doing,
since I’m not really into websites anymore. I made sure all old URLs either redirect to the new versions or return
a proper <a href="/410-gone/">HTTP 410 Gone</a> page.</p>
<span id="continue-reading"></span>
<p>While converting my posts, I noticed that Jekyll’s <code>post_url</code> tag to link to other posts independent of their URL didn’t
work for Octopress. Googling suggested a solution that provided a Octopress-compatible <code>post_url</code> tag which would
however no longer work because it conflicts with the tag provided by Jekyll. I fixed that by renaming the tag to
<code>opost_url</code>:</p>
<pre data-lang="ruby" style="background-color:#393939;color:#dedede;" class="language-ruby "><code class="language-ruby" data-lang="ruby"><span style="color:#fed6af;">module </span><span style="font-weight:bold;color:#ff8000;">Jekyll
</span><span>
</span><span> </span><span style="color:#fed6af;">class </span><span style="color:#f4a020;">OctoPostComparer
</span><span> MATCHER </span><span style="color:#ececec;">= </span><span style="color:#d6d6d680;">/</span><span style="color:#c76f41;">^(.+</span><span style="font-weight:bold;color:#ff8080;">\/</span><span style="color:#c76f41;">)*(</span><span style="font-weight:bold;color:#ff8080;">\d</span><span style="color:#c76f41;">+-</span><span style="font-weight:bold;color:#ff8080;">\d</span><span style="color:#c76f41;">+-</span><span style="font-weight:bold;color:#ff8080;">\d</span><span style="color:#c76f41;">+)-(.*)$</span><span style="color:#d6d6d680;">/
</span><span>
</span><span> </span><span style="color:#fed6af;">attr_accessor </span><span style="font-weight:bold;color:#cc9495;">:date</span><span>, </span><span style="font-weight:bold;color:#cc9495;">:slug
</span><span>
</span><span> </span><span style="color:#fed6af;">def </span><span style="color:#fffd87;">initialize</span><span>(name)
</span><span> _, _, date, slug </span><span style="color:#ececec;">= *</span><span style="color:#fffd87;">name</span><span>.</span><span style="color:#fffd87;">match</span><span>(</span><span style="color:#d6d6ae;">MATCHER</span><span>)
</span><span> </span><span style="color:#fed6af;">@slug </span><span style="color:#ececec;">=</span><span> slug
</span><span> </span><span style="color:#fed6af;">@date </span><span style="color:#ececec;">= </span><span style="color:#f4a020;">Time</span><span>.parse(date)
</span><span> </span><span style="color:#fed6af;">end
</span><span>
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Octopress creates dates with hours and crap in them which are not in the file name
</span><span> </span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> so just compare based on the date parts
</span><span> </span><span style="color:#fed6af;">def </span><span style="color:#fffd87;">==</span><span>(post)
</span><span> cmp </span><span style="color:#ececec;">= </span><span>self.date.strftime(</span><span style="color:#d6d6d680;">'</span><span style="font-weight:bold;color:#cc9495;">%Y</span><span style="color:#d68686;">-</span><span style="font-weight:bold;color:#cc9495;">%m</span><span style="color:#d68686;">-</span><span style="font-weight:bold;color:#cc9495;">%d</span><span style="color:#d6d6d680;">'</span><span>) </span><span style="color:#ececec;"><=></span><span> post.date.strftime(</span><span style="color:#d6d6d680;">'</span><span style="font-weight:bold;color:#cc9495;">%Y</span><span style="color:#d68686;">-</span><span style="font-weight:bold;color:#cc9495;">%m</span><span style="color:#d68686;">-</span><span style="font-weight:bold;color:#cc9495;">%d</span><span style="color:#d6d6d680;">'</span><span>)
</span><span> </span><span style="color:#fed6af;">if </span><span style="font-weight:bold;color:#87d6d5;">0 </span><span style="color:#ececec;">==</span><span> cmp
</span><span> cmp </span><span style="color:#ececec;">= </span><span>self.slug </span><span style="color:#ececec;"><=></span><span> post.slug
</span><span> </span><span style="color:#fed6af;">end
</span><span> </span><span style="color:#fed6af;">return </span><span style="font-weight:bold;color:#87d6d5;">0 </span><span style="color:#ececec;">==</span><span> cmp
</span><span> </span><span style="color:#fed6af;">end
</span><span> </span><span style="color:#fed6af;">end
</span><span>
</span><span> </span><span style="color:#fed6af;">class </span><span style="color:#f4a020;">OctoPostUrl </span><span>< </span><span style="color:#d78d1b;">Liquid::Tag
</span><span> </span><span style="color:#fed6af;">def </span><span style="color:#fffd87;">initialize</span><span>(tag_name, post, tokens)
</span><span> </span><span style="color:#fed6af;">super
</span><span> </span><span style="color:#fed6af;">@orig_post </span><span style="color:#ececec;">=</span><span> post.strip
</span><span> </span><span style="color:#fed6af;">@post </span><span style="color:#ececec;">= </span><span style="color:#f4a020;">OctoPostComparer</span><span>.</span><span style="color:#fed6af;">new</span><span>(</span><span style="color:#fed6af;">@orig_post</span><span>)
</span><span> </span><span style="color:#fed6af;">end
</span><span>
</span><span> </span><span style="color:#fed6af;">def </span><span style="color:#fffd87;">render</span><span>(context)
</span><span> site </span><span style="color:#ececec;">=</span><span> context.registers[</span><span style="font-weight:bold;color:#cc9495;">:site</span><span>]
</span><span>
</span><span> site.posts.each </span><span style="color:#fed6af;">do </span><span>|p|
</span><span> </span><span style="color:#fed6af;">if @post </span><span style="color:#ececec;">== </span><span style="color:#fffd87;">p
</span><span> </span><span style="color:#fed6af;">return </span><span style="color:#fffd87;">p</span><span>.url
</span><span> </span><span style="color:#fed6af;">end
</span><span> </span><span style="color:#fed6af;">end
</span><span>
</span><span> </span><span style="color:#fffd87;">puts </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">ERROR: opost_url: </span><span style="font-weight:bold;color:#ff8080;">\"</span><span>#{</span><span style="color:#fed6af;">@orig_post</span><span>}</span><span style="font-weight:bold;color:#ff8080;">\"</span><span style="color:#d68686;"> could not be found</span><span style="color:#d6d6d680;">"
</span><span>
</span><span> </span><span style="color:#fed6af;">return </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">#</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fed6af;">end
</span><span> </span><span style="color:#fed6af;">end
</span><span style="color:#fed6af;">end
</span><span>
</span><span style="color:#f4a020;">Liquid</span><span>::</span><span style="color:#f4a020;">Template</span><span>.register_tag(</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">opost_url</span><span style="color:#d6d6d680;">'</span><span>, </span><span style="color:#f4a020;">Jekyll</span><span>::OctoPostUrl)
</span></code></pre>
<p>While writing the “About Me” section for the sidebar I wanted a way to automatically update my age without manually
changing the file every year. So I wrote another plugin to achieve that:</p>
<pre data-lang="ruby" style="background-color:#393939;color:#dedede;" class="language-ruby "><code class="language-ruby" data-lang="ruby"><span style="color:#fed6af;">module </span><span style="font-weight:bold;color:#ff8000;">Jekyll
</span><span>
</span><span> </span><span style="color:#fed6af;">class </span><span style="color:#f4a020;">MyAge </span><span>< </span><span style="color:#d78d1b;">Liquid::Tag
</span><span> </span><span style="color:#fed6af;">def </span><span style="color:#fffd87;">initialize</span><span>(tag_name, post, tokens)
</span><span> </span><span style="color:#fed6af;">super
</span><span> </span><span style="color:#fed6af;">end
</span><span>
</span><span> </span><span style="color:#fed6af;">def </span><span style="color:#fffd87;">render</span><span>(context)
</span><span> site </span><span style="color:#ececec;">=</span><span> context.registers[</span><span style="font-weight:bold;color:#cc9495;">:site</span><span>]
</span><span>
</span><span> year </span><span style="color:#ececec;">=</span><span> site.config[</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">age</span><span style="color:#d6d6d680;">'</span><span>][</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">bday_year</span><span style="color:#d6d6d680;">'</span><span>]
</span><span> month </span><span style="color:#ececec;">=</span><span> site.config[</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">age</span><span style="color:#d6d6d680;">'</span><span>][</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">bday_month</span><span style="color:#d6d6d680;">'</span><span>]
</span><span> day </span><span style="color:#ececec;">=</span><span> site.config[</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">age</span><span style="color:#d6d6d680;">'</span><span>][</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">bday_day</span><span style="color:#d6d6d680;">'</span><span>]
</span><span>
</span><span> age </span><span style="color:#ececec;">=</span><span> site.time.year </span><span style="color:#ececec;">-</span><span> year
</span><span> </span><span style="color:#fed6af;">if</span><span> site.time </span><span style="color:#ececec;">- </span><span style="color:#f4a020;">Time</span><span>.</span><span style="color:#fed6af;">new</span><span>(site.time.year, month, day) </span><span style="color:#ececec;">< </span><span style="font-weight:bold;color:#87d6d5;">0
</span><span> age </span><span style="color:#ececec;">-= </span><span style="font-weight:bold;color:#87d6d5;">1
</span><span> </span><span style="color:#fed6af;">end
</span><span>
</span><span> </span><span style="color:#fed6af;">return</span><span> age
</span><span> </span><span style="color:#fed6af;">end
</span><span> </span><span style="color:#fed6af;">end
</span><span style="color:#fed6af;">end
</span><span>
</span><span style="color:#f4a020;">Liquid</span><span>::</span><span style="color:#f4a020;">Template</span><span>.register_tag(</span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">myage</span><span style="color:#d6d6d680;">'</span><span>, </span><span style="color:#f4a020;">Jekyll</span><span>::MyAge)
</span></code></pre>
<p>To use it, add an entry to your <code>_config.yml</code> as follows:</p>
<pre data-lang="yaml" style="background-color:#393939;color:#dedede;" class="language-yaml "><code class="language-yaml" data-lang="yaml"><span style="font-weight:bold;color:#fed6af;">age</span><span>:
</span><span> </span><span style="font-weight:bold;color:#fed6af;">bday_year</span><span>: </span><span style="font-weight:bold;color:#87d6d5;">1990
</span><span> </span><span style="font-weight:bold;color:#fed6af;">bday_month</span><span>: </span><span style="font-weight:bold;color:#87d6d5;">1
</span><span> </span><span style="font-weight:bold;color:#fed6af;">bday_day</span><span>: </span><span style="font-weight:bold;color:#87d6d5;">1
</span></code></pre>
<p>I’m planning to add comments using <a href="http://blog.posativ.org/2013/isso-ich-schrei-sonst/">Isso</a> later, but I wanted to get this out ASAP and improve the site gradually
rather than making it perfect but a never-ending project. For now, I welcome any feedback via email.</p>
Chaos Communication Camp Badges2011-08-11T01:00:41+00:002011-08-11T01:00:41+00:00https://neverpanic.de/blog/2011/08/11/chaos-communication-camp-badges/<p>I’m currently attending <a href="http://events.ccc.de/camp/2011/">Chaos Communication Camp 2011</a> in Finowfurt near Berlin.</p>
<p>Apart from the cool location within an <a href="http://maps.google.de/maps?q=Luftfahrtmuseum+Finowfurt,+Museumsstra%C3%9Fe&hl=en&ie=UTF8&sll=51.151786,10.415039&sspn=17.035085,46.538086&vpsrc=0&t=h&z=16" title="aviation museum Finowfurt on Google Maps">used-to-be military airport open-air aviation museum</a>, the coolest thing
so far definitely is the electronic badge handed out at the entrance: The <a href="http://r0ket.badge.events.ccc.de/">r0ket</a>, an ARMv7 micro controller with
back-lit LED-display, several LEDs (of which one is also used as a light sensor to determine whether it’s night and the
badge display needs back light), a multi-way jog, a rechargeable battery and some extension-connectors.</p>
<p>The badge also features mesh-networking capabilities, although the firmware for that does not seem very stable or
reliably usable at the moment. I will definitely be keeping this <del>tool</del> <ins>toy</ins> – there’s no
better name-tag than this, I guess.</p>
<div>
<img
class="center"
src="/processed_images/a88622ba4c78be1500.jpg"
alt="A rocket-shaped PCB with a display"
width="790"
height="1320"
/>
</div>
GSoC Welcome Package2011-05-27T13:18:24+00:002011-05-27T13:18:24+00:00https://neverpanic.de/blog/2011/05/27/gsoc-welcome-package/<p>I have been accepted into <a href="http://code.google.com/soc/">Google’s Summer of Code program</a> this year to work for the <a href="https://www.macports.org/">MacPorts</a>
project. In a nutshell, I will be writing an equivalent to Gentoo’s revdep-rebuild – if you’re interested in the
details, <a href="http://www.google-melange.com/gsoc/proposal/public/google/gsoc2011/neverpanic/5668600916475904" title="my public proposal at the GSoC website">check out my proposal</a>.</p>
<p>At the beginning of GSoC, every student receives a welcome package from Google,
with a few goodies and a prepaid Visa card through which they will be paid. I
have received this package yesterday and want to share my unboxing experience
;)</p>
<span id="continue-reading"></span><div>
<img
class="center"
src="/processed_images/e161abe72c8196a200.jpg"
title="The box before opening"
alt="FedEx shipping box"
width="790"
height="472"
/>
</div>
<p>This is what the package looked like from the outside. Welcome packages are shipped by FedEx; in my case they left it
with a neighbour, since I wasn’t at home when it arrived.</p>
<div>
<img
class="center"
src="/processed_images/4b091e3a0905a9b700.jpg"
title="The box contents"
alt="Several Google-branded goodies, paperwork"
width="790"
height="472"
/>
</div>
<p>The package contains a pen, a notebook, a GSoC sticker that presumably glows in the dark (I’ve only read that, not
tested it on my own), a welcome letter from Google (on US Letter paper, which feels weird to me being in Europe) and the
payment card along with some notes and ToS for it.</p>
<p>I particulary like the payment card itself, because it just feels awesome to have a Visa card with Google’s colors and
the Google “g” on it:</p>
<div>
<img
class="center"
src="/processed_images/02d6a384cef94c3400.jpg"
title="Yes, I did shoop my card number. What did you expect?"
alt="VISA credit card in Google colors with the Google g"
width="790"
height="472"
/>
</div>
Christ's Resurrection… in Little Switzerland2010-07-31T00:55:11+00:002010-07-31T00:55:11+00:00https://neverpanic.de/blog/2010/07/31/christs-resurrection-dot-dot-dot-in-little-switzerland/<p>Seen in a village in <a href="http://en.wikipedia.org/wiki/Little_Switzerland_%28Germany%29" title="Little Switzerland on Wikipedia">Little Switzerland</a>:</p>
<div>
<img
class="center"
src="/processed_images/5f52a5710ff9e78900.jpg"
alt="Sign saying: Christs Resurrection. No obvious context."
width="790"
height="592"
/>
</div>
<p>The text reads “Christ’s Resurrection”. I had a thorough look, but couldn’t spot what that sign was trying to point me
to… maybe it’s stating the obvious: Jerusalem is located in Little Switzerland? ;)</p>
whatthecommit.com Git Hook2010-07-03T00:00:00+00:002010-07-03T00:00:00+00:00https://neverpanic.de/blog/2010/07/03/whatthecommit-dot-com-git-hook/<p><a href="http://whatthecommit.com">whatthecommit.com</a> generates commit messages for the lazy… but being one of the laziest people, this isn’t just
easy enough!</p>
<p>I currently use <a href="http://gitscm.org/">Git</a> for most of my version control needs. I’m keeping all of my hand-ins for university under
version control to be able to sync them between university and my laptop easily and to make it easy for others to
contribute (and sometimes they actually do!). But those of you using version control systems know what the biggest
problem with version control is: Thinking of a commit message. Wait no moar! The ultimate solution is here!</p>
<p><a href="http://whatthecommit.com">whatthecommit.com</a> is a website that provides you with a fresh commit message every time you load it. So all you
have to do, is copy and paste the line into your commit window. Still too much work? That’s why git comes with hook
scripts. Paste the following code in <code>.git/hooks/prepare-commit-msg</code> in your working copy and make the file executable
and you’ll be provided with a wonderful commit message every time you type <code>git commit</code> automatically!</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> A hook script to prepare the commit log message.
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> Called by "git commit" with the name of the file that has the
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> commit message, followed by the description of the commit
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> message's source. The hook's purpose is to edit the commit
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> message file. If the hook fails with a non-zero status,
</span><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> the commit is aborted.
</span><span>
</span><span style="color:#fed6af;">case </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">2,</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">3</span><span style="color:#d6d6d680;">" </span><span style="color:#fed6af;">in
</span><span> ,</span><span style="color:#ececec;">|</span><span>template,</span><span style="color:#fed6af;">)
</span><span> line</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">( curl -L http://whatthecommit.com/ </span><span style="font-weight:bold;color:#87d6d5;">2</span><span style="color:#ececec;">></span><span style="color:#d68686;">/dev/null </span><span style="color:#ececec;">| </span><span style="color:#d68686;">grep -Po </span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">(?<=\<p\>).*$</span><span style="color:#d6d6d680;">' </span><span style="color:#d68686;">)
</span><span> file</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">( sed </span><span style="color:#d6d6d680;">'</span><span style="color:#d68686;">1d</span><span style="color:#d6d6d680;">' "</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{1}</span><span style="color:#d6d6d680;">" </span><span style="color:#d68686;">)
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{line}</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{1}</span><span style="color:#d6d6d680;">"
</span><span> </span><span style="color:#fffd87;">echo </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{file}</span><span style="color:#d6d6d680;">" </span><span style="color:#ececec;">>> </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">{1}</span><span style="color:#d6d6d680;">"
</span><span> ;;
</span><span> </span><span style="color:#ececec;">*</span><span style="color:#fed6af;">) </span><span>;;
</span><span style="color:#fed6af;">esac
</span></code></pre>
Windows 7 on MSI K9N Platinum (nForce 570 SLI)2009-12-26T02:41:29+00:002009-12-26T02:41:29+00:00https://neverpanic.de/blog/2009/12/26/windows-7-on-msi-k9n-platinum-nforce-570-sli/<p>Recently, a fresh Windows install was due on my home PC primarly used by my parents and my family after my
<a href="/blog/2008/11/30/university-and-macbook-pro/">switch to a MacBook</a> about a year ago.</p>
<p>However, whenever I tried to start the installation routine, the setup would boot into the “Windows is starting”-screen
with the nicely animated glowing 7-logo, but stay there indefinitely without an error message. When starting the setup
in safe mode, the setup would just hang after</p>
<pre style="background-color:#393939;color:#dedede;"><code><span>Loaded: \Windows\system32\drivers\disk.sys
</span></code></pre>
<p>I started searching the internet for similar problems and found a couple of recommendations related to nForce chipsets.
Some told you to disable your on-board LAN ports (which I’m not using anyway) and a lot of similar disable-some-hardware
tips, which, unfortunately, did not help at all. I had almost given up on Windows 7 and re-installed XP, when I decided
to try a BIOS-update as a last resort. After using MSI’s rather comfortable LiveUpdate – an Internet Explorer Active-X
plugin optimized for IE5 and 800x600 which works surprisingly well – the Windows 7 setup did work fine. It felt a little
weird when the BIOS update utility ran within Windows and does a hard-reset after updating the BIOS, but that’s probably
better than rebooting with a possibly non-working BIOS.</p>
Bon Appetit at the Erlangen Cafeteria2009-09-29T19:31:31+00:002009-09-29T19:31:31+00:00https://neverpanic.de/blog/2009/09/29/bon-appetit-at-the-erlangen-cafeteria/<p>It’s no secret the canteen on the Erlangen university’s southern campus is not the best – but I haven’t seen something
like <strong>that</strong> before.</p>
<div>
<img
class="center"
src="/processed_images/57a8753bd85ede1b00.jpg"
title="Bon Appetit!"
alt="Worm on a plate with noodles"
width="790"
height="592"
/>
</div>
<p>A little bit disgusting if you ask me – I was lucky, it wasn’t my meal, though…</p>
Subversion Property Copy2009-02-16T00:03:49+00:002009-02-16T00:03:49+00:00https://neverpanic.de/blog/2009/02/16/subversion-property-copy/<p>Although there are a <a href="http://www.lachoseinteractive.net/en/community/subversion/svnx/features/" title="svnX (free)">couple</a> <a href="http://versionsapp.com/" title="Versions">of</a> <a href="http://www.zennaware.com/cornerstone/" title="Cornerstone">Subversion</a> <a href="http://zigversion.com/" title="ZigVersion (free for students)">GUIs</a> for Macs, I usually use
the command line interface. I manage all source code I write during my studies using Subversion and usually add revision
number and date of last checkin to the file using the <code>svn:keywords</code>-Property. However, I always forget the set of
keywords I usually add: <code>Author Date Id Revision URL</code>. Unfortunately, there is no way to copy a property from one file
to another in the standard subversion binary. There is, however, a little shortcut:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span>svn propset </span><span style="color:#fed6af;">$</span><span>propertyName </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">`svn propget </span><span style="color:#fed6af;">$</span><span style="color:#d68686;">propertyName </span><span style="color:#fed6af;">$</span><span style="color:#d68686;">fromFileName`</span><span style="color:#d6d6d680;">" </span><span style="color:#fed6af;">$</span><span>toFileName
</span></code></pre>
<p>Typing this monster isn’t any userfriendly at all, though – a little <code>.bashrc</code> magic does the trick:</p>
<pre data-lang="bash" style="background-color:#393939;color:#dedede;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#a0cfa1;">#</span><span style="color:#87ae86;"> add svn propcopy
</span><span style="color:#fffb9d;">function </span><span style="color:#fffd87;">svn</span><span>() {
</span><span> </span><span style="color:#fed6af;">case </span><span style="color:#d6d6d680;">"</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">1</span><span style="color:#d6d6d680;">" </span><span style="color:#fed6af;">in
</span><span> pc</span><span style="color:#ececec;">|</span><span>propcopy</span><span style="color:#fed6af;">)
</span><span> propName</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">2
</span><span> fromFile</span><span style="color:#ececec;">=</span><span style="color:#fed6af;">$</span><span style="color:#d68686;">3
</span><span> </span><span style="color:#fffd87;">shift</span><span> 3
</span><span> `which svn` propset </span><span style="color:#fed6af;">$</span><span>propName </span><span style="color:#d6d6d680;">"</span><span style="color:#d68686;">`</span><span style="font-weight:bold;color:#ff8080;">\`</span><span style="color:#d68686;">which svn</span><span style="font-weight:bold;color:#ff8080;">\`</span><span style="color:#d68686;"> propget </span><span style="color:#fed6af;">$</span><span style="color:#d68686;">propName </span><span style="color:#fed6af;">$</span><span style="color:#d68686;">fromFile`</span><span style="color:#d6d6d680;">" </span><span style="color:#fed6af;">$</span><span>@
</span><span> ;;
</span><span> </span><span style="color:#ececec;">*</span><span style="color:#fed6af;">)
</span><span> `which svn` </span><span style="color:#fed6af;">$</span><span>@
</span><span> ;;
</span><span> </span><span style="color:#fed6af;">esac
</span><span>}
</span></code></pre>
<h3 id="update">Update</h3>
<p>Thanks to <a href="http://raim.codingfarm.de/" title="Rainer Müller">Raim</a> for the wildcard support. Using subversion auto-props is an option for files, but
unfortunately, <a href="http://subversion.tigris.org/issues/show_bug.cgi?id=1989" title="issue 1989 on auto-props on directories">auto-props don’t work on directories</a> yet.</p>
University & MacBook Pro2008-11-30T18:34:01+00:002008-11-30T18:34:01+00:00https://neverpanic.de/blog/2008/11/30/university-and-macbook-pro/<p>It’s been a while since my last update — not because there’s nothing to tell, but because I didn’t find any time to
write something down.</p>
<p>So why’s that? I’m attending university now: I’m in my first semester of computer science at the
<a href="https://fau.de/" title="Friedrich-Alexander-University of Erlangen-Nuremberg">University of Erlangen</a> and it’s been a tough start figuring out how things work at university, finding the
rooms, getting the work done, etc.</p>
<span id="continue-reading"></span>
<p>Another thing I desperately needed for university was a laptop, as I didn’t bother moving my home computer (it’s the
last one left at home for parents and everybody else anyway). It took me a good while to decide what kind of laptop to
get, but I finally chose a MacBook Pro with a whooping 4 Gigabytes of RAM because I didn’t want to fiddle with neither
Windows Vista nor XP anymore. Linux would have been an option, but having a GUI to configure your PC isn’t that bad
after all either. Lots of people buy Macs for their product design, but the point that convinced me was the possibility
to run every software that works on Linux on the Mac as well (you gotta love a real bash shell, not some emulated cr*p
on Windows). So far I’m very satisfied with my choice and the MacBook — I might blog a little bit more in detail about
that soon.</p>
<p>So far this was just a short update from me to let you know I’m still alive.</p>