neverpanic.de

Secure Erase on OS X El Capitan

| Comments

With the update to OS X El Capitan, Apple has rewritten Disk Utility. The pre-10.11 Disk Utility used to have an option to securely erase a disk – a feature I needed because I plan to throw a faulty disk away.

El Capitan Disk Utility erase dialog without security options button Now, Apple still documents the option in KB article PH22241, but has implemented code that hides the “Security Options” button in certain situations. Unfortunately, they did not document which conditions need to be fulfilled for the button to be shown, which leads to the situation that I do not see it on any of my disks. My guess would be that the option is not available for encrypted disks, but since I do not have any unencrypted drives I cannot verify that assumption.

Why would you wipe an encrypted disk?

For an encrypted volume, wiping the header that contains the master encryption key should be enough to ensure that no data can be recovered. Conveniently, Apple does not provide an option to wipe the volume’s encryption header and documentation on Apple’s CoreStorage format it scarce, which means I don’t know where the header actually is. So a full wipe it is.

Luckily just because the GUI does not support the feature anymore does not mean that it cannot be done. The diskutil command line tool still has a secureErase option that supports overwriting entire volumes. Because I was doing this with CoreStorage volumes, I first had to delete that volume before secureErase would unmount the physical disk below:

Deleting a CoreStorage volume
1
2
3
4
5
6
:) clemens@cBookPro:~$ diskutil cs deleteVolume CD3D75E0-F317-42B6-B44F-FDCB1A9448CD
The Core Storage Logical Volume UUID is CD3D75E0-F317-42B6-B44F-FDCB1A9448CD
Started CoreStorage operation on disk7 cTM
Unmounting disk7
Removing Logical Volume from Logical Volume Group
Finished CoreStorage operation on disk7 cTM

Once the logical volume was gone, I was able to start the wipe with diskutil secureErase:

Securely erasing a disk in OS X El Capitan
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
:) clemens@cBookPro:~$ diskutil secureErase
Usage:  diskutil secureErase [freespace] level MountPoint|DiskIdentifier|DeviceNode
Securely erases either a whole disk or a volume's freespace.
Level should be one of the following:
        0 - Single-pass zeros.
        1 - Single-pass random numbers.
        2 - US DoD 7-pass secure erase.
        3 - Gutmann algorithm 35-pass secure erase.
        4 - US DoE 3-pass secure erase.
Ownership of the affected disk is required.
Note: Level 2, 3, or 4 secure erases can take an extremely long time.
:( clemens@cBookPro:~$ diskutil secureErase 2 disk4
Started erase on disk4
Pass: 1
Pass: 2
Pass: 3
Pass: 4
[ - 0%..10%..20%..30%..40%..50%.......................... ] 52% 25:03:07

I did a little research that suggests that a single wipe is sufficient to prevent data recovery on modern disks, so the DoD 7-pass I used might seem like overdoing it, but since I’m throwing the disk out because it was causing write errors I am also using this as a last benchmark to see if it would trash the disk completely.

OnePlus One Review

| Comments

OnePlus logo on the OnePlus Oneʼs packaging

The One by OnePlus is a flagship phone designed and produced by the Chinese startup OnePlus founded in December 2013. Only a few months later, the company announced the phone in April 2014. The astonishing speed can be explained if you know that the company’s founder, Pete Lau, previously was Vice President at Oppo Electronics and is no newcomer to the smartphone business.

The phone’s specs are clearly targeted at the high-end market. For example, it features a 2.5 GHz quad-core CPU, 3 gigabytes of DDR3 RAM, a 1080p IPS display and a 3100 mAh battery. The official website has the details – there really is no point in repeating all of them here.

I swear, it’s that large

There obviously already is a myriad of reviews on the OnePlus One (for example on YouTube), so I’ll just skip ahead to the points that are relevant to me as computer scientist and the features that surprised me. My biggest concern when ordering the phone was its size. At 5.5 inch, the screen is huge, after all. I was pleasantly surprised to see the 15.3 x 7.6 cm phone fitting in my front pocket comfortably. It does get a little cumbersome at times while driving, but that’s entirely manageable and only manifests itself during long drives. On the contrary, it was interesting to see how quickly I adjusted to the available screen real estate. Even before I actually switched my SIM over to the new phone, I was asking myself why I bothered for so long with the vile 4.3 inch, 480x800 screen of my old HTC Desire HD.

Goodbye University, Hello Professional Life

| Comments

Part of my university diploma.

A period of my life is coming to an end. Yesterday’s mail made that all the more obvious to me, since it contained my university diploma. I have now officially graduated Friedrich-Alexander-University of Erlangen-Nuremberg with a Master’s degree in computer science. This is reason for celebration, especially since I managed to pass with distinction, but it is also an opportunity to look back. Since I will not stay at university or in Erlangen, graduation comes with a farewell.

I have enjoyed the last few years in Erlangen, especially at the System Software Group and its KESO Research Project where I wrote my Master’s thesis on “Compiler-Assisted Memory Management Using Escape Analysis in the KESO JVM”. However, in the last few months in Erlangen I’ve realized that it was time to move on and seek new challenges. And I have.

On September 1st I will take up a job as “software integrator Linux” at BMW Car IT in the city of Ulm. I’m hoping my experience with continuous integration from KESO, package management, and build systems from MacPorts may be helpful at my position. I’m really looking forward to working for BMW and moving to Ulm, and what I’ve seen so far has been fantastic! :-)

Off to pastures new!

What's New in MacPorts 2.3.0

| Comments

MacPorts 2.3.0 has been released. But what’s new for users, and why should they use the new features?

This release contains a lot of changes under the hood that users probably won’t notice. For example, MacPorts no longer uses the system-provided version of Tcl, but ships its own copy. That might seem like a step backward at the first glance, but simplifies compatibility with older systems such as Tiger or Leopard (hello, PPC users), allows us to clean up some of the cruft in the codebase and fix some long-standing issues like signal handling in future releases.

Another change most users won’t notice is the use of HTTP pipelining (I know, I know, what took us so long?), which should be beneficial especially when downloading a lot of binary packages from our mirrors. Also related to downloads, but very much noticeable are the new progress bars. Previously download progress information was only available when run in verbose mode, but 2.3.0 comes with a nice progress indicator for downloads taking longer than a few seconds. You’ll also see the same progress bar in rev-upgrade, which previously indicated its progress using a simple percentage number.

One of the changes I’ve been waiting for (and working on) is “trace mode”. Trace mode is a poor man’s sandbox initially developed for the darwinbuild project at Apple. It is based on library preloading, a technique known from Linux systems using the environment variable LD_PRELOAD. That makes it inherently insecure, but since security (i.e. protection against malicious attackers) has never been a goal for this sandbox, that’s not critical. Trace mode adjusts the environment of a build in MacPorts by hiding all files that shouldn’t be there in a vanilla installation of OS X and files in the MacPorts prefix that aren’t installed by a dependency of the current port. Trace mode is a great tool for both port authors and users: Missing dependencies are easily identified and files in /usr/local can no longer interfere with a MacPorts build with trace mode enabled. This last point is especially important since lots of third party installers and other package managers (looking at you, homebrew) install files in /usr/local. The next time a port fails to build for you, clean and re-try with port -t instead.

Other minor, but helpful new features include a check for the presence of the Xcode Command Line Tools and Xcode license agreement acceptance and a helpful new overview for the select feature at port select --summary.

Constant Resyncs With Windows 7 Software RAID

| Comments

Despite my switch to a MacBook Pro almost six years ago I still have a Windows box I occasionally use, mostly when I forget to bring my MacBook’s PSU (which happens surprisingly often, despite having two of them for exactly that reason).

Since disk space on rotational drives is cheap these days I switched to a RAID 1 configuration when I last upgraded the hardware in said computer. I went with two Western Digital WD20EARX 2 TB drives and first tried my mainboard’s fake RAID (AMD RAIDXpert on an AMD SB710 chipset). Long story short, I was unsatisfied with the performance and afraid of data loss in case my mainboard dies and I couldn’t get one with the same chipset.

I’ve seen software RAID on Linux and it was working much better for me than my mainboard’s attempt at it, so I figured I’d try the software RAID implemented in Windows 7 (Professional, Enterprise and Ultimate only). Simple to setup and with acceptable sync speeds I thought I had found what I was looking for – but then it seemed every time I rebooted, the disk array would be inconsistent and resync from scratch. Needless to say, performance was plummeting. The machine would hang for seconds waiting for I/O and every reboot would make it all start over. Even worse, the resync couldn’t be aborted and the disk array couldn’t be disbanded either (who at Microsoft thought that was a good idea?).

Turns out the culprit was a known one. KB 2913050 says “Mirrored RAID volumes report Resynching status after you restart Windows 7 […]”, and it would happen after each hotfix package installation, so for infrequently used computers basically every (re-)boot. I especially liked the “resolution”:

Microsoft is aware of this issue and intends to address it in a future release of Windows.

Translation:

It’s broken, but we’re not going to bother fixing it in Windows 7. Give us some money, if you want RAID support.

Or you could set the following magic registry keys, but we’re not going to tell you what they do, how they affect the Volume Shadow Copy Service or why we’re not setting them with a hotfix for everybody.

Please, somebody remind me why I thought Microsoft had a good reputation for their support of business-grade software…

Autoconf: AC_CONFIG_SUBDIRS With Custom Flags for Subprojects

| Comments

Starting with version 2.3.0, MacPorts will use its own copy of Tcl rather than relying on the Tcl shipped by Apple with OS X. Since MacPorts still works on versions of OS X down to Tiger that only have Tcl 8.4, all features introduced in Tcl 8.5 have been off limits and workarounds had to be used. That turned out to be unsatisfactory, especially avoiding {*} argument expansion (with the ugly workaround of using eval).

The idea of bundling a private copy of Tcl first came up in July 2013 on the macports-dev mailing list, originally in the context of the Apple distribution of Tcl changing in OS X Mavericks in a way that would no longer allow MacPorts to build from source if the optional Command Line Tools package wasn’t installed.

The Problem

MacPorts uses GNU autoconf in its build system. GNU autoconf supports bundling dependencies in subdirectories using the AC_CONFIG_SUBDIRS macro – but it wasn’t sufficient for two reasons:

  • The Tcl configure script creates a file that is needed by MacPorts' configure to find the correct Tcl interpreter and build setup. AC_CONFIG_SUBDIRS will delay configuring subpackages to the very end, but we needed it to be done earlier.
  • AC_CONFIG_SUBDIRS will always pass the same arguments given to the main configure script, which includes the prefix setting. That would install our local copy of Tcl to a location that’s being used by the MacPorts tcl port (which is version 8.6 and at the moment incompatible with some of the MacPorts code).

There have been a few attempts at solving similar problems, one of which is a patch against autoconf and has been sent to the autoconf mailinglist in April 2011, but was apparently never applied. I didn’t want to require a patched autoconf to generate the MacPorts configure script either, so applying the patch was not an option.

The Solution

An unsolved problem in a technology I haven’t used a lot yet? That sounded like a great opportunity to learn something new – and so I wrote the missing macro, mostly by reading and copying the source of AC_CONFIG_SUBDIRS and adjusting it where needed. I added extracting from a tarball so I didn’t have to commit the extracted Tcl sources. MP_CONFIG_TARBALL (source) takes the path to a tarball, the directory that’s created by extracting the tarball that contains the configure script and a list of configure parameters to pass to the subproject. Each given configure parameter overrides those given on the main project’s command line and preserves those that have not been overwritten like AC_CONFIG_SUBDIRS would.

Downloading Google Web Fonts for Local Hosting

| Comments

I am trying to make this blog completely tracker-free. To achieve that, I need to avoid fetching resources from any third-party servers, such as Google Web Fonts. There are some articles discussing how to download the different formats required for cross-browser compatibility, but manually downloading a series of files using different browsers isn’t what I had in mind – I’m a computer science student after all. So I toolchained it. Meet the Google Web Font download script (requires Bash 4.x), available from its new home on GitHub.

The script provides a command line interface to specify the fonts to be downloaded as given in the ?family= parameter of the URL generated by Google’s Web Font service. It uses curl to emulate different user agents (since the CSS file returned by Google’s setup depends on it), downloads the font files into the current directory and generates a CSS file that provides the font definition using Fontspring’s Bulletproof @Font-face Syntax. The generated file is a drop-in replacement for the file previously fetched from Google’s servers. Since it includes all font files and doesn’t depend on the user agent it might not be as efficient as the original, though (I have not actually verified this). If efficiency is key for you, you may want to limit the number of formats you provide, because WOFF support is pretty widespread in modern browsers.

Don’t forget to configure the MIME types for the font files on your web server.

Changelog

  • Version 1.2, 2015-06-13
  • Version 1.1.1, 2015-04-09
    • Switch user agent to IE 8 for WOFF to fix problems
  • Version 1.1, 2014-06-21
    • Remove colons and spaces from file names for Windows compatibility, patch from campino2k
    • Add check for Bash version, 4.x is required
    • Correctly handle fonts without a local PostScript name
    • Change format('ttf') to format('truetype') in CSS output
    • Add license header and comments
    • Added sed extended regex flag detection
  • Version 1.0, 2014-03-19

Shiny New Blog

| Comments

This blog has been overhauled completely. My old website had not been updated in three years and was beginning to fall apart in some places. Note that I have not completely moved all content from the old site, which made this move easier and quicker.

First things first: If you were subscribed to the old blog’s RSS feed, please check and update, if necessary, your feed URL from http://www.neverpanic.de/blog/atom/ or http://www.neverpanic.de/blog/rss2/ to https://neverpanic.de/atom.xml. The old URLs now are permanent redirects to the correct new URL, so if you see this post in your feed reader you’re probably fine and don’t have to touch anything.

This blog was previously using the free 1.9.x version of ExpressionEngine from EllisLabs called ExpressionEngine Core. The software had been giving me update notices for quite a while, but when I wanted to update it to the latest version I noticed the free version had been pulled from the downloads page. It’s back in version 2.x now, but I’m not going to make the same mistake of relying on closed source software again. I switched to Octopress, a static site generator based on Jekyll. I don’t particularly like software written in Ruby because most Ruby projects aren’t even trying to get into the standard Linux/BSD/OS X package managers, and this one is no exception, but it works reasonably well for me. I didn’t follow the installation instructions where it says to use rbenv or RVM but just installed ruby 1.9 and bundler from MacPorts and ran

Install Octopress dependencies using bundler
1
bundle install --binstubs --path=vendor/bundle

That puts all the dependencies self-contained into vendor/bundle (no, I don’t want to install gems as root into system locations or install yet another package manager) and creates a series of wrapper scripts for binaries in ./bin. Now I just have to remember to use bin/rake $command instead of rake $command to work with Octopress.

I chose to selectively pick the posts I moved from the previous blog software and completely removed the separate “About me” and “Portfolio” sections. I felt most of the stuff I removed didn’t represent very well what I’m currently doing, since I’m not really into websites anymore. I made sure all old URLs either redirect to the new versions or return a proper HTTP 410 Gone page.

Chaos Communication Camp Badges

| Comments

I’m currently attending Chaos Communication Camp 2011 in Finowfurt near Berlin.

Apart from the cool location within an used-to-be military airport open-air aviation museum, the coolest thing so far definitely is the electronic badge handed out at the entrance: The r0ket, an ARMv7 micro controller with back-lit LED-display, several LEDs (of which one is also used as a light sensor to determine whether it’s night and the badge display needs back light), a multi-way jog, a rechargeable battery and some extension-connectors.

The badge also features mesh-networking capabilities, although the firmware for that does not seem very stable or reliably usable at the moment. I will definitely be keeping this tool toy – there’s no better name-tag than this, I guess.

A rocket-shaped PCB with a display

GSoC Welcome Package

| Comments

I have been accepted into Google’s Summer of Code program this year to work for the MacPorts project. In a nutshell, I will be writing an equivalent to Gentoo’s revdep-rebuild – if you’re interested in the details, check out my proposal.

At the beginning of GSoC, every student receives a welcome package from Google, with a few goodies and a prepaid Visa card through which they will be paid. I have received this package yesterday and want to share my unboxing experience ;)