Simplicity Through Obscurity is Bad

Making things easy is hard; it can never be said enough.

However, there is a tendency by many technology developers to attempt at solving this difficult problem by hiding things. In my humble opinion, this concept is usually taken too far. This week’s example involves Apple.

Somewhere along the lines, the Notepad app on my iPhone lost a whole slew of notes I had entered over the years. I don’t know exactly when or why. There are several fundamental issues exposed by this:

1. Computers and services must treat your data as sacred—as Ford would say, Quality is Job #1. If I enter something, your system must do its very best to ensure it is never accidentally deleted. If you can’t debug the current system, than it is too complex, and the only solution is to throw out that code and start over. (This should be considered simply a variation on Alan Cooper’s theme of prototyping in Visual Basic, throwing away the code, and redoing everything in C (substitute the prototyping and production languages of your choice). It matters not that what you’re throwing away was coded with a production language and that you’ve been shipping it for years. It is defective, and any further fixes will only make it more complex. Adding more noodles to spaghetti code will not make it better. You have to take away as many noodles (carbs/bloat) and add more meatballs (protein/goodness) as possible.
2. You need to test more.
3. When all else fails, make recovering from the problem simple.

On this point, Apple must pay more attention to #1 and #3 (#2 becomes increasingly impossible the more twisted noodles you add to the dish), and #3 is what frustrated me enough to compose this post.

I realized a few weeks ago that I was missing these notes, but finally had time this weekend to go spelunking for them. Fortunately, I’ve been using Genie Timeline as my backup solution for most of this year, and have a Time Machine–like backup on my NAS at home, so I figured it would be possible. Here’s what I’ve had to do because Apple obscures the IPhone backup/restore process.

1. Since neither iTunes nor iCloud seems to provide anything more than one backup at a time, I’m lucky that my backup solution has been storing multiple versions of my iPhone backups for months. Kudos to Genie9 for making this work.
2. Since the backup is on my NAS, I have to do most of the sleuthing at home.
3. Because I configured Genie Timeline to use compression, the combination of that and the sheer number of files within one iPhone backup (> 6,300) takes several hours to restore, even over GigE. But why do I have to restore over 6K files just to find the one that contains the Notes database? Because Apple obscures the actual filenames as 40-character hashes. I put up with this once to make sure I had all the data necessary, and restored my first backup from January.
4. But because I’m not sure exactly when my iPhone deleted all those notes, I will need to restore several time points to find the most complete set of notes. Even using a binary search, this would be painfully slow.
5. The next problem was that iTunes can only do a monolithic restore, wiping out everything just for me to look at the restored notes. Bad Apple.
6. So I bought the iPhone Backup Extractor, which so far has done an amazing job of extracting data from iPhone backups. It succeeded in restoring all of the notes I had in January to individual HTML files. Awesome!
7. The developer, Reincubate Ltd., wins more kudos for having the answer to #4 on their web site: (click then scroll down to read why are the filenames garbled?).
8. With that info, and a bit of trial-and-error, I was able to determine that all I needed to restore from each massive iPhone backup were two files: Manifest.mbdb and ca3bc056d4da0bbf88b5fb3be254f3b7147e639c.
9. Update: Turns out restoring Info.plist, Manifest.plist, and Status.plist in addition to the above files makes the application work better—at least in that the actual date the backup was made is displayed, so that you know which backup you’re restoring from. Still better than restoring 6,338 files.
10. So now I can quickly restore a few backups from different dates over the past four months, and browse the resulting HTML files to discover the one that is most complete, and paste them into something a bit more reliable than the iOS Notebook app.

I can only guess that Apple obscures these filenames to reduce the number of support calls they need to take from non-programmer end-users looking for help restoring their precious content. But this obscurity is only necessary because Apple has not made the Notebook app reliable and provided an easy-to-use backup restore module in iTunes that addresses the obvious and likely desires of its customers.

The moral here: It is better to fix a problem at its foundations instead of covering it up with spaghetti and sauce, however tasty and appealing the sauce is.