The softaholic blog

Requirements definition is often a finite operation, but its branching depth inevitably overwhelms the human mind.

This post was initially about a bad experience with Spanair regarding missing items in my baggage.

Considering what happened a few hours ago to the passengers of flight JKK 5022, I don’t think this discussion is appropriate or respectful anymore.

Let’s face it: one person project is fine, two is ok, three or more is a mess. Say you are a project manager; you have two ways to do you job:

• You can get strongly involved in project’s everyday tasks: requirements, architecture, implementation details… you know the project and answers come easily when hard questions arise. Extra points if you code some parts of the system.
• You can use the outsider’s approach and do objective project management: take tasks, take resources, mix’em up and then monitor the whole thing day after day. Extra points if you don’t even know what the project is about or what the programmers do to get that damn thing working.

Either approach is great in my view, but your will to live can really get sucked out in either of them, too. If you get deeply involved, you get no time to do any project management: time flies when you engage into a hard technical challenge.

If you take it impersonally, make sure your life outside work is just great, otherwise it’ll take you less than four weeks to start considering suicide.

I guess it ends up like everything else in life: drawing a line at the right point.

During a speech at the Hungarian University of Economy, someone from the audience threw eggs to Mr. Ballmer, causing him to run enter duck & cover mode. The crossfire was preceded by a cryptic speech reclaiming tax money back (huh?).

My guess is economists don’t understand Ballmer’s sense of humor… wait, nor does anyone else, hmm.

Seen at Gizmodo.

Freaks gathered again in Puerto Real.

No hay que temer aquello de lo que nada se sabe.
Ni al sexo, ni al amor ni a la muerte

Lucía Etxebarria - Beatriz y los cuerpos celestes

Now that we have our images in a file format readable by any JPEG-capable software, all we have to do is find an automated (or manual, depending on the amount of pictures we are dealing with) way to convert the 2Mb chunks into more optimized files (Photoshop cut my file lengths down to 600Kb per image).

This is no big deal for anyone who understood the previous parts. Gimp, ImageMagick or Photoshop are just a few examples of applications able to read and write JPEG files in a variety of quality and compression levels. As easy as opening each image separately and saving it again with the same file format. All the unnecessary junk will be left behind and the image will look the same using much less disk space.

This is it! I hope someone found this tutorial useful. Critics and comments are welcome.

Okay, so now we have a raw copy of the memory card in a file. What to do with it? The more obvious thing would be to get a hex editor, look for JPEG headers and figure out a way to extract all the data after those to separate files.

Actually, that is what we are going to do but in a more automatic way. Say we can write code in any language powerful enough to do some basic file input/output operations efficiently, then why not writing a short program which scans our big data dump from top to bottom, and then dumps chunks of a fixed size to sequential files?

Now before you get all excited about how easy and perfect this method is, let me tell you one thing: the JPEG format is a tricky lady. Computing the size (in bytes) of a JPEG file from its header requires a deep knowledge of the structures and algorithms involved in the JPEG image compression/encoding standard. My approach is to just extract the 2 MBs of data following each occurrence of a JPEG header (I know my digital camera never produces files larger than that). Part IV will explain how to optimize those 2MB files once we have recovered them.

I could get really technical here on how my code works and all that, but I think there is no rocket science in a pattern search program like this and therefore the source code is way enough to get an idea, so here’s the source code.

Anyone can build the source using a decent C compiler like GCC or Microsoft C/C++ compiler (both are free by the way). Once you have compiled the source code, you can run it from the command line like this:

./jpegrescue memory-card-dump.bin 2097152 jpg

With this, you are telling the program to scan the file “memory-card-dump.bin” and dump the 2 first MBs after each occurrence of the JPEG header, using “.jpg” as the output file extension. And… that’s basically it! after a few seconds (or minutes, depending on the input data size), we should have a bunch of jpeg files located within the same directory we ran the application from. If your card has been used enough times, you’ll be astonished about the amount (and age) of pictures recovered.

Now let’s get our hands dirty. First step is getting a safe copy of our memory card contents into a file within our hard disk. This will allow us to play around with its contents way faster than we would by just reading the card every single time.

If you haven’t written anymore to your card since the lost files where removed, your chances for a flawless data recovery are almost of a 100%. As many files you (or your camera) have written to the card, less chances of getting back your files.

This happens because most filesystems don’t erase file data when the file is deleted. They just mark that space as “free”, so that other files can use that available space. If no files were written to your card after deleting those files, then you can be sure your data is still there, only the path to it is lost.

Back to the memory card data dump issue, we well need to open the Terminal program in order to pull some UNIX swiss-army-knife tricks. I assume your memory card is inserted in your card slot, or camera’s slot while the camera is hooked to your PC via USB. If you did this correctly (and if you didn’t, stop reading this post), you should be able to access the card’s contents using Finder without much problem. Now switch back to the Terminal and type “mount”, then you should get some stuff like this:

/dev/disk0s2 on / (local, journaled)
devfs on /dev (local)
fdesc on /dev (union)
on /.vol
/dev/disk0s3 on /Volumes/Untitled (local, read-only)
automount -nsl [185] on /Network (automounted)
automount -fstab [189] on /automount/Servers (automounted)
automount -static [189] on /automount/static (automounted)

Find out which line represents your memory card device and note down the device filename for it (the first path before the “on” word), stripping the last two characters (sX), where X is a integer number.

Finally, we’ll use UNIX command “dd” to dump all the raw binary contents of the card to a file in our HD (replace “/dev/disk2″ with the path you got in the previous step):

dd if=/dev/disk2 of=~/memory-card-dump.bin

Dumping the data will take some minutes, depending on the size of the memory card and the transfer rate of it. For instance, dumping my humble 128 memory stick took around five minutes.

After the command finishes, you will have a perfect binary copy of your memory card (containing you valued pictures somewhere), ready to be analyzed and processed as many as we need. Don’t you start loving this thing?.

Problem: a butthead like me screws up while importing digital pictures from its Sony digital camera into iPhoto, total mess, no backups, no trash-canned files, nothing… just lost data.

What to do now? a few Google searches reveal no big deal of MemoryStick data recovery tools out there. Every vendor offers its very own pay-your-ass-for-rescue deal and, what the heck, I’m not made of money. But I decide to not give up so fast and try to do the recovery myself.

Most people think that data recovery is a complex process involving rather sophisticated algorithms and intensive processing duties. I totally agree with them, but it turns out in some cases a relatively simple and expensive way to recover lost data is available. You just have to understand a few bits from here and there (filesystems, image file formats, programming, digital imaging tools…) and have some patience.

This is the first of a blog post series (around 3 of them) about how to perform DIY data recovery on any flash memory card with flawless results.