Hiram Software Blog


Learnings from writing EXIF data

Last week Hiram Software took on a contract to add automated photo organization features to a popular CRM product. We discovered a couple details about EXIF that frustrated us, and wanted to take our emotions out on a blogpost.

Thumbnails

One of the most basic features of any photo product is to generate thumbnails. In the online world (think Instagram or Tumblr), ImageMagick is the most common way to generate thumbnails. But what if you can't run scripts or add binaries to the servers? What if, for security reasons, you must use Java only?

In that case, there are a couple options. One is to read the image into the Java Graphics library and manually apply transformations in your code. The other is to use imgscalr from The Buzz Media.

We tried imgscalar, and it worked well, but we had reason to normalize output with business logic and couldn't use imgscalr without modification. We ended up implementing image thumbnails using Java Graphics outselves.

Our test case included pictures from a Samsung Galaxy, an iPhone, a Nikon, and a Canon.

Interestingly, the iPhone thumbnail pictures appeared upside down. This was different from the Nikon and Canon images which were oriented correctly.

What surprised us is the iPhone (at least through iOS 7) writes the bitmap data to the JPEG in the same way, regardless of how the phone was oriented. The phone then writes a value to the "Orientation" field that tells how it thinks the phone was oriented. Unfortunately, the other cameras did not write to this field and instead changed how they wrote the bitmap data making a one-size-fits-all rule impractical.

Programs responsible for generating thumbnails or otherwise viewing iPhone images need to read this EXIF field in addition to the bitmap data in order for people to think the images are right. The details for how the field is written (it's bit encoding) are defined in the spec.

Adding Missing EXIF Data

Once we realized we couldn't escape EXIF fields, even just to view the images, we got tempted to add in or correct missing EXIF Data. For example, the date time field in EXIF notoriously does not include the timezone, which makes relying on EXIF near impossible.

What if we re-wrote the EXIF datetime field in UTC for all photos? Our CRM product knows when the record got added and from which timezone the salesperson lives, so the server is capable of ensuring the datetime is "correct" regardless of what the camera said.

We ran into a snag with the Nikon camera. First, you should know that the EXIF section is limited to 64 kb in JPEG files. That should be more than enough, right?

Well, the Nikon calculates a thumbnail for each photo it takes and stores the thumbnail inside the EXIF section. The Nikon camera appears to size its thumbnail to take up all the remaining space in the EXIF, thus making it near impossible to write to the EXIF section without corrupting the whole JPEG.

We ended up removing the thumbnail, but that had other impacts to how the windows and mac operating systems treated the files. They now had to recompute the thumbnail on the fly, making most operations of the photos "slower" from the user's perspective.

Making this change also made it difficult to identify dupes, requiring us to implement our own version of pHash and its ilk.

Conclusion

Photos are a rabbit hole. EXIF is a mess, at least how each vendor uses it differently.

And, unfortunately, you have to read EXIF if you want to do something as simple as show a JPEG on screen.