Hiram Software Blog


Amazon Prime Photos: Impressions after 300 GB

On Tuesday, November 4th, 2014, Amazon announced that Prime members now are eligible to upload an "unlimited" number of photos to the company's Cloud Drive product.

Prime Photos Promo Banner

If you have Amazon Prime or a Kindle, you already are a Cloud Drive customer, but it's unlikely you used the free 5 GB previously offered. Tuesday's new feature got a lot of buzz because it may have "changed the game" for photos.

As a Prime member and avid Kindle reader, I was intrigued. It's free, so why not kick the tires? We at Hiram Software have dabbled in photo projects, including implementations of pHash and similar in Scala for a project related to organizing photos in a CRM product. That project led us to conclude managing photos is harder than it looks, even for simple cases like displaying and generating thumbnails. We're not experts, but we also aren't novices coming in with a blank slate. We really care about this space, and my colleague wrote a piece asking the market to deliver a better product.

We have not, as of yesterday, found a well-priced and decent backup product that "just worked" for priceless and sentimental files like photos and videos.

The Amazon offering is the first one we've found satisfactory for photos.

Details that matter to us

My key questions for the Amazon product:

  1. Can I upload RAW files as well as JPGs?
  2. Does Cloud Drive preserve the binary files?
  3. How many photos can I get in? Is it a reliable backup option?
  4. Would I be able keep my phone and my wife's phone backed-up on the same account?
  5. Does it play well with my existing backup system?

I'm not interested in using Amazon as my only backup option, but if it has zero marginal cost for a service I'm already paying for, what is the downside to adding one more storage location to my existing backup system?

Day 1 experience using Prime Photos

Even though I am an avid Kindle reader and Prime customer, I've never used Cloud Drive.

A bit about me: I have a modest MacBook Air 13" and an iPhone 4s. My desktop is a modest iMac. I don't have a Windows computer nearby, and all my servers are Ubuntu 14 LTS.

So, Cloud Drive doesn't currently have a Mac app... the only option is to upload files using the browser. I selected a top-level folder for all pictures from a recent trip and let to go. This folder included JPG, NEF, and MOV files.

Upload step 1

The browser-based webapp processes each file serially, and it preserves folder structure. Some files failed to upload, and the webapp reported these failures.

Upload step 2

Knowing the filename is a great start, but unfortunately it alone is a bit too inconvenient since the underlying files were buried within subdirectories. It would have been nice to know more of the path information so I could retry just the files that failed.

Nonetheless, I re-ran the upload process a second time to retry those failed files. Prime Photos asked how to handle conflicts, and I chose "Skip."

Upload step 3

Presumably Prime Photos doesn't allow partial uploads, so this is a "good enough" option from the browser.

On whole, this was a good experience. Once uploaded, I saw all the photos organized by date in the photos part of the app, and I also saw the files organized in the original folders. Best of both worlds?

Can I upload RAW files as well as JPGs?

Short answer: Yes, the NEF files were counted as part of the free Prime Photos benefit.

RAW files were accepted and counted as "free" photos with unlimited storage.

Here is my storage details after uploading the first event.

Storage details after test upload

I went back and checked, and there were 15 MOV files. There were 1500 NEF files and 2500 JPGs. Amazon was storing my NEF files for free!

The documentation seemed to suggest otherwise, so I have three theories:

  1. The documentation is wrong
  2. Amazon's system miscounted
  3. Amazon incorrectly included NEF files as free photos.

Amazon may have earned a reputation for tough oncall rotations, but the company has a reputation of employing solid engineers. I reject the possibility they overlooked NEF files and #3. If it ends up being this case, I would recommend avoiding this product at all costs. RAW files and photography are just too obvious today.

I compared the stats Amazon reports with ls, and the numbers match. I reject #2.

I suspect the documentation is out of date. How many times have you launched a product before updating all the docs?

Does Cloud Drive preserve the binary files?

Short answer: Yes, Amazon did not change the JPG or NEF files uploaded.

I posted this test Tuesday on Hacker News. In short, the filenames, file sizes, and hash values stayed the same between the uploaded and downloaded files. I could detect no instance of binary manipulation.

Output from shasum from files I uploaded:

bead69d062c53436f19b6577ec7b524b9aa54445  DSC_0010.JPG
83ade975de96b81d272642740e06480fbc1484e1  DSC_0010.NEF

Output form shasum from files I downloaded:

bead69d062c53436f19b6577ec7b524b9aa54445  DSC_0010.JPG
83ade975de96b81d272642740e06480fbc1484e1  DSC_0010.NEF

Why is this a big deal? In short, other than S3 and Dropbox, most other products seem to change the original file. Facebook does, flickr does (Edit: flickr changes the filename but preserves the binary data. Thanks dmd), iCloud does, iPhoto and Aperture both do. Everyone thinks they can "improve" the JPG files. It somestimes is to save space, sometimes to add in more metadata, sometimes to protect privacy. When broadcasting a file I like that service strips metadata. When archiving, I don't. Prime Photos doesn't.

So given Prime Photos has zero marginal cost and Dropbox has a $120 marginal cost, Prime Photos is a much better deal for me.

How many photos can I get in? Is it a reliable backup option?

Short Answer: As many as I could throw at it with effort. I have no data on reliability, but some parts of uploading reduced confidence.

I uploaded a core group of the most cherished photos starting last night at 9:00 pm local. By 9:00 am local I had uploaded about 350 GB of photos. You'll notice mixed in there were various MOV and AVI files that were stored alongside what otherwise would be called "photos."

Storage details after nightly upload

300 GB isn't the world's largest collection, but it is big enough to cause headaches. Moreover, I could view the photos on my Kindle Fire, iPad, and now my sister on the East coast could see them all in one place. Previously she would only get highlights during each family gathering. Other solutions I've tried always seemed to require a lot of time to load. Also, my older family members really only understand how to view photos on tablets today... it's easier for them to swipe than click.

To get to this number, though, you should know I had 5 tabs open and uploading simultaneously. I also have a gigabit connection, and I had to write a greasemonkey script to reload a 6th tab so the session cookie stayed fresh. I noticed that the AJAX calls used to upload data wouldn't keep the session open and after about 30 minutes the upload process would "hang."

That hanging combined with uncertainty about "partial uploads" left me a little concerned if everything had been uploaded. Let's call it 95% confident everything was backed up. Room for improvement, sure, but definitely not abysmal. Otherwise, it was a slick experience, and I am optimistic when a Mac app appears this kind of silliness will go away.

The upload speeds felt about 50% slower than uploading to S3, so that may be because of heavy demand or because of additional processing. It was fast, but by default not terribly realistic to get your entire collection in. Because of the babysitting, it's not yet an option I can recommend to friends who have other priorities in life than babysitting uploads.

The perfect solution would let them "set it and forget it." Prime Photos has the opportunity to get to that state, but it is not yet there.

Would I be able keep my phone and my wife's phone backed-up on the same account?

Short Answer: Yes.

I installed the app on each of our phones, and now we are both backing up to the same account.

It's perfect. It just uploads the files. Other than some weird filenames, the checksums match between what is uploaded and what I downloaded from the Image Capture (at least for spot-checking 3 files).

The most important thing to me is now my wife's phone is backed up somewhere other than iCloud. While I am diligent about backing up my files, she is usually skeptical when I put my hands on her phone. Plus, we now can see each other's photos, which for her is a feature...

Wife: I always ask you to send me the photos, and now I don't have to? Sold!

Does it play well with my existing backup system?

Short Answer: Uploading, mostly yes. Downloading, not yet.

This option makes Prime Photos a great place to store a copy of photos, but it is not yet ready for me to be the master copy of my photos. There is no reliable way to "synchronize" the data on Cloud Drive with different hard drives that I will disconnect and put in safe deposit boxes.

One day it may happen that sync works on a Mac, but today it's not yet ready.

On the flip side, it's not a bad option to rely on multiple tabs and skipping files for syncing. It's not the best, but I'm willing to put the effort in to save $120 with DropBox, for example.

What happens if I run out of space?

Short Answer: You can keep uploading photos, but it keeps nagging you about needing to buy more storage.

Even though during my test I intended to upload directories with photos, invariably some movie files were stored alongside photos. These AVI and MOV files were not "free" and they led to this dialog and "skipped files."

Storage details after nightly upload

This may be the one unfortunate part of the whole Prime Photos experience: I now don't know if all the photos were uploaded. The error shows 3,000 files skipped but only complained about a handful of movies. Without investigating further, I have no way to know if it skipped these files because I previously uploaded them (possible) or if they were movies and it was unable to upload them due to running out of space.

I then went back and uploaded more photos after filling my account, and I can confirm that Cloud Drive let me upload more JPG and NEF files even once it was "full" ‐ I suspect the 3,000 skipped files were because I had previously uploaded them, but once the Mac app comes out I'll have more confidence.

Impressions

If you have Prime, this is a great option to store digital photos.

It is a great place for one of your offsite backups. I'm a big believer you can't have too many backup copies.

Some highlights:

  1. Files are unchanged in this service. This is a proper way to backup photos without fear (so far) of the service upgrading the metadata.
  2. My NEF files were accepted and included in the "free storage." Huge and pleasantly surprising.
  3. The browser can handle about 30 minutes of uploads unattended. I used multiple tabs and a script as a workaround, but the Mac app cannot come fast enough.
  4. The organization isn't great, but Cloud Drive preserved the folder structure uploaded from the hard drive.
  5. The iPhone app was well done and excellent to keep camera roll updated so I don't have to worry about paying for overpriced iCloud.
  6. My sister on the East coast for the first time could see all the photos I've collected. She's been asking to browse these photos for years.

What is the point of describing all this? I look at Prime Photos as another place to store photos, as if Amazon gave me another hard drive for free. It will not replace the system I have in place any time soon, but even so, giving me one more hard drive for free that obviates my need for sigal is still valuable.

Is it valuable enough to keep Prime? Probably not, but so far all I can see is upside for using Prime Photos. I have other copies so I wouldn't lose anything if Prime Photos goes away or I cancel Prime. That risk of shutdown is true for any Software-As-A-Service. It doesn't appear to be as good of a lock-in (for me) as it may be designed for or others fear. But that's ok.

What Prime Photos does do is increase my brand affinity for Amazon Prime. Prime really isn't about shipping anymore, it's a proper membership program that has perks at one of the largest online marketplaces. And I'm proud to say I have that membership card.

Is it better than Dropbox or do-it-yourself? Not necessarily. But the world isn't always organized into Red vs Blue camps. Sometimes the "best" solutions include pieces of Red and Blue.

Conclusion

I hope this post helps educate you on what to expect with Prime Photos.

So far no real limits on JPG or NEF files. Maybe not enough of a feature to sell Prime memberships, but since it has zero marginal cost, why not store another copy of priceless files?

If you have questions, contact the Hiram Software team at hello@hiramsoftware.com

Discuss at Hacker News