S3 is Sweet (One App Down)

This weekend I ported my big filesystem-based app to S3, and it went like a dream. It's a image-management application, with all the actual images stored on disk. In addition to the standard import/edit/delete, the app provides automatic on-the-fly thumbnail generation, along with primitive editing capabilities (crop, resize, rotate, etc.). With images on local disk, that's all really easy: read them in, do whatever, write them back out. I figured using S3 would make things both more cumbersome and less performant. Both suspicions turned out to be unwarranted.

Building on the 's3Url' UDF that I published last week, I whipped up a little CFC to manage file storage on S3 with a very simple API. It has s3Url, putFileOnS3, getFileFromS3, s3FileExists, and deleteS3File methods, which all do about what you'd expect. You can grab the code here: amazons3.cfc.txt (make sure you remove the ".txt" extension) or visit the project page. It uses the simple HTTP-based interface, so after the authentication is handled, it's all very simple and fast. I haven't looked at the SOAP interface – why bother complicating a simple task?

With that CFC (and an application-specific wrapper to take care of some path-related transforms), porting the whole app took about two hours. I also realized after I was mostly done that the CF image tools accept URLs as well as files, so I switched my image reads to just use URLs instead of pulling the file local and reading it from disk.

As for moving all the actual content, S3Sync was a champ, moving about 4.5GB of data from my Cari server to S3 in a few hours, including gracefully handling a couple errors raised by S3 (which a retry – performed automatically – solved), and a stop/restart in the middle. Total cost: about 65 cents.

Next is porting the blogs, including all the Picasa-based galleries. Unfortunately, that means writing PHP, but with how easy the CF stuff was, I don't think it'll be too much effort.

5 responses to “S3 is Sweet (One App Down)”

  1. Gerald Guido

    Very nice. Thanx for sharing the cfc.

  2. Joe Danziger

    You may also find helpful the Amazon S3 Rest Wrapper available at http://amazons3.riaforge.org/

  3. Gerald Guido
  4. derek

    Hi Barney
    I have a site that allows different users to upload/download files as well as attach files to tasks(upload and download). I've got the whole database and userid thing going and will store the bucket and filename per user per task.

    The problem I have is that I do not really know how to code the upload or download that will actually invoke your methods. (New to CFC's)

    I am doing this part time and I guess I can work it out but it will take me days. I was hoping you could give me a few 1-liners to invoke your methods.

    If not, thanks for a great insight anyway.


  5. Derek

    Hi Barney

    Further to above, I need to track each upload and download and so a simple url link will not help to track unless you know of some fancy javascripting