My Amazon Toolkit (Thus Far)

I'm early in the move to Amazon, of course, but already some specific tools are indispensable.  I'm sure the list will grow, but here's where I'm at right now:

  • S3Sync – A simple rsync-like command line tool (called 's3sync') for syncing stuff from a computer to S3 or the reverse.  Also includes the 's3cmd' tool that roughly implements the web service API (list your buckets, put a file, etc.).  This is the cornerstone of the plan for moving all my data files from my current server and backups to S3.  Once the migration is complete, s3cmd will probably be the tool of choice for manipulating S3 programatically.  Written in Ruby, and requires 1.8.4+; my CentOS 4 box couldn't find a new enough RPM, so I had to compile from source (which was totally painless).
  • S3 Firefox Organizer (S3Fox)- a client for S3 following the standard FTP client paradigms.  It has it's own proprietary definition of folders, but they're unobstrusive.  Since I'm getting stuff into S3 mostly with s3sync, I'm mostly using this for read-only oversight.
  • EC2 UI – a client for managing your EC2 "stuff" from Firefox.  While not FTP-like at all, it shares a lot of the same UI as S3Fox for setting up accounts and the like.

6 responses to “My Amazon Toolkit (Thus Far)”

  1. Mike Henke

    I found JetS3t works great. I use the synchronize command line tool between my Windows and Linux computers.

  2. Naama

    S3Sync didn't work for me well. The backup time was very long, It alway re upload the modify files all over again :(
    I move to and now using rsync to s3. Modified file are partly upload the my backup is much faster.

    Naama G

  3. Naama

    Hi barneyb,

    I don't have EC2 so it save me the trouble (and cost) to deal with EC2.

    > s3rsync is still doing a full-file load into S3

    Why should I care, as long as it fast and easy to use?

  4. Naama


    > full load into S3 means you're paying for the full bandwidth every time.

    No, I'm paying only for the bandwidth from my Server to S3rsync servers (which very efficient base on rsync) I'm not pay for the bandwidth from S3rsync servers to S3 storage.