1. launch a virgin EC2 instance (I used the console and based it on ami-7f418316).
2. create a data volume and attach it to the instance.
3. allocate an Elastic IP and associate it with the instance.
4. set up an A record for the Elastic IP.
5. build a setup script which will configure the instance as needed.  I feel it's important to use a script for this so that if your instance dies for some reason you can create a new one without too much fuss.  It's not strictly necessary, but part of the cloud mantra is "don't repair, replace" because new resources are so inexpensive.  Don't forget to store it on your volume, not the root drive or an ephemeral store.  Here's one useful snippet for modifying /etc/sudoers that took me a little digging to figure out:

   bash -c "chmod 660 /etc/sudoers;sed -i -e 's/^\# \(%wheel.*NOPASSWD.*\)/\1/' /etc/sudoers;chmod 440 /etc/sudoers"

6. rsync all the various data files from the current server to the new one (everything goes on the volume; symlink – via your setup script – where necessary).  Again, use a script.
7. once you're happy that your scripts work, kill your instance,
8. launch a new virgin EC2 instance,
9. attach your data volume,
10. associate your Elastic IP,
11. run your setup script,
12. if anything didn't turn out the way you wanted, fix it, and go back to step 8.
13. shut down all the state-mutating daemons on the old box.
14. shut down all the daemons on the new instance.
15. set up a downtime message in Apache on the old box.  I used these directives:

    RewriteEngine  On
    RewriteRule    ^/.+/.*    /index.html    [R]
    DocumentRoot   /var/www/downtime

16. run the rsync script.
17. turn on all the daemons on your new instance.
18. add /etc/hosts records to the old box and update DNS with the Elastic IP.
19. change Apache on the old box to proxy to the new instance (so people will get the new site without having to wait for DNS to flush).

    ProxyPreserveHost   On
    ProxyPass           /   http://www.barneyb.com/
    ProxyPassReverse    /   http://www.barneyb.com/

    These directives are why you need the rules in /etc/hosts, otherwise you'll be in an endless proxy loop.  You'll need to tweak them slightly for your SSL vhost.  The ProxyPreserveHost directive is important so that the new instance still gets the original Host header, allowing it to serve from the proper virtual host.  This lets you proxy all your traffic with a single directive and still have it split by host on the new box.

The net result was a nearly painless transition.  There was a bit of downtime during the rsync copy (I had to sync about 4GB of data), but only a few minutes.  Once the new box was populated and ready to go, the proxy rules allowed everyone to keep using the sites, even before DNS was fully propagated.  Now, a few hours later, the only traffic still going to my old box is from Baiduspider/2.0; +http://www.baidu.com/search/spider.html, whatever that is.  Hopefully it'll update it's DNS cache like a well-behaved spider should, but not according to my TTLs.  Hmph.

Steps 1-12 (the setup) took me about 4 hours to do for my box.  Just for reference, I host a couple Magnolia-backed sites, about 10 WordPress sites (including this one), a WordPressMU site, and a whole pile of CFML apps (all running within a single Railo).  I also host MySQL on the same box which everything uses for storage.  Steps 13-19 took about an hour, most of that being waiting for the rsync and then running through all the DNS changes (about 20 domains with between 1 and 10 records each).

And now I have extra RAM.  Which is a good thing.  I'm sure a few little bits and pieces will turn up broken over the next few days, but I'm quite happy with both the process and the result.

UPDATE: AWS contacted me and apparently the problem was documentation, not a regression.

<cfparam name="url.path" default="" />

<cfset s3 = createObject("component", "amazons3").init(
  "YOUR_AWS_KEY",
  "YOUR_AWS_SECRET"
) />

<cfoutput>
<cfset bp = "" />
<h1>
<a href="?path=#bp#">ROOT</a>
<cfloop list="#url.path#" index="segment" delimiters="/">
  <cfset bp = listAppend(bp, segment, "/") />
  / <a href="?path=#bp#">#segment#</a>
</cfloop>
</h1>

<cfif url.path EQ "">
  <cfset b = s3.listBuckets() />
  <ul>
    <cfloop query="b">
      <li><a href="?path=/#name#">#name#/</a> #dateLastModified#</li>
    </cfloop>
  </ul>
<cfelse>
  <cfset q = s3.listObjects(listFirst(url.path, '/'), listRest(url.path, '/')) />
  <ul>
    <li><a href="?path=#reverse(listRest(reverse(url.path), '/'))#">..</a></li>
    <cfloop query="q">
      <li>
      <cfif type EQ "dir">
        <a href="?path=#listAppend(directory, name, '/')#">#name#/</a>
      <cfelse>
        <a href="#s3.s3Url(bucket, objectKey)#">#name#</a>
      </cfif>
      </li>
    </cfloop>
  </ul>
</cfif>
</cfoutput>

The default mode of operation assumes a delimiter of '/' (just like a filesystem).  If you want to do non-delimited operations (like generic prefix matching), you'll want to supply an empty delimiter, or you'll get weird results.  For example:

<cfset k_objects = s3.listObjects('my-bucket', 'k', '') />

If you omit the third parameter, the default 'k' will be used, and you'll get back objects within the 'k' psuedo-directory, rather than objects that begin with a 'k'.  This is the reverse of the default position of the raw S3 API, which assumes you want simple prefixing and makes you explicitly add the delimiter if you want psuedo-directory contents.

This dichotomy can also lead to weird results in the resulting recordset.  Every recordset comes with both 'bucket' and 'objectKey' columns that match the raw S3 nomenclature and 'directory' and 'name' columns that match the filesystem "view" of S3.  If you're doing raw prefixes you'll want to use bucket/objectKey (as the directory/name semantic doesn't work with prefixes).  If you're doing filesystem type stuff you'll probably want directory/name (though bucket/objectKey will still be correct).

<cfset cloudfront = createObject("component", "amazoncloudfront").init(keyPairId, privateKeyFile) />
<cfset signedUrl = cloudfront.signUrlWithTimeout(resourceUrl, 600) />

This will generate a signed URL for the CloudFront resourceUrl (direct domain or CNAMEd) which expires in 600 seconds (10 minutes).  Very much like the S3 CFC.  Here we're dealing with resourceURLs directly (which correspond to a bucket and object key) rather than separate buckets and object keys.  CloudFront doesn't distinguish between the two parts, so the CFC doesn't either.

The biggest gotcha, however, is with the signing mechanism.  S3 uses a simple pre-shared key, but CloudFront uses an RSA private key which is significantly more complicated to deal with.  Unfortunately, Amazon provides it's keys in PEM format, but core Java can only read DER format, so you must either convert your private key to DER format with OpenSSL, or use a third party library.  Fortunately, both are pretty simple.

Here's the command to convert your key with OpenSSL:

openssl pkcs8 -topk8 -in pk-KEYPAIRID.pem -nocrypt -outform DER -out pk-KEYPAIRID.der

Alternatively, you can use the not-yet-commons-ssl package, which provides support for reading keys in PEM format (among a pile of other things).  If you can add the JAR to your classpath, this is definitely a superior solution to manual conversion, since you can transparently use pretty much any RSA private key.  And there's nothing to enable in the CFC; if not-yet-commons-ssl is available on the classpath, it'll automatically use it for reading in the private key.  As an aside, the name 'not-yet-commons-ssl' reflects the fact that the package has applied fro Apache Commons incubation, but it hasn't been accepted yet.  The code is orgniazed in the 'org.apache.commons.ssl' package assuming it's acceptance, but it is still unofficial.

As always, updates and such are available on the project page.

In particular, the CFC assumes that it is the only interface to the S3-stored assets that it is used to interface with. If you use any other mechanism to manipulate those assets (including multiple CF applications), you'll run into issues. The cache itself is the canonical source for cache state, so emptying the cache folder will always revert the CFC back to S3's state if the cache is out of sync.

If you cluster multiple CF instance together, you can still use the local cache, but you must use a single cache for all CF instances. I.e. the cache must reside on a disk shared by all instances, rather than each instance having it's own separate cache. This reduces the performance benefit slightly (since you must use a non-local disk), but it will still be faster than S3.

The CFC exposes a deleteCacheFor() method that accepts a bucket and objectKey pair that can be used for managing the cache outside of actual S3 operations. If you have multiple CF instances that cannot share a single local cache, or for which the network overhead for a shared cache is still undesirable, you can use this method to synchronize the instances' caches via JMS or something. Obviously that's far outside the scope of the CFC itself, but the hook is there to support it. Note that you must delete cache when overwriting an asset on S3, as the local cache will not pick up the change in S3, it will continue to return the old version if it's not cleared.

Building on the 's3Url' UDF that I published last week, I whipped up a little CFC to manage file storage on S3 with a very simple API. It has s3Url, putFileOnS3, getFileFromS3, s3FileExists, and deleteS3File methods, which all do about what you'd expect. You can grab the code here: amazons3.cfc.txt (make sure you remove the ".txt" extension) or visit the project page. It uses the simple HTTP-based interface, so after the authentication is handled, it's all very simple and fast. I haven't looked at the SOAP interface – why bother complicating a simple task?

With that CFC (and an application-specific wrapper to take care of some path-related transforms), porting the whole app took about two hours. I also realized after I was mostly done that the CF image tools accept URLs as well as files, so I switched my image reads to just use URLs instead of pulling the file local and reading it from disk.

As for moving all the actual content, S3Sync was a champ, moving about 4.5GB of data from my Cari server to S3 in a few hours, including gracefully handling a couple errors raised by S3 (which a retry – performed automatically – solved), and a stop/restart in the middle. Total cost: about 65 cents.

Next is porting the blogs, including all the Picasa-based galleries. Unfortunately, that means writing PHP, but with how easy the CF stuff was, I don't think it'll be too much effort.

• S3Sync – A simple rsync-like command line tool (called 's3sync') for syncing stuff from a computer to S3 or the reverse.  Also includes the 's3cmd' tool that roughly implements the web service API (list your buckets, put a file, etc.).  This is the cornerstone of the plan for moving all my data files from my current server and backups to S3.  Once the migration is complete, s3cmd will probably be the tool of choice for manipulating S3 programatically.  Written in Ruby, and requires 1.8.4+; my CentOS 4 box couldn't find a new enough RPM, so I had to compile from source (which was totally painless).
• S3 Firefox Organizer (S3Fox)- a client for S3 following the standard FTP client paradigms.  It has it's own proprietary definition of folders, but they're unobstrusive.  Since I'm getting stuff into S3 mostly with s3sync, I'm mostly using this for read-only oversight.
• EC2 UI – a client for managing your EC2 "stuff" from Firefox.  While not FTP-like at all, it shares a lot of the same UI as S3Fox for setting up accounts and the like.

<cffunction name="s3Url" output="false" returntype="string">
  <cfargument name="awsKey" type="string" required="true" />
  <cfargument name="awsSecret" type="string" required="true" />
  <cfargument name="bucket" type="string" required="true" />
  <cfargument name="objectKey" type="string" required="true" />
  <cfargument name="requestType" type="string" default="vhost"
    hint="Must be one of 'regular', 'ssl', 'vhost', or 'cname'.  'Vhost' and 'cname' are only valid if your bucket name conforms to the S3 virtual host conventions, and cname requires a CNAME record configured in your DNS." />
  <cfargument name="timeout" type="numeric" default="900"
    hint="The number of seconds the URL is good for.  Defaults to 900 (15 minutes)." />
  <cfscript>
    var expires = "";
    var stringToSign = "";
    var algo = "HmacSHA1";
    var signingKey = "";
    var mac = "";
    var signature = "";
    var destUrl = "";

    expires = int(getTickCount() / 1000) + timeout;
    stringToSign = "GET" & chr(10)
      & chr(10)
      & chr(10)
      & expires & chr(10)
      & "/#bucket#/#objectKey#";
    signingKey = createObject("java", "javax.crypto.spec.SecretKeySpec").init(awsSecret.getBytes(), algo);
    mac = createObject("java", "javax.crypto.Mac").getInstance(algo);
    mac.init(signingKey);
    signature = toBase64(mac.doFinal(stringToSign.getBytes()));
    if (requestType EQ "ssl" OR requestType EQ "regular") {
      destUrl = "http" & iif(requestType EQ "ssl", de("s"), de("")) & "://s3.amazonaws.com/#bucket#/#objectKey#?AWSAccessKeyId=#awsKey#&Signature=#urlEncodedFormat(signature)#&Expires=#expires#";
    } else if (requestType EQ "cname") {
      destUrl = "http://#bucket#/#objectKey#?AWSAccessKeyId=#awsKey#&Signature=#urlEncodedFormat(signature)#&Expires=#expires#";
    } else { // vhost
      destUrl = "http://#bucket#.s3.amazonaws.com/#objectKey#?AWSAccessKeyId=#awsKey#&Signature=#urlEncodedFormat(signature)#&Expires=#expires#";
    }

    return destUrl;
  </cfscript>
</cffunction>

To use it, do something like this:

s3Url(aws_key, aws_secret, "s3.barneyb.com", "test.txt", 'cname');

That will generate a request to the file "test.txt" in the "s3.barneyb.com" bucket, using a CNAME-style URL. Obviously you'll have to know my AWS key and secret for it to work, and I'm not telling, but substitute your own values. You can use regular (bucket name in the request), vhost (bucket name in an S3 subdomain), cname (a vanity CNAME pointing at S3), or ssl (regular over HTTPS) for the 5th type parameter to control the style of URL generated.

Edit: here's a link to the project page.

• Using S3 for all my backups and data storage will definitely give me some piece of mind that I've been lacking.
• The virtualized nature of the servers means doing upgrades is totally safe: launch a new copy of the box, do the upgrade, and if everything's golden, switch the IP to the new box. Cost is $0.10/hr which is close enough to zero to not matter.
• I get a processor "upgrade" from my Celeron at Cari to a similarly clocked Xeon equivalent. weight loss The latter is paravirtualized, of course, but it should still help since most of my apps are CPU-bound. I also get some more RAM, but that's less important.
• Last, but not least, Cari has had a lot of network issues in the year I've hosted there while Amazon hasn't.

First task is to move storage over to S3, and update the applications that currently access stuff off the filesystem (like autogeneration of thumbnails).