Ant For Server Configuration

I use Apache ant for a lot of things, almost none of which have anything
to do with building software. Simeon knows
much of what I do with it from the course of various discussions, and while
he was using it for something a week or two ago, he suggested that I blog
about some of my experience. Within 10 minutes. It didn't happen, of
course, but hopefully late is better than never.

ant, for those who don't know, is a Java-based build tool, somewhat of the
same vein as make. Major differences include that it's XML based, runs on
Java, and has built-in commands, rather than relying on the shell. Build tool
or not, where I use ant the most is in server configuration tasks. I'm going
to consider BIND config files for my example, as it's relatively straightfoward.
Apache config is my other major use; it makes it very easy to have a single
config template managed with version control, but still be able to build
actual configuration files for multiple servers in the cluster, all of which
are not exactly equal. But that's not what I'm talking about here.

DNS is pretty simple to deal with, but there is a LOT of repetition,
especially across a large number of domains that are all basically aliases
for a single application (which is what I've got). So rather than maintain
a couple hundred nearly identical zone files, I use ant to do all the dirty
work for me. Before I delve into the guts, here's a typical zone file:

$TTL 1h ; default TTL
@ IN SOA ns1.piersystem.com. root.piersystem.com. (
2005090901
3h
15m
30d
1h
)

; NS records
@ IN NS ns1.piersystem.com.
IN NS ns2.piersystem.com.

; Address records
@ IN A 216.57.200.38
www IN A 216.57.200.38

; other stuff
@ IN TXT "v=spf1 a mx ptr ip4:216.57.200.32/27 ip4:66.235.70.224/27 ~all"

I have about 90 copies of that zone, another 40 or so that are very close to
copies, and finally perhaps 10 that are pretty different. This is where ant
really shines, because it lets me templatize and parameterize the zone files
so that the entire set can be created from a very small amount of data.

How this works is via ant's wonderful filtering and property expansion
capabilities. Basically, when you copy a file from one place to another with
ant, you can also define filters to be performed as part of the copy. One of
those filters does property expansion, where properties are things like
${myPropName}, and defined in external properties files. Details to
come later. So those 90 cloned zone templates just contain a single line:

${basic.zone.pier}

That expands via this definition:

basic.zone.pier=${ttl} \n\
${soa} \n\
\n\
; NS records \n\
${ns} \n\
\n\
; Address records \n\
@ IN A ${ip.pier} \n\
www IN A ${ip.pier} \n\
\n\
; other stuff \n\
${spf1} \n\

As you can see, that definition includes even more properties, which continue
to expand until you arrive at the zone file I showed above. So all the data for
every single zone file is enclosed in two properties files (or for structure,
and another for IP addresses). But that's just the 90 or so clones.

The next batch of about 40 almost-clones are all additive changes. Most
require defining a subdomain or three, or some records for external infrastructure
that we don't manage. So those zone templates look like this (this for the
uscgstormwatch.com zone):

${basic.zone.pier}

dennis IN CNAME www
emily IN CNAME www
katrina IN CNAME www

Not much to see there, just the same thing with a couple extra records
defined afterwards. Now the last 10 or so totally custom zones. Here's an
example (the audiencecentral.com
template):

${ttl}
${soa}

${ns}

intranet IN NS ns1.piersystem.com.
IN NS ns2.piersystem.com.

; Address records
@ IN A ${ip.audiencecentral}
www IN A ${ip.audiencecentral}
testdrive IN A ${ip.audiencecentral}
shrike IN A ${ip.shrike}

; PIER sites
news IN A ${ip.pier}
sales IN A ${ip.pier}

; Other stuff
office IN A ${ip.office}

${mx.plands}

; SPF record
${spf1}

If you look back at the expansion of the basic.zone.pier, you'll
see a lot of similarities. Almost all of the pieces are reused, and there are a
few new pieces mixed in as wel. There are also some new IP addresses.

That's enough examples, lets get to the meat and potatoes of this whole thing,
the ant build file: build.xml. Here's the guts of it:

<target name="generate" depends="getSerial">
<property file="ip.properties" />
<property file="common.properties" />

<copy todir="${build.dir}" overwrite="true">
<fileset dir="${src.dir}">
<include name="**/*.tmpl" />
</fileset>
<mapper type="glob" from="*.tmpl" to="*.dns" />
<filterchain>
<expandproperties/>
</filterchain>
</copy>

<copy todir="${dest.dir}" overwrite="true">
<fileset dir="${static.dir}">
<include name="*" />
</fileset>
</copy>

<move todir="${dest.dir}" overwrite="true">
<fileset dir="${build.dir}">
<include name="**/*.dns" />
</fileset>
<mapper type="flatten" />
</move>
</target>

This defines a target (ant's name for a piece of work) named "generate", and
that it depends on the target "getSerial". GetSerial, as you might imagine,
creates the serial number for the zone files and stores it in a property so that
it can be injected as part of the ${soa} expansion. Anyone who's interested in
how that works, let me know; I'm going to skip it here because it's complex,
nasty, and doesn't really lend anything to this post.

First thing the target does is include a couple property files (which I've
mentioned before), that contain all the expansions, including the
basic.zone.pier one. Next it does a couple copy operations and
then finishes up with a move operation.

The first copy tag copies "something"
to my build dir (specified by the ${build.dir} property, which happens to point
at ./build. The fileset tag it contains specifies what that
something is: all files in the ${src.dir} directory (including subdirectories)
that end with .tmpl. Next, the mapper tag converts all file extensions from
.tmpl to .dns as part of the copy (since that's my extension of choice
for zone files). Finally, the innocent looking filterchain and expandproperties
tags, do all the magic of expanding all those properties in the files that are
copied.

The second copy is much simpler, doing nothing more than a vanilla copy
of the files in the ${static.dir} to ${dest.dir} (which points to
/var/named). Note that this is different than where the first
copy went for reasons we shall see in a moment.

The last piece of magic happens in the move tag. It moves the newly created
.dns files from the build directory into the destination directory where the
static files just went, and it applies another mapper that flattens the directory
structure. ant only allows a single mapper per copy/move operation,
which is why I copy to a temp location, and then move to the real place. As you
can probably guess, doing a flatten allows me to keep all my zone templates
organized into a neat hierarchy for easy management, but not have to deal with
the pathing issues when it comes time to actually give BIND the zone files.