Build-Time Aggregation of JS/CSS Assets

Ben Nadel posted about compiling multiple linked files (JS/CSS) into a single file this morning, and he does it at runtime. I commented about doing it at build-time instead, and a couple people were wondering more, so here's a brief explaination.

The first part is a properties file (which can be read by both Ant and CF (or whatever)). Here's an example (named agg.js.properties):

# the type of file being aggregated (used to do minification)
type         = js
# the URL path the files are relative to.
urlBasePath  = /marketing/js/
# the list of filenames to aggregate.  The first line (with the equals
# sign) should be a filename and a slash, all other lines should be a
# comma, a filename, and a slash  Indentation is irrelevant.
filenames    = date.js\
  ,jquery-latest.js\
  ,ui.datepicker.js\
  ,ui.mouse.js\
  ,ui.slider.js\
  ,ui.draggable.js\
  ,jquery.dimensions.js\
  ,jquery.easing.1.2.js\
  ,jquery-easing-compatibility.1.2.js\
  ,coda-slider.1.1.1.js\
  ,jquery.tooltip.min.js\
  ,jScrollPane.min.js\
  ,jquery.metadata.js\
  ,prototype.classes.js\
  ,reporting.js\
  ,jquery.ajaxQueue-min.js\
  ,script.js

This sets up the everything needed for the aggregation. Within our project, we have this file as a peer of the property file (named agg.js.cfm):

<cfscript>
filename = replace(getCurrentTemplatePath(), ".cfm", ".properties");
fis = createObject("java", "java.io.FileInputStream").init(filename);
bis = createObject("java", "java.io.BufferedInputStream").init(fis);
props = createObject("java", "java.util.Properties").init();
props.load(bis);
urlBasePath = props.getProperty("urlBasePath");
type = props.getProperty("type");
filenames = listToArray(props.getProperty('filenames'));
for (i = 1; i LTE arrayLen(filenames); i = i + 1) {
	if (type EQ "css") {
		writeOutput('<link rel="stylesheet" href="#urlBasePath##filenames[i]#" type="text/css" />');
	} else { // js
		writeOutput('<script src="#urlBasePath##filenames[i]#" type="text/javascript"></script>');
	}
	writeOutput(chr(10));
}
</cfscript>

It reads the properties file, and writes out either LINK or SCRIPT tags as appropriate to the individual assets. This facilitates easy debugging in development, because nothing is modified from it's source. The file is included into the HEAD of our layout templates to get everything in page.

The real magic happens with Ant, which we use for our deployments. Within the build file, we have a call to the aggregateAssets target for each properties file:

<antcall target="aggregateAssets">
  <param name="propfile" value="${output}/wwwroot/marketing/templates/agg.js.properties" />
  <param name="rootdir" value="${output}/wwwroot/marketing/js" />
</antcall>

The params specify the properties file and the root directory. Note that the rootdir param corresponds with the urlBasePath in the properties file. The target itself looks like this:

<target name="aggregateAssets">
  <!-- read the aggregation properties -->
  <property file="${propfile}" prefix="agg" />

  <!-- get the root -->
  <propertyregex property="agg.root"
    input="${propfile}"
    regexp="^(.*)\.properties$"
    select="\1" />

  <!-- split the root into file and path sections -->
  <propertyregex property="agg.fileroot"
    input="${agg.root}"
    regexp="^.*/([^/]+)$"
    select="\1" />
  <propertyregex property="agg.pathroot"
    input="${agg.root}"
    regexp="^(.*/)[^/]+$"
    select="\1" />

  <!-- set up the output file stuff -->
  <property name="agg.outfile" value="${rootdir}/${agg.fileroot}" />
  <property name="agg.cfmfile" value="${agg.root}.cfm" />
  <property name="minsuffix" value=".yuimin" />

  <!-- run everything through the YUI Compressor -->
  <for list="${agg.filenames}" param="filename">
    <sequential>
      <echo message="compressing @{filename} to @{filename}${minsuffix} (in ${rootdir})" />
      <java classname="com.yahoo.platform.yui.compressor.YUICompressor"
        failonerror="true"
        output="${rootdir}/@{filename}${minsuffix}"
        append="true"
        logError="true"
        fork="true">
        <arg value="--type"/>
        <arg value="${agg.type}"/>
        <arg value="--nomunge"/>
        <arg file="${rootdir}/@{filename}" />
        <classpath>
          <pathelement path="${java.class.path}"/>
        </classpath>
      </java>
    </sequential>
  </for>

  <!-- aggregate all the compressed files together -->
  <echo file="${agg.outfile}" message="// built by Ant using YUI Compressor" />
  <for list="${agg.filenames}" param="filename">
    <sequential>
      <concat destfile="${agg.outfile}" append="true">
        <header trimleading="true">
          // @{filename}
        </header>
        <filelist dir="${rootdir}" files="@{filename}${minsuffix}" />
      </concat>
    </sequential>
  </for>

  <!-- delete all the compressed files -->
  <delete>
    <fileset dir="${rootdir}" includes="*${minsuffix}" />
  </delete>

  <!-- write the CFM file to pull in the compressed and aggregated file -->
  <if>
    <equals arg1="${agg.type}" arg2="css" />
    <then>
      <echo file="${agg.cfmfile}"><![CDATA[<link rel="stylesheet" href="${agg.urlBasePath}${agg.fileroot}" type="text/css" />]]></echo>
    </then>
    <else>
      <echo file="${agg.cfmfile}"><![CDATA[<script src="${agg.urlBasePath}${agg.fileroot}" type="text/javascript"></script>]]></echo>
    </else>
  </if>
</target>

First, it reads the properties file, runs each listed asset through the YUI Compressor, and then aggregates the result. Finally, it overwrites agg.js.cfm (from above) with one that contains a single LINK/SCRIPT element to the aggregation result. End result is a single aggregated, compressed asset in production for speed, and separate uncompressed assets in development for easy debugging.

Edit: Do note that you'll need both the ant-contrib package and the YUI Compressor JARs to be installed into Ant for this to work.

13 responses to “Build-Time Aggregation of JS/CSS Assets”

  1. Jim Priest

    Good stuff. Added to my wiki (and my todo list)…

  2. Paul Marcotte

    Hi Barney,

    I was thinking recently that it would be interesting to run both js and css through the YUICompressor at build time as well. I'm an ant n00b so I wouldn't know where to start.

    How difficult would it be to add that as another target in the ant build?

  3. Paul Marcotte

    Ugh. Spot the fellow didn't read closely… This is very cool. I will definitely us this when I get up to speed with ant!

  4. Jim Priest

    Paul – I've got a section on my wiki about this – there are few ways to do it…

    http://www.thecrumb.com/wiki/ant#javascript_and_css_compression

  5. Paul Marcotte

    Thanks, Jim. I'll be sure to check that out. Excellent wiki, btw. Great resource.

  6. Ben Nadel

    @Barney,

    Looking very cool. This is way beyond my scope of experience, but I like what I think I see :) At the very least, I should check out this YUI Compressor.

  7. Adam Haskell

    One additional note, if memory serves me correctly, the "for" ANT task is not a core task of ANT and you will need the ant-contrib jar on the classpath as well: http://ant-contrib.sourceforge.net/

  8. Arjun

    Just what i was looking for. Would you have an example of this packaged in a Maven setup ?
    Ready to go too ? :)

  9. Arjun

    Thanks; I wasn't aware they provided aggregation also. Though one additional feature I'm looking for is the ability to replace SCRIPTS and CSS links from all files except the main template file.

    I use apache Wicket; which tends to allow Header Aggregation across multiple pages. Example:

    TEMPLATE PAGE:
    … import scripts for all site …

    CUSTOM PAGE extends TEMPLATE
    … import for the page …

    FINAL OUT PAGE :

    … import scripts for all site …
    … import for the page …

    Now if Im aggregating, what Id like is for the TEMPLATE link to be replaced in the HTML (fine that can be done manually easily) …. but more so to remove the Links that are being aggregated from the various pages.

    Thing is during DEVELOPMENT, its useful to know which script is responsible for which page. And hence I want this @ Build or Deploy time to run over the out WAR.

    I guess im going way beyond the scope of this intent. But if you know anything that can help in this also Id appreciate it.

    Thanks a ton

Leave a Reply

You must escape tags within your comment. Use &lt; for < and &gt; for >.