Ten years ago, being a web developer sucked.  Deployment was easy (rsync to production and done), but the tooling available to us was dismal.  And I mean crying-naked-in-a-snowstorm dismal.  Browsers were inconsistent, their built-in programming environment (read JavaScript) was reasonably functional but horridly slow, and hardware wasn't beefy enough to deal with scripting languages for hard-core number crunching.

But we had the Flash Player.  Flash provided an environment that beat all three of those problems, and beat them soundly.  It was consistent across browsers and operating systems, used a similar language to what we were used to (both JavaScript and ActionScript are ECMAScript implementations), and it let the developer compile the script into something a little lower level to run in a dedicated VM on the user's machine which meant it was faster.  Of course, Flash is an animation toolkit, but we figured out how to bastardize it with the single-frame movie with a include-and-stop script in it so we could build applications entirely with script (like we wanted), but leverage the Flash player to actually run them.  Not to mention the rich support for visual stuff.  All this led to the concept of a RIA (Rich Internet Application).  Something similar to what we had on the desktop, but deployed to the web with all the benefits (and some drawbacks) that has.

Then browsers got their act together.  We started seeing a unifying focus on application development with the browser as the environment.  People got serious about fast JavaScript runtimes.  Standards were written (e.g., CSS2/3, Canvas) and largely adhered to.  And hardware got faster.  JavaScript application frameworks (EXT, YUI, GWT, etc.) showed up to leverage all that, and now it was possible to build RIAs using standards supported by a wide array of vendors.

In order to compete with this, Adobe released Flex, which is an application development framework for deploying to Flash Player.  It was horridly expensive, difficult to work with, had all kinds of implementation problems, but was better than what you got with JavaScript.  For a while.  Unfortunately a single software company, even one of the largest in the world, couldn't hope to compete with the widespread interest and momentum around browser-native RIA development.  Flex died as a web application framework pretty much before it was released.  Which isn't to say it wasn't used (it was and still is), but the browser RIA juggernaut crushed it like a bug.

Then we saw the huge surge in mobile devices.  It started with smartphones and now includes tablets and e-readers of various form factors.  Fortunately for both consumers and manufacturers, the web wasn't new, so they were able to jump right on top of all the standards and browser capabilities which had been created for the desktop.  A huge market segment opened up for web application developers and browser-native was reasonable right out of the box.

Adobe again tried to compete by creating Mobile Flash Player, but the benefits of Flash Player are small on new mobile devices, especially considering that so much of a mobile device experience is through web-connected native applications, not traditional web applications.  And here we have Adobe's smartest move yet around Flash Player: killing it for the mobile market in favor of restructuring the ecosystem around using it as a development environment for native applications.

Unfortunately, I don't think it's going to matter because Flash is still really slow and heavy and it's not really that much better to develop with than the truly native dev kits.  Yes, it offers the promise of cross-platform deploy, but just as Java demonstrated 10-15 years ago, that isn't much of a selling point for a single application.  People expect not just native execution, they expect native idioms, which means you have to develop for a specific platform, even if you're using a cross-platform toolkit.

So where does that leave us?  As consumers it leaves us in a great spot: there are lots of ways developers can deliver engaging applications to us, on all our devices.  As developers it leaves us in a weird spot: we're stuck with being either web or native developers, with Flash now trying to occupy a sort of middle ground (develop web-style, deploy native).  On the desktop I think Flash (via AIR) holds some promise, but ultimately I don't think it'll last.  The platform just isn't compelling enough to justify the dedication it requires to use it.  Java was designed for very much the same purpose as AIR, and virtually no one uses it.  Java moved almost entirely server side, something which Flash isn't likely to do.

Most interesting is the PhoneGap/Titanium/etc. movement which is very much paralleling the browser resurgence I talked about earlier.  Huge communities of people are working to take all the skills we have as web application developers and give us a build process to take web-ish apps and compile them into native applications, in much the same way Adobe has use AIR to compile Flash into native apps.  However, I think Flash is going to lose in exactly the same way, and for exactly the same reasons, as it lost in the browser.

Bottom line, if you use the web or a web-connected device (read: everybody) the world is going to be glorious in a couple years and only get better.  If you're a developer trying to work in that space, you need to learn browser technologies.  It's the way of the future.  Flash had it's run, but it's been on the way out for a long while.  It'll stick around, just like COBOL and Fortran have, but without an alternative path like Java ended up taking it isn't going to stay relevant to mainstream developers.

HTML is great because it's easy to build and well understood by user agents, but you lack drawing capabilities.  SVG addressed parts of that, and if XHTML ever gets adopted in any meaningful way SVG will see a resurgence.  The canvas element has stepped in, but it's not universally supported.

Flash is the other end of the spectrum: you have extreme customizability, but developing applications is significantly more work and requires people with a higher skill level than for HTML apps.  Flex, OpenLazlo, etc. attempt to ease the developer's pain, but it's still not nearly as easy as HTML.  Silverlight (Microsoft's Flash competitor) follows the same vein, though with the addition of binding to Microsoft's technology stack.  I'll only be refering to Flash from here out, because as far as I'm concerned here, they're basically equivalent.

Flash also has the significant downside that it's hard to go half way.  Applications tend to be either all Flash or all HTML.  Static HTML applications can add snippets of Ajax in an incremental fashion where they're best suited without going whole hog.  Of course, if you want to go whole hog with HTML, that can be a lot of work too.

But what does this have to do with FlexChart, you ask?

FlexChart provides a way for dynamic HTML applications to leverage the drawing capabilities of Flash in a nearly completely transparent way.  If you want to render a chart on a page in a static HTML app, you'll probably employ some server-side technology to build an image and embed that image in the HTML with the IMG tag.  But if you want to control the chart's appearance with JavaScript, you're in a bit of a bind, because you have to go server-side to generate a new image and then update the image source with the new URL.

Google Charts provides something of a solution for this problem, because the URL of the image implicitly causes a chart regeneration.  So you don't have to manually go server-side, it happens behind the scenes.  But as anyone who has worked with the API knows, it can be a bit of a bear to figure out all your data ranges, encode your data, lay out your axes and labels, compute bar widths, etc.  Plus, while the number of charts available is pretty good, they're fairly inflexible.  For instance, you can't plot a trend line across a column (vertical bar) chart.

FlexChart provides what I feel is a better solution for many applications.  It uses a simple XML dialect for specifying both chart structure and the data to display.  It leverages the Flex's charting engine to take care of all the chart layout tasks so you don't have to do that.  It's designed to be loaded once and draw multiple charts over the SWF's lifetime by passing it new XML descriptors via JavaScript (though it can also be used statically with descriptors from FlashVars).  Since it's Flash, it also has the ability to provide interaction with the user.  For example, when a user clicks a given data point on your chart, you can detect that with JavaScript and load a new drilled-down chart.  The tradeoff is that it's Flash, and while Flash has good penetration, it's significantly behind PNG support (what Google Charts uses), particularly on mobile devices.

As such, FlexChart addresses a narrow slice of the inability/reluctance to build hybrid HTML/Flash applications by providing a JavaScript-centric API for drawing rich charts.  Note that this API is not just for drawing static charts, you can register JavaScript functions for handling data tip rendering, click handling, and axis labeling.  This allows a very high level of interactivity that I've not seen replicated in any other charting package for HTML/JavaScript applications with similar browser support (or really at all).

Another significant advantage is the technology-neutral XML dialect that FlexChart uses.  There is absolutely nothing FlexChart-specific in the structure of the XML, so any charting engine could theoretically read and render the XML.  I'm currently in the initial stages of a Google Charts-backed implementation that does exactly that.  Yes, that means you can supply FlexChart XML to this new project and get back a Google Charts URL that displays the chart.  Obviously it won't be a complete implementation because of limitations in Google Charts (like the mixed line/bar chart issue), but for supported aspects it will be equivalent.  This will allow an application to render charts either as Flash (with rich interactivity) or as PNG images (for broad support) without having to change the format used to describe the charts.

Note that the XML descriptor language does NOT provide for the full gamut of capabilities that Flex's charting components have to offer.  Nor will it cover all that Google Charts has to offer.  The idea is that it provides a simple way to describe most common chart types, and do it in a rendering-neutral way.  I've also gone to great lengths to hide much of the complexity that Flex (and soon to be Google Charts) foists on chart developers.  For example, Flex forces you to create a huge number of redundant axis references when laying out complicated charts, but with the XML descriptor you can do it once at the top level.  In the other direction, Google Charts forces you to explicitly specify all your axis labels and their positions, but the XML descriptor is such that they can be figured out for you by the implementation.

As I alluded to above, if XHTML actually gains adoption and SVG makes a comeback, that's another very likely candidate.  I've used an SVG-based charting engine that I wrote five of six years ago on several applications (currently using Batik to conver the SVG to PNGs for display), so that would be another potential implementation.  There are several canvas-based charting engines as well, all of which could be potential targets.  Finally, there are preexisting XML-configured Flash-based charting packages (FusionCharts is one example).  Leveraging one of those packages from the unified XML dialect would come for the cost of some XSL templates.  The possibilities are quite promising.

FlexChart itself is only half of the puzzle.  Sure, it's greate to have nice charts in a HTML/JavaScript application, but the bigger picture is just as important.  Having a single way to describe charts across applications and rendering mechanisms is a huge win.  Even more useful is bridging the gap between HTML and Flash with a way to create hybrid applications, instead of people seeing them only as two alternatives.  This is a huge win for all web app developers.

There are no real outward changes to the engine.  It's still shipped as a SWF and with a CFML demo app and custom tag.  It's loadable by SWFObject if you prefer that, and since it's interplay with the world is strictly via JavaScript and XML, it's totally application platform independent.  It's designed with Ajax in mind, as you might imagine, and fits into that realm very well.

Under the hood, however, you'll find all kinds of changes.  The first version was a proof-of-concept app that I build one Friday afternoon and then extended with a few more bits over time.  It did direct XML-to-UIComponent translation with no validation, no unit tests, and no debugging assitance.  It was also a huge pain to work with, because the inside all operated on either an XML DOM tree or a ChartBase instance, neither of which is fun to debug.

The new version is a four phase engine: parse, transform, validate, build.  Unifying them all is an intermediate representation that the parser creates, the transformer manipulates, the validator validates, and the builder uses to construct the ChartBase instance for actually adding to the stage.  There are FlexUnit unit tests for the first three phases, along with a couple for the builder phase.  I couldn't figure out how to test the builder reliably (because it's "output" is all UIComponents) until quite late in the process, so the coverage is only for a couple bugs that came up.

Because of the more explicit nature of the engine, error handling is far better.  Internal errors caused by improper descriptors are also caught and displayed to the user for easier debugging.  There is not currently a way to suppress this behaviour, maybe in a future version.

The rewrite also allowed me to hide some evilness lurking inside the Flex charting components.  In particular, if you've used grouped series and the grouped legend, you'll possibly need to revisit things a bit.  This is the one place where backwards compatibility was sacrificed, but the changes required in the XML are minor and a significant number of edge cases that the first engine couldn't handle are now dealt with correctly.  Note that these weren't really bugs in the first version, but rather limitations in the Flex charting components that the first version didn't hide away from the user.  I.e. in order to build certain charts you had to have an intimate understanding of the Flex charting components' internals.  That's been fixed.

Bugs, comments, and questions welcome as always.

As always, the demo app (with a new preconfigured chart illustrating the functionality) is available at http://barneyb.com/flexchart/, including instructions for downloading both the source and the binaries.

I've also received a couple questions about cross-platform support.  There is absolutely nothing CFML specific about the component.  I happen to package a CF custom tag with the SWF, and the demo app is written in CFML, but the component itself only deals with the Flash Player and XML.  I've personally fed it XML with GroovyServlet and PHP, as well as ColdFusion and Railo.

What is this good for?  I need to create a funnel chart, and while the built-in stacked-column chart is close, it computes the percentages in the data tips the wrong way.  Reimplementation was a snap with the context, just pull the relevant numbers out of the descriptor XML and do the division.

The ability to style charts has been extended a bit, though it's still not as highly polished as I could wish.  For example, supplying a stroke weight for a line series causes the stroke color to default to black, instead of the automatically assigned color (orange, green, blue, …).  In the reverse case, if you supply custom colors on a Pie chart, they render correctly, but the legend (if one is used) uses the default colors (orange, green, blue, …).  Gradient fills are now available as well.

I've also improved handling of empty charts.  The stock custom tag requires a descriptor, but if you don't have data at page render time, you usually end up providing "<chart />" as the descriptor.  The engine now detects this case (whether on the initial load or later passed in), and is a bit more intelligent about ensuring it clears it's stage.  Previously you could end up with an empty CartesianChart in some cases.

Finally, I made a number of improvements to performance and handling of data values.  This was mostly accomplished by explicitly converting the XML nodes into real objects for the chart to render, rather than using the XML directly.  There were some implicit type conversions that didn't happen consistently out of XML nodes, but work fine out of generic objects.

With the coloring support, the 'Grouped Series' example makes a lot more visual sense, so if you thought WTF yesterday, go look at it again.  ;)

First is grouped series, which is hard to explain, but easy to understand.  Go hit the demo and select "Grouped Series" from the dropdown.  The chart layout has always been possible, but the legend just displayed a flat series list with no awareness of the groupings.  To see the old behaviour, change the 'legendStyle' attribute of the chart tag to "simple", and hit "Update Chart".

Second is direct support for PNG exports of the chart.  A Base64-encoded PNG can be requested via an ExternalInterface callback which you can then do whatever you want with.  The demo app just sends it server side (in a new window) for deserialization with CFIMAGE and immediately sends it back to the client.  Hardly interesting, but definitely illustrative.

Finally, I've abandoned the compile-in-page model, Flex SDK 2.0.1, and a single file architecture completely.  You can drop the binary anywhere you want without any dependency on FlexBuilder, a Flex SDK, or any server component, which is nice.  Of course, if you want to customize, it required FlexBuilder 3.  I think that's the right tradeoff.  I've also started splitting up the engine for easier maintenance, now that there's no need to keep it single file (which there was when it was server-compiled).  I haven't done much, but that'll come.  Definitely make it easier to work with.

Soon to come down the pipe are custom colors for series and sets, support for multiple y-axes, and a grid legend style which is basically a headerless datagrid under the chart with a row per series and a column per x-axis item, all lined up evenly.

The repackaging was to a binary (precompiled) format, rather than requiring the server compilation I'd originally built it with.  The old style is still available where it always has been, but the preferred usage is now either the distribution directory (svn) or the FlexBuilder project itself.  With the new model, you just drop the compiled binaries (yours or mine) into any web-accessible directory and use them, regardless of whether you have LCDS installed to do the compilation.

Doing it this way means I lost the demo of having the chart request a new descriptor from the server-side on it's own, but the client-side descriptor injection is a "neater" capability, I think.   I haven't posted an updated zip of the source (including the demo app), but it's available from Subversion at the URL listed at the bottom of the demo.