I just finished whipping up a quick little expense tracking app for Heather and me, and I'm again amazed at how wonderful the development experience is with FB3Lite, ColdSpring, and CFGroovy (Groovy and Hibernate for CFML). When I was finished with the core functionality, I couldn't figure out where the app was. There were my entities (in Groovy), my dsp_ files (in CFML), fbx_Switch.cfm, and a single CFC with a bunch of 1-3 line methods. It's absolutely ridiculous how much code Hibernate saves you from writing with it's three pronged attack: HQL, state-based persistence, and transitive persistence (in an out). It's further ridiculous how much code Groovy saves you in implementing entities. With those two in place, CFML simply shines implementing the UI layer (with a snippet or three of Groovy in the mix). Oh, and it's fast.
Monthly Archive for October, 2008
The current state of web applications is a fragmented mess. HTML is still the standard approach, but dynamic data loading with Ajax and application embedding with Flash are both incredibly popular, particularly for data-heavy applications.
HTML is great because it's easy to build and well understood by user agents, but you lack drawing capabilities. SVG addressed parts of that, and if XHTML ever gets adopted in any meaningful way SVG will see a resurgence. The canvas element has stepped in, but it's not universally supported.
Flash is the other end of the spectrum: you have extreme customizability, but developing applications is significantly more work and requires people with a higher skill level than for HTML apps. Flex, OpenLazlo, etc. attempt to ease the developer's pain, but it's still not nearly as easy as HTML. Silverlight (Microsoft's Flash competitor) follows the same vein, though with the addition of binding to Microsoft's technology stack. I'll only be refering to Flash from here out, because as far as I'm concerned here, they're basically equivalent.
Flash also has the significant downside that it's hard to go half way. Applications tend to be either all Flash or all HTML. Static HTML applications can add snippets of Ajax in an incremental fashion where they're best suited without going whole hog. Of course, if you want to go whole hog with HTML, that can be a lot of work too.
But what does this have to do with FlexChart, you ask?
FlexChart provides a way for dynamic HTML applications to leverage the drawing capabilities of Flash in a nearly completely transparent way. If you want to render a chart on a page in a static HTML app, you'll probably employ some server-side technology to build an image and embed that image in the HTML with the IMG tag. But if you want to control the chart's appearance with JavaScript, you're in a bit of a bind, because you have to go server-side to generate a new image and then update the image source with the new URL.
Google Charts provides something of a solution for this problem, because the URL of the image implicitly causes a chart regeneration. So you don't have to manually go server-side, it happens behind the scenes. But as anyone who has worked with the API knows, it can be a bit of a bear to figure out all your data ranges, encode your data, lay out your axes and labels, compute bar widths, etc. Plus, while the number of charts available is pretty good, they're fairly inflexible. For instance, you can't plot a trend line across a column (vertical bar) chart.
FlexChart provides what I feel is a better solution for many applications. It uses a simple XML dialect for specifying both chart structure and the data to display. It leverages the Flex's charting engine to take care of all the chart layout tasks so you don't have to do that. It's designed to be loaded once and draw multiple charts over the SWF's lifetime by passing it new XML descriptors via JavaScript (though it can also be used statically with descriptors from FlashVars). Since it's Flash, it also has the ability to provide interaction with the user. For example, when a user clicks a given data point on your chart, you can detect that with JavaScript and load a new drilled-down chart. The tradeoff is that it's Flash, and while Flash has good penetration, it's significantly behind PNG support (what Google Charts uses), particularly on mobile devices.
As such, FlexChart addresses a narrow slice of the inability/reluctance to build hybrid HTML/Flash applications by providing a JavaScript-centric API for drawing rich charts. Note that this API is not just for drawing static charts, you can register JavaScript functions for handling data tip rendering, click handling, and axis labeling. This allows a very high level of interactivity that I've not seen replicated in any other charting package for HTML/JavaScript applications with similar browser support (or really at all).
Another significant advantage is the technology-neutral XML dialect that FlexChart uses. There is absolutely nothing FlexChart-specific in the structure of the XML, so any charting engine could theoretically read and render the XML. I'm currently in the initial stages of a Google Charts-backed implementation that does exactly that. Yes, that means you can supply FlexChart XML to this new project and get back a Google Charts URL that displays the chart. Obviously it won't be a complete implementation because of limitations in Google Charts (like the mixed line/bar chart issue), but for supported aspects it will be equivalent. This will allow an application to render charts either as Flash (with rich interactivity) or as PNG images (for broad support) without having to change the format used to describe the charts.
Note that the XML descriptor language does NOT provide for the full gamut of capabilities that Flex's charting components have to offer. Nor will it cover all that Google Charts has to offer. The idea is that it provides a simple way to describe most common chart types, and do it in a rendering-neutral way. I've also gone to great lengths to hide much of the complexity that Flex (and soon to be Google Charts) foists on chart developers. For example, Flex forces you to create a huge number of redundant axis references when laying out complicated charts, but with the XML descriptor you can do it once at the top level. In the other direction, Google Charts forces you to explicitly specify all your axis labels and their positions, but the XML descriptor is such that they can be figured out for you by the implementation.
As I alluded to above, if XHTML actually gains adoption and SVG makes a comeback, that's another very likely candidate. I've used an SVG-based charting engine that I wrote five of six years ago on several applications (currently using Batik to conver the SVG to PNGs for display), so that would be another potential implementation. There are several canvas-based charting engines as well, all of which could be potential targets. Finally, there are preexisting XML-configured Flash-based charting packages (FusionCharts is one example). Leveraging one of those packages from the unified XML dialect would come for the cost of some XSL templates. The possibilities are quite promising.
FlexChart itself is only half of the puzzle. Sure, it's greate to have nice charts in a HTML/JavaScript application, but the bigger picture is just as important. Having a single way to describe charts across applications and rendering mechanisms is a huge win. Even more useful is bridging the gap between HTML and Flash with a way to create hybrid applications, instead of people seeing them only as two alternatives. This is a huge win for all web app developers.
I've released FlexChart 2.0 today. Binary download is available here: flexchart_2-0-r4392.zip. The demo app is still available, of course. Backwards compatibility is not quite 100%, as I'll explain below, but the non-compatibilities only affect certain classes of charts, and they're the complex ones. For simple stuff it should be drop-and-go.
There are no real outward changes to the engine. It's still shipped as a SWF and with a CFML demo app and custom tag. It's loadable by SWFObject if you prefer that, and since it's interplay with the world is strictly via JavaScript and XML, it's totally application platform independent. It's designed with Ajax in mind, as you might imagine, and fits into that realm very well.
Under the hood, however, you'll find all kinds of changes. The first version was a proof-of-concept app that I build one Friday afternoon and then extended with a few more bits over time. It did direct XML-to-UIComponent translation with no validation, no unit tests, and no debugging assitance. It was also a huge pain to work with, because the inside all operated on either an XML DOM tree or a ChartBase instance, neither of which is fun to debug.
The new version is a four phase engine: parse, transform, validate, build. Unifying them all is an intermediate representation that the parser creates, the transformer manipulates, the validator validates, and the builder uses to construct the ChartBase instance for actually adding to the stage. There are FlexUnit unit tests for the first three phases, along with a couple for the builder phase. I couldn't figure out how to test the builder reliably (because it's "output" is all UIComponents) until quite late in the process, so the coverage is only for a couple bugs that came up.
Because of the more explicit nature of the engine, error handling is far better. Internal errors caused by improper descriptors are also caught and displayed to the user for easier debugging. There is not currently a way to suppress this behaviour, maybe in a future version.
The rewrite also allowed me to hide some evilness lurking inside the Flex charting components. In particular, if you've used grouped series and the grouped legend, you'll possibly need to revisit things a bit. This is the one place where backwards compatibility was sacrificed, but the changes required in the XML are minor and a significant number of edge cases that the first engine couldn't handle are now dealt with correctly. Note that these weren't really bugs in the first version, but rather limitations in the Flex charting components that the first version didn't hide away from the user. I.e. in order to build certain charts you had to have an intimate understanding of the Flex charting components' internals. That's been fixed.
Bugs, comments, and questions welcome as always.
Just did the 1.6.0_10 upgrade on my server. Oh how I love Linux. Just unarchved the new JDK, repointed a single symlink and restarted my Java servers. There didn't seem to be a significant difference in spinup time for my ColdSpring AOP-heavy apps - perhaps a touch faster. No problems with Magnolia either as near as I've been able to tell. WordPress, not surprisingly, was unaffected.
I've been having troubles with thumbnail generation on one of my apps recently. Just sporadically, with no obvious pattern as to why. The error is about a missing Huffman code (used for JPEG compression). Turns out that certain images when being written at certain sizes, throw this error when you use the built-in imageWrite().
The solution is to skip the CF solution and manually do it the Java way. So what used to be this:
<cfset imageWrite(image, filename) />
now looks like this:
<cfset createObject("java", "javax.imageio.ImageIO").write(
imageGetBufferedImage(image),
"jpeg",
createObject("java", "java.io.File").init(filename)
) />
Fortunately, Adobe had the foresight to include the wonderfully named imageGetBufferedImage() built-in so you can access the raw image data with the native Java APIs. Note that with the Java you have to specify the encoding type explicitly, instead of letting CF figure it out based on the filename's extension. Since I'm only dealing with JPEG images, I just hard coded it.
I use Google Spreadsheets for a few things (and other doc types for a few others), but I usually do hard-core editing in Excel. Export from the web, edit in Excel (or more likely Calc), reimport into Docs. Hardly elegant, but when I'm going to be cranking a workbook for two or three hours it's worth it compared to editing online.
I was getting set up to do just that this evening, and it occurred to me that the online experience probably doens't suck nearly as much in Chrome. So I went and grabbed it, set it up with the latest dev build (instructions), and fired 'er up.
Much to my delight, it's actually quite good. They also added a multi-line editor for cell values that you can resize as needed, which makes editing huge formulas way easier. No more copying out of the Excel bar into TextPad to edit a formula and then copy it back in. Still a few issues with keyboard navigation, but they seem to be bound to the fact that "type in the cell" and "edit the cell" are expressed in very similar ways, so you can't tell which mode you're in without thinking pretty hard. I'm trying to figure out what's different with Excel (where I don't recall having this problem), but I can't put my finger on it.
Spreadsheet's filter function is also pretty slick. Excel has roughly equivalent functionality in it's DB functions, but filter is way slick. You don't have to define a database range, it just figures it out, and you don't need explicit criteria cell pairs, you express the filter clauses directly in the formula. It took me a few minutes of struggling to figure out what the syntax really meant by the examples in the docs and how it applied to my problems, but it quickly became obvious once I willed the nasty Excel way out of my brain.
In any case, I think I'm done with the roundtripping via Excel. With a kick-ass JS engine behind it, the online version is close enough to a real spreadsheet app. Recomputing complex formulas is still slower than I could like, but that's about my only remaining complaint. And no, I didn't do any actual comparative profiling, but again, I don't recall seeing significant lag with Excel when I've done complicated stuff on there.
I've made another minor enhancement to FlexChart, this time around grouped legends. Previously, if you mixed grouped series and ungrouped series (e.g. a stacked column chart with a line series overlaid), the legend would be all kinds of wonky for the line series. I've fixed that so it'll now render in a reasonable way.
As always, the demo app (with a new preconfigured chart illustrating the functionality) is available at http://barneyb.com/flexchart/, including instructions for downloading both the source and the binaries.
I've also received a couple questions about cross-platform support. There is absolutely nothing CFML specific about the component. I happen to package a CF custom tag with the SWF, and the demo app is written in CFML, but the component itself only deals with the Flash Player and XML. I've personally fed it XML with GroovyServlet and PHP, as well as ColdFusion and Railo.
A couple users complained that my CF Groovy demo app doesn't work on MS SQL Server because Hibernate doesn't escape the table names it creates, and "user" is a reserved word. Since I do all my work on MySQL, I never saw the issue.
I've updated the code to use an @Table annotation to specify an alternate table name ("user_t") to use, to alleviate the issue. No other code changed. If you update to the new code, your existing data will "disappear". In reality, it doesn't disappear, Hibernate just created a new blank table with the new name. If you want the old data back, just do a "insert into user_t select from User;" (or whatever the equivalent is), and then drop the "User" table.
As always, the demo is available at: http://barneyb.com/cfgroovy/, including instructions on how to get the latest code.
The iPhone has a number of problems, but the inability to background applications is one that just got fixed. There's a new app on Cydia for backgrounding an arbitrary application, called surprisingly enough, Backgrounder. It adds a couple behaviours to your home button for backgrounding, as well as supporting the ability configure apps to always background.
Obviously have to have a jailbroken phone to do it, but with QuickPwn it's a breeze. Now you can listen to Pandora while you surf the web or check your email just like you can with the iPod app. Or stay signed into chat while you go look up a contact. Now if only they'd fix copy and paste, attaching images to emails, text selection, etc.
