Monthly Archive for July, 2008

iPhone and Wi-Fi

The wi-fi on the iPhone is totally kick ass. I was really impressed. At home I had to manually type in my long hex WEP key, but at work (where it's a pain) the phone detected an "enterprise wi-fi" network, prompted me for my Active Directory credentials and that was it. Note that this is a iPhone getting in to a Windows environment.

With my laptop (which is a Windows) machine, I've completely failed to get on the wi-fi, instead resorting to an Ethernet cable to a spare jack. Seems silly. So props to Apple for getting this dead on.

On the typing front, I've decided that the one finger approach is way faster than trying to use my thumbs. The phone just moves around too much when I use my thumbs, so I miss a lot. I can't hit the keys quite as fast with one finger, but it's WAY more accurate.

From My iPhone

I finally got my iPhone 3G today, and am trying out the WordPress app. Typing kind of sucks, but I'll live. I've also attached a photo of my desk at the office, just to see how that works.

photo

Weka Mining Update: It Works!

Back in April, I posted about how I was using Weka to do some asset prioritization.  The gist of it was that users would rank assets on a 1-5 scale, and then then the system would recommend other assets that it thought they'd like.  This is the Netflix problem, if you're familiar with that, though without the time component.

Over the intervening months, I've gone back and tuned the mechanism slightly.  The algorithm hasn't undergone any fundamental changes, just made a few little tweaks here and there.  A cluster of related assets doesn't tell you which one a given person wants, but if you can figure out which cluster they like the best, other assets from that cluster are probably desirable.  The specifics of ranking the clusters and selecting a best fit is where most of those tweaks were made, not to the Weka implementation itself.

Here's a chart that show average rank and total ranks broken down by month:

As you can see, before the Weka switchover, the average rank for assets was  hovering right around 3.2.  That trend actually goes back a couple more years, I'm just not showing it here.  Once I started using Weka, that has gone up significantly, and is now around 3.7.  That's a pretty significant increase, especially when you consider that some asset selection remains totally random to help seed the asset pool.

MX Unit is Slick

After seeing that MX Unit on Railo had some issues, I switched over to ColdFusion and I'm impressed.  Marc, Bill, et al have done a really great job.  They've resolved a number of issues that I had with CFCUnit, and really embraced the "Integrated" in "Integrated Development Environment."

I know I'm late to the party, but it's good stuff.  When I was doing a lot of Java, having JUnit right there as a fast view in Eclipse was great.  Having to go launch a browser, find the runner app, type in the test case I wanted, and then hit submit with CFCUnit just seemed to stupid.  And if you don't want (or maybe don't have) the plugin for a specific test, running it from the browser is a snap.  Just hit the 'runTestRemote' method of the CFC and you're done.  No external runner app required.

One thing that I found confusing was webroot, component path, and facade location settings.  It seems focused on one Eclipse install per CF instance, while I pretty much run one project per CF (or Railo) instance.  And having to specify the remote facade explicitly seems unnecessary.  You should be able to give it at the root URL of the application, and it'll figure out the default one from there.  Obviously specifying a full path would be desired, but for the default, having to figure out what to type seems unnecessary.  Once I figured it all out, it wasn't a big deal, but it seemed like more work than it ought to be.

Finally, the UI is kickass.  Not that a web-based test runner really needs a nice interface, but it's appreciated.  And having the ability to switch between the rich Ext-based UI and the simple HTML one (along with the XML formats) is appreciated.

No MXUnit For Railo

I just pulled down the 1.0 release of MX Unit, and was most disappointed to see that it only works on ColdFusion:

...blah...
no definition for the class ... [coldfusion.cfc.CFCProxy] could be found
...blah...

Since I do most of my CFML development on Railo now, it kind of leaves me stuck.  Fortunately, CFCUnit works flawlessly, but with it's definitely the less compelling choice of the two.

ColdFusion Struct Literals Fail Again

ColdFusion introduced CFML struct and array literals in CF 8.  They sucked.  Assignment only, no nesting, and the use of '=' for key-name pairs instead of ':', like every other language.  CF 8.0.1 fixed the nesting issue, but not the others.  I've been trying to figure out why it's an assignment-only construct since CF 8 was released, and today I figured it out.  It's stupid.  Take this code (which fails on CF8):

<cfset s = "cat,dog,bird" />
<cfset s = {
  first = listFirst(s),
  rest = listRest(s)
}/>

If you were to convert that to non-literal syntax, it'd look like this:

<cfset s = "cat,dog,bird" />
<cfset temp = structNew() />
<cfset temp.first = listFirst(s) />
<cfset temp.rest = listRest(s) />
<cfset s = temp />

Unless you're the ColdFusion (CF8 or CF8.0.1) runtime, in which case you'd convert it to this:

<cfset s = "cat,dog,bird" />
<cfset s = structNew() />
<cfset s.first = listFirst(s) />
<cfset s.rest = listRest(s) />

As you can see, the implementation is flawed - there's no way this can work.  Worse, it doesn't give you a useful error, it says "Complex object types cannot be converted to simple values", which if you consider the original snippet, is totally perplexing.

Note that this is a ColdFusion problem, not a CFML one.  Railo compiles it correctly (and it doesn't share the assignment-only requirement either).  OBD doesn't support literals at the moment.

CF Groovy 1.0RC (With Hibernate!)

I've just released 1.0RC of CF Groovy, including Hibernate support.  You can download it, or view the demo app.  The download includes both the demo and the runtime engine.

The big new feature is Hibernate support, of course.  Here are a couple snippets from the demo app.  First, the entity class:

package com.barneyb

import javax.persistence.*

@Entity
class User extends AbstractEntity {

    String firstName
    String lastName
    String email

    @Column(unique = true)
    String username
}

Then the engine setup (in code here, ColdSpring would be better):

<cfset application.cfhibernate = createObject("component", "HibernatePlugin").init() />
<cfset application.cfhibernate.setEntityPath(
    getDirectoryFromPath(getCurrentTemplatePath()) & "entities"
) />
<cfset application.cfhibernate.setColdFusionDsn("cfgroovydemo") />
<cfset application.cfhibernate.setAnnotatedClasses(listToArray("com.barneyb.User")) />
<cfset application.cfgroovy = createObject("component", "cfgroovy").init() />
<cfset application.cfgroovy.addPlugin(application.cfhibernate) />

Finally, doing some persistence operations:

<cfset request.sess = application.cfhibernate.getSessionForRequest() />
<g:script>
import com.barneyb.*

variables.user = new User([
    firstName: attributes.firstName,
    lastName: attributes.lastName,
    username: attributes.username,
    email: attributes.email
])
request.sess.save(variables.user)
request.sess.flush()
</g:script>
<cfset userList = request.sess.createQuery(
    "from User order by lastName, firstName, username"
).list() />

As you can see, very minimal fuss to do some pretty powerful things.  This only scratches the surface of what Hibernate can do; the full power of the tool is available (custom annotations, the full XML mapping capabilities, caching, etc.).

Another significant change is that all non-inline Groovy is precompiled into Java classes and loaded dynamically.  This gives you compiler checks on your Groovy code and engine startup, more helpful message for code errors, and some performance improvements.

Finally, as illustrated above, the <g:setPath> tag has been superceded by an explicitly managed CF Groovy runtime.  This gives you complete control over the lifecycle of the engine, which lets you target performance in production and development speed in development.  You can still use <g:setPath> instead of managing the runtime yourself, but that should be considered a deprecated practice.

Thread.setContextClassLoader Doesn't Work on CF 8.0.0

I know the number of CF 8.0.0 installs is probably pretty minimal compared to CF 8.0.1, but thought this was worth pointing out.  If you use Thread.setContextClassLoader on CF 8.0.0, it raises no exception, but it doesn't actually set the ClassLoader for the thread.

I ran into this today using my CF Groovy/Hibernate integration on one of my existing CF installs that is still 8.0.0.  It totally bombs, because the classloading trickery fails if it can't set up a classloader.  After spending a bunch of time on it, I just gave up, as nothing I tried had any effect.

Both CF 8.0.1 and Railo 3.0b2 correctly allow overriding the context ClassLoader, and can therefore use the Hibernate via CF Groovy.  I haven't tried in a while, but OBD exhibited similar symptoms as CF 8.0.0 when trying to run Hibernate, so it may suffer from the same problem.  This only affects the Hibernate stuff I'm doing, the base Groovy integration requires no ClassLoader trickery, so it works on all four without issue (and probably CF 7 too).

I should also note that this is NOT the underlying JVM.  I use the same JVM (a 1.6 series from Sun) for all my stuff, both development and production.

Junction, from Sysinternals

I discovered Junction today, while looking for a way to easily test my Groovy/Hibernate integration on all three major CFML runtimes.  It's part of Sysinternals, which is a suite of tools aimed at making Windows not suck so much.  It's got all the low-level utilities you need to efficiently manage and monitor your Windows machines that Microsoft figured people would be happy to write and compile some C++ to use.

Junction, as you might guess, is a utility for managing NTFS junction nodes, which are the equivalent of *NIX symlinks.  With symlinks, I can have a single working copy "live" inside three different web roots, one for Railo, one for CF, and one for OBD.  Any changes made to that working copy are instantly available in all three places without any need to sync anything.

As you can imagine, this makes testing a breeze: just save and go refresh the tab for the runtime in question, and when it's set, go refresh the other tabs to ensure it still works everywhere else.  Until this point, I'd always used separate VMs, but that's way overkill for a standalone CFML app.  If you've got database and web server integration and all that, VMs are a lifesaver, but I just wanted to run the same 30 files in three web contexts and Junction is perfect.

Watch Out for CFQUERYPARAM and Garbage Collection

In the past few months I've spent a fair amount of time doing large data transformations for a couple projects, in to and out of MSSQL databases. In both projects, I've ended up refactoring my code to eliminate CFQUERYPARAM on oft-run queries, because it seems to leak memory (yes, I have debugging disabled), as well as be less performant.

In both cases, the primary offender was repetitive INSERT statements, ranging from 10,000 executions per request up to around 50,000. Stripping the CFQUERYPARAM tags gave about a 40% improvement in performance (batches in ~4 seconds instead of 7-8 seconds), and a huge improvement in memory consumption.

I've also noticed that while CF does do garbage collection within requests (thank god that got fixed), it doesn't do aggressive collection. A few well placed System.gc() calls have been a lifesaver for the larger processes, particularly when combined with judicious use of structDelete to eliminate non-essential references. Just using structDelete makes a difference, but the effect is magnified with the explicit collections.

NB: neither of these are solutions to be taken lightly; both have potentially serious side effects, and go against generally accepted best practices. Not using CFQUERYPARAM exposes you to SQL injection attacks among other things, and garbage collection is expensive in terms of processor utilization.

NB: these results are for ColdFusion 8 (not 8.0.1) against 64-bit MSSQL 9.0.3054 Developer Edition, using the stock CF driver. Other CF runtimes and other databases (and possibly even other MSSQL database drivers) will undoubtedly yield different results.