Impetus for an OO Backend

After a long, multi-faceted discussions on CFCDev a few weeks ago,
one of the participants contact me off-list wondering about a sample
app that illustrated some of the concepts I'd mentioned in the
discussion. I don't have one, and while I could make one up, the
implementation is the easy part. It's the reasoning behind doing
things a certain way that is what's important. So what I'm going
to endeavor upon is a quick overview of how I've come to building apps
the way I have. It will undoubtedly be long winded, but I'll try
to be as brief as I can. ;) Hopefully at the end you'll see
three things:
First, OO is hard to do 'right', since 'right' is both subjective and
ever-changing, second, that the 'right' way is always defined by need
and not by how well you use Design Pattern X, and third, that I don't
have any magical OO power, I've just spent the past three years
tuning a single app and building a large body of experience.

To
put the cart before the horse, here's an image that roughly lays out
what I'm going for. At the top we have the Presentation and UI
Controller layers. The first is just HTML, the second is a
framework of your choice (I use FB3 primarily). The dark blue
section, however, is what I'm going to be talking about. And note
that this diagram is about how the tiers work, not necessarily how the
implementation is structured.

First some back-story. I took over an app that was utter spaghetti
code. About 80,000 lines of it. I spent a month converting
everything to Fusebox 3 (the current version at the time), and that
dropped to about 50,000 lines simply from code reuse. (ed.
note: That's a 37% decrease in code size; use a framework people) There was no MVC in the app, it was all single
fuseactions, so the only reuse happened at the fuse level. This
is all on CF4.5, and when CFMX 6.1 came out (and alleviated most of the
bugs and inadequacies in CFMX 6.0's CFC implementation), we upgraded,
and the CFC-ization began in earnest.

My first objective was to stop having
to write so many damn queries, so automated entity persistence was top
of the pile. Fortunately, that's really easy to do, since entity
persistence operations are very closely allied to your database
schema. A few hours of working and I'd built a generator that
would read a table schema from the DB and generate a skeleton BO and
fully implemented DAO for performing persistence operations for
it. It'd also generate a boilerplate factory/manager for the
entity type (getNewUser, getUserById, createUser, deleteUser,
updateUser). With that, I no longer had to have any single-entity
queries in my fuses, rather I request a BO from the appropriate manager
(all of which were singletons in the application scope), do what I
needed, and then call createXXX or updateXXX on the manager with the
modified BO. That saved me enormous amounts of time, particularly
with changes to entity fields (like adding a country to user info),
because I could just regenerate the DAO, and all the persistence
operations were magically updated.

That saved a lot of work, but
it didn't help abstract the business logic out of the UI. My
fbx_Switch.cfm files were still littered with a mix of business logic
and UI processing. So the next step was to start creating service
objects to put the business logic in. This became doubly
important since about this time we started exposing certain
functionality over web services as well as the HTML UI, and that was
starting to lead to enough duplicate logic code to trigger warning
bells. So much of the business logic moved into the service
objects.

An important point to make here is that while managers, BOs, and DAOs are
all pretty much one-to-one-to-one, services are not. An example
would be a permission-based security model, where you have users,
groups, and permissions. That's three distinct manager/BO/DAO
sets, but you probably only need a single SecurityService that deal
with all of them. And you'll probably have a LoggingService that
doesn't care about security, but does care about some aspects of users.

As soon as service objects popped into existence, however, up reared
a major issue with encapsulation. A good issue, mind you, but one that
required creating a solution for. Simply put, the services needed to be
able to talk to each other, but couldn't do that without going to the
application scope to get a reference. The same thing applied to manager
instance as well. The "right" solution, like so many other things, was
driven by time constraints not good design (though it was good enough).
At this time, all managers (and now services) were instantiated into the
application scope directly (i.e. application.securityservice). That was
quickly revised to instantiate them into a managers and services struct
respectively, copy the keys into the application scope, and the pass the
struct into each service (via a setManagers and a setServices method in
the AbstractService superclass). None of the existing app needed to change,
but all the services had references to everything else, and it took all
of ten minutes to put together.

The right solution (from a design perspective) is to use a factory,
and then just pass that factory around. The struct solution is similar
in concept, though not nearly as encapsulated. Needless to say, the
lesson was learned, and factories are now properly used from the start,
rather than being an afterthought.

Ok, time for a breather/cigarette/shot of whiskey. Whew! And here
we go again…

The next problem was that the services often needed to do arbitrary queries
as part of their business logic. This is a two-pronged issue:
SELECT queries, and other queries. And with the SELECT queries,
there are queries that the UI will also need (which still resided in
qry_ files at this point), and others that are strictly needed by the
backend. Along came gateways to solve the first part of the problem.

The set of gateways falls somewhere between the set of services and the
set of BOs in makeup. For example, a UserGateway for users and a
SecurityGateway for groups and permissions. Gateways were also the first
objects to be created via an application-scope factory and use lazy loading
for faster app startup. The methods of the gateways included all the SELECT
statements needed by both the UI and the services, though that may change at
some point in the future.

The problem of the non-SELECT queries was solved by deciding to let the
services perform those queries directly against the DB. It's worth mentioning
that this was only for queries that weren't tied to a single entity; those are
always performed through the entity. Initially, I thought this was a rather
poor way of handling it, but upon further reflection, I've become pretty
comfortable with it. The other solution would be dedicated DB modifier object
that the service delegates to. I'm not sure it's worth the complexity,
unless having all your SQL in SQL-specific objects is important for your app,
because you lose the ability to modify the business logic all in one place.

So now we've got business logic in the services, but we have BOs
(Business Objects, mind you) that are little more than DTOs
(Data Transfer Objects) for easing persistence operations. So entity-specific
business logic was moved into the entities from the services. An example
would be a 'post document' operation. Previously, the DocumentService
would pull the Document BO, set 'isPosted' to true, 'postDate' to now(), etc.,
persist it, and then go on with the other tasks (like distributing
notifications, or clearing the cached list of "recent updates" for the
site). With the new setup, the DocumentService simply calls postDocument()
on the Document BO, which takes care of all the document-related stuff in
one magic step. The service is still in charge of the other non-entity
operations, but all the entity operations are now part of the entity.

So what do we have at the end of it all? A rather complex arrangement of
CFCs (all told, about 200) that makes working with the application much easier.

What other things would I like to see? A real centralized application object
that contains the entire application, and can be passed around instead of the
structs of services and managers for one. This is already partially implemented,
but it's not complete. I'd also like to see user security and logging be
integrated in a more transparent way. Right now, both must be explicitly coded
for, which has led to errors in the past. I'd much rather have that applied
magically by some framework (probably dynamic wrapping of the service objects)
so that it's guaranteed to be consistent across the board.

What about problems? Funny you should ask. ;) Probably the biggest problem
is dealing with DB transactions. CF only exposes the CFTRANSACTION tag, which like
other tags, acts upon it's body. More to the point, there isn't a way to say "start
a transaction, if one isn't already active." Certain business operations
are both standalone operations and part of larger operations, and can be invoked
either way. The solution we've employed is to have both transactional and
non-transactional versions of those methods (the transactional method being nothing
more than a CFTRANSACTION tag wrapping a call to the non-transactional version).
It works, but it's hardly elegant, particularly with methods that take a lot of
arguments.

One solution that I tried out on another app I threw together was to manage
transactions via TransactionManager object that didn't use CFTRANSACTION at all,
but rather use CFQUERY to talk to the database directly. It worked fairly well,
but it depends on an implementation detail of CFMX. Namely that a given request
gets a single DB connection for the duration of the request, and that it is the
ONLY request that has access to the connection. Just for reference BD doesn't
have this feature. It also gets complicated because you must
ensure that your transaction either gets committed or rolled back before the request
finishes, or you'll have some weird issues. However, it does allow you to have
the "start a transaction, if one isn't already active" functionality, which makes
things ENORMOUSLY easier. I'd really love to see this behaviour appear in future
versions of CF, but who knows.

Another problem is that the sheer weight of a complex system can make otherwise
simple tasks fairly daunting. In that case, as long as encapsulation isn't broken,
we simplify. There are a few subsystems that are comprised only of a service object
that contains everything. To come clean, the 'permission' entity from my security
examples above is one. There isn't a permission manager, BO, or DAO, just the methods
in the SecurityService. These are also a major source of the non-SELECT queries that
the service objects need to perform. But like everything else, it's all about picking
the right compromises in each situation. If we need to have a full OO implementation
of permissions, injecting it will be very simple, since the service methods won't
change at all, they'll just stop doing everything themselves and start delegating to
the new backing objects. And that's the real power of encapsulation.

Just for reference, that flat, service-only architecture is usually where I start
with everything because it's quick to develop, and easily facilitates growth down the
road. One good (and publicly known) example is Ray's BlogCFC. It's all in there as
one massive file and that's exactly as it should be, particularly since installation
simplicity is important.

So, after much typing, I'm calling it quits. Hopefully I've met the three
objectives I laid out at the beginning, and haven't caused anyone to shoot smoke out
their ears or pass out on their keyboard. At some point, I may actually show some
code, but I'd like to at least pretend that anyone who gets example code will have
understood the reasons behind why it is the way it is.