I've been doing a lot of system design and architecture work over the past year, and looking back, wanted to share some thoughts. A little background to start off with. The company I'm leaving in a couple weeks has a product that they've been selling since 1999, and it's showing it's age. Last January the decision was made to rebuild it from scratch, and that's what I've been doing since. Feature complete was slated for the end of the year, but that's been bumped a month or so, due in part to my imminent departure.
First, system architecture is friggin' hard. I don't claim to be a programming god, but I do consider myself fairly competent. It boggles the mind how many times I did stuff wrong over the past year. The mistakes ranged from incredibly minor to somewhat significant, but nothing that took more than a day or so to fix. All things considered, however, I'd say it went remarkably smoothly, and I'm quite happy with the outcome and how little time slippage there has been. In fact, most of the delays have been attributable to my unfamiliarity with the minutiae of Hibernate and SpringMVC and having to learn as I go. Both packages, however, have more made up for the delays with improvements elsewhere.
Second, I went into the project with a heavy dose of "build a fairly complete UML model first," and have abandoned that perspective over the year. I did do a pretty elabrorate model up front, and I'm not sorry I did, but the long term benefits are quite a bit less than I expected. Part of that's because the desired result has evolved over the year, but a lot of it has been the evolution of the system codebase. As time has passed, certain idioms that I'd designed have fallen out of favor and new ones have come to life. Certain technology pieces have also shaped the evolution of the product. The general shape of the original model is still visible in the code today, but most of the specifics have changed.
As my perception of a model's benefits have changed, my use of UML has also changed, and this is really what I wanted to write about. I do most of my design on paper these days, and most of the time the first and last things written on a given sheet are done less than an hour apart. Most of the papers end up burried under others within a day, and aren't looked at again until I realized I've got 40 pages of pictures on my desk and move the old ones to the recycling. I still use a lot of UML nomenclature in the drawings, I just do them by hand. One very interesting facet that I noticed was my member declarations on class diagrams.
What I found was that I almost never supply both a type and a name for a member. It's almost always one or the other, with the missing one implied. More specifically, simple properties and methods usually only have names, while compound properties (i.e. collections) only have types. I think that really drives home the fact that names (both type and member) are really important. I can attest that that's borne out over the life of the project. Intelligible and consistent nomenclature is incredibly important for developer sanity. The primary exception to the rule was collections of simple types, or multiple collections of the same type. List doesn't have much contextual information, for example, let alone if there are multiple properties with that type.
So what have I learned? UML is great, but trying to make a blueprint up front is folly. Much better to just sketch. If you name stuff well, you'll save yourself a lot of work. Don't expect to get stuff right up front. If your first shot is mostly right 60% of the time, you're doing awesome; if you're ever perfectly right, you're divine. Designing in a bubble doesn't work; you have to have conversations to drive the design, even if they're not with techies. Non-programmers can't articulate ideas with the clarity of a programmer, but they can still articulate them, and having to do the translation in your head makes you really get into the guts of the task at hand. And, of course, code is massively more expensive to refactor than some pencil sketches (or even some formal diagrams). That's really the whole point of separating the design/architecture phase from the coding phase, and it's essential for anything more complex than a static HTML "hello, world" app.
Great post. In fact, the concept I think is implicit in your post applies to the whole Software Development Life Cycle. That is, creating software from problem definition through implementation is not about a specific a technique, methodology or technology. Rather it's about understanding the problem(s) you are trying to solve and then using a good selection of the aforementioned to solve them.
To expand on this idea further, I recommend the article "There is no spoon: why the best software design and development process is all in your head [http://www.agileproductdesign.com/blog/there_is_no_spoon.html]", which I think captures this sentiment better than I can express.
Hi Barney,
Nice post and sounds like a cool project to have worked on! Just gotta say +1 for the pencil sketches. most of what I've been doing on my blog in the last month is "refactoring ideas" as there are certain very fundamental design choices that are extremely influential and a lot harder to refactor when embodies in code than when they're just a set of musings or whiteboard drawings. If you're just building another cms like the last 40 you did, it isn't so important, but if you're doing something either fundamentally new (or fundamentally new to you) I'd argue well over 80% of the project is done on paper.
It was like writing LightWire. I'm gonna say it took me 50 hours to really grok DI, and the code base took less than 10 hours to write – including a fairly length false trip into pseudo constructors.
Same with application generation. If the concepts are elegant enough, the rest is just metadata and a little bit of code . . .
Barney,
Nice post. Always glad to read about people's real-world experience with application architecture and design.
My question to you is this: How would you compare the Rational approach (UML, Sequence Interface Diagrams, Use Cases, etc..) with FLiP? In my experience, the Rational approach doesn't really fit well with 'web' application development (too time consuming, too much documentation that gets tossed out, etc..). On the other hand, FLiP has its own shortcomings in that it doesn't require (or even suggest) that the architects do much documentation beyond wireframes, prototypes, and fusedocs (or their equivalent).
A good question, Damon. I'm going to defer until I can spend a bit of time on the topic, as I think it warrants more than a "from the hip" comment. In a nutshell, however, comparing the two is like apples and oranges; they're targeted at rather different aspects of development.
Hey Damon,
I was just reading Eric Meyers domain driven design and I'm a fan of screen based elicitation of requirements for web apps. The truth is you don't need to know the domain. You only need to know how to create the screens, actions and steps to support the use cases for your application. We've developed a process similar to FLiP but a little broader (and honestly it was more influenced by our own experience, 37signals, Use Case modeling and Feature Driven Design than anything else). I wrote something about that for CFDJ:
http://coldfusion.sys-con.com/read/311316.htm
Bottom line is I think a screen based approach is typically more efficient for apps where UI is the core of the application and other approaches fit better for apps where UI doesn't capture much of the complexity inherent in the problem space.
Duh – take foot out of mouth. DDD is Eric Evans – Mr. Meyers is Mr. CSS. And while I'm not sure it was clear, I was suggesting that DDD is often NOT an optimal approach for web apps as you spend too much time learning stuff that you don't absolutely have to know. It's lovely if you have the time, but for the clients and price points I work with it's a luxury we can't justify!
Peter, I definitely agree. For simple web apps a UI-centric approach is quite appropriate. That's really the central tenet behind FLiP: the UI is the application. I wasn't very clear (mostly because it's obvious to me, I suppose), but the app I've been working is not of that paradigm. The primary objective of the process has been to create a application platform that the company can use for the next 7-10 years, not so much on creating a new version of what we have today. THe UI has been something of an afterthought, exposing enough to replace our current product, but leaving the majority of the advanced functionality hidden, waiting to be exposed down the road.