Some Good Specification Practices, or, I Am Out of Clever Titles

This post was co-written on Google Wave with my colleague Scott C. Reynolds, who makes me insane every day.  It is part of his How We Do Things series.

Where We Came From

We began our large LIS project in late 2004, starting with about 6 full months of writing specifications and designing the database. We created reams of documentation for each application complete with use cases that we knew we wouldn’t even implement until after the first version was released (sidebar: some of those still haven’t been done). We defined more than 100 tables of heavily normalized database schema. Before one line of code was written, we had spent six months of project budget.

As development continued on the project we found ourselves diverging from the specifications that had been created. The documents became little more than a roadmap, with the bold headers on functional groups and the basic usage stories becoming the guidelines that we followed. Much of the detailed specification and database schema went out the window as the system took shape and people began to see and give feedback on what was being produced. Too much specification gave us a false sense of "doing it right" and led us down many wrong paths. Balancing rework with completing planned features became a costly chore.

By the time the project was complete, it was clear that much of that up-front work had been wasted time and money. Had we top-lined the major milestones of such a large project, detailed a small set of functionality, and started developing iteratively, the system would have taken form much sooner, allowing for a tighter feedback loop and much less overall waste.

How We Do It Now

Lessons learned, we set about improving how we do specification. Scott already talked a little about this in the posts on planning, so please review those if you haven’t seen them yet (here and here).

When planning a new project of any size, we take an iterative approach. We start at a very high level, and strive to understand the goals of the project. What need does it serve? Who will be using it? Are we competing with something else for market share? Is there a strategic timeline? What’s the corporate vision here? What are the bullet points? We will fill in details later. We only want broad targets to guide us further.

When we get closer to development, we start to identify smaller chunks of functionality within each of those broad areas. This is still pretty high level, but done by business analysts and management on the IT team. We start to identify MMFs (minimal marketable features) and group them up in order of importance to help determine next steps.

MMFs in hand, we take the first one and start defining it further. This is where the information from the planning post comes in. We start to write stories (Cat has a great post detailing how to build effective stories). The other information gathered to this point sits dormant, with as little specification work done as possible, until such time as we are getting closer to working on it.

Over time, and only as needed, we put more and more specification on the system, and this is done in parallel with development. In fact, often the most specific information can only surface during development of those features, as they take shape, and as we understand how users will react. Specification should be every bit as iterative as coding.

Reduced to its essence, we JIT specification at many levels to allow maximum flexibility to change direction with minimal wasted work.

Gemba As an Essential Part of Specification

Lean has a concept of the gemba attitude and genchi genbutsu; essentially, being in the place where the action happens. Developers and business analysts have to work with the domain experts, in the place where the work happens, and observe the true workflows in place before hoping to design a system to support them. You cannot get this information from sit down meetings and emails with domain experts. You must go and see for yourself, or you will miss things.

It’s often the case that a team relies on "domain experts" to provide them with the specifications they need to build software. This is the BDUF way – gather in committee and have the all-knowing user tell you what to build. Fail.

My Experience With Gemba

Our dev team works out of the corporate office in Florida, while the actual laboratory is located in upstate New York. As a result, it is often an exercise of the imagination to figure out the best way to create things for them. Before I visited the lab, I relied on information from others, emails back and forth to the lab crews working all weird hours of the night, and my knowledge of the applications currently in existence. I really only ever got half the picture, and it was difficult to ensure that the things I was specifying would fit into the lab users workflow well without actually knowing what was going on up there. When I visited the lab my whole world was changed. Actually seeing how users interacted with the software and seeing ways they would work around what we didn’t have (open Excel or Word documents on their desktop, a wall covered in post-it notes). Just from walking around talking to people for 2 days, I probably got 50 requests. And they never would have asked for them; they would have just kept suffering. Being there showed me everything about how they worked and a million ways I could improve their lives. The experience was invaluable to both me and them, and each subsequent trip has just improved my knowledge of the way we work and the way they interact with our software.

                                                                                                            ~Cat

There is no substitute for a certain amount of domain expertise being resident in the team room. Your domain experts are experts only in their domain. They may know the workings of a histology laboratory inside and out, but if you ask them to design a system to support that lab, they’ll come back with something that looks an awful lot like excel and post-it notes.

 

postitwall

Yes, this is real.  Yes, I died a little inside.

In the next post we will talk specifically about the tools and techniques we use to specify work, and how we combine them to form complete pictures of a system (spoiler alert: it ain’t just stories).

Advertisements

4 comments so far

  1. Scott Bellware on

    I think this obsession that the software community has with “MMF’s” is going to be yet another bit of obsessive/compulsive consultant/speaker lexicon entitlement that will continue to slow down the software community’s understanding of managing product development. We can do better than being distracted by yet another orthodoxy. I think we should know by now that this kind of thing is our pattern and our susceptibility.

  2. catschwamm on

    While I see your point, as there are a lot of words like that in our industry, I’m really not concerned about the terminology. MMFs are really handy on our team because our developers can work on stuff in the same areas that they are currently working on; they don’t have to context switch and come back later to do things related to another feature. It gets work done faster and better, and we can see large swaths of functionality at a time instead of having to imagine what the rest will look like while we’re in a demo.

  3. Dave on

    “MMFs are really handy on our team because our developers can work on stuff in the same areas that they are currently working on”

    That’s an excellent point, Cat.

    Back in the cold blooded old days of XP, we’d call that working by theme. What goes around (that’s good) comes around (and is good).

  4. […] Using the Right Medium Posted November 13, 2009 Filed under: Uncategorized | This post was co-written on Google Wave with my colleague Scott C. Reynolds and is a follow up to our previous post about good specification practices. […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: