How to Get Real in Corporate Development, Part VI

This is the sixth part of a series of essays on how to leverage the concepts of Agile software product development from Getting Real from 37signals in a corporate development organization. Part I introduces the concepts of Getting Real, part II looks at the idea of building less, part III is on focusing on one idea, part IV is on getting something functional out quickly and part V talks about focusing on how your users will use the application.

Now it's time to look at:

Focus on fulfilling business stories, not big specification documents.

In my 20+ years of technology experience, I have seen this as being the biggest and most pervasive problem. And I understand that it comes from a very valid perspective from the engineering community.

Engineers can not easily manage huge amounts of change to requirements. Imagine building a bridge if the length of the span changes weekly, the material under the footings shift regularly and the sponsor wants the bridge in steel; no, titanium; no, wood; no, aluminum; no, steel -- you get the point. Therefore, engineers seek to define the requirements and get agreement and sign-off on those documents so they can build something once. Plus, we all know that change gets more expensive as a project progresses, especially if you have a waterfall or spiral approach. All that inflexible foundation work is very expensive to change. (Another reason to stay light and agile!)

Because of the cost of constant change, engineers push to define requirements up front so that they know what to build and where to start. They require definitions, use cases, scenarios, storyboards, and so forth. From this collected material, they define the grand scheme, break it down into delivery phases and get agreement and sign-off. Again, they want to build something once.

The flaw with this approach is that the requirements cannot be entirely known in advance. Therefore, just as soon as the project starts, the huge requirements specification is out of date. I once worked on a project with over one thousand distinct requirements. Because we were inventing new science and technology while producing a commercial product, the requirements churn was huge. We touched every single requirement dozens of times; the requirements churn topped 1000%!! That's like writing over 10,000 requirements! It was nuts!

Here is the fix: Focus on business stories. Start with the most important story. It works like this.

The business problem is, "We have to generate an inventory spreadsheet daily by running a report in our inventory management software (IMS), copying the date and transforming it into Excel, then cleaning up the data by re-arranging the columns and saving it as a semi-colon delimited file so it can be emailed to the web services group so they can update the inventory counts on the website. Errors and delays in the process can mean displaying products as in-stock when they are currently back-ordered, which annoys customers."

The business story becomes, "We want to automate the daily inventory reporting from our IMS to our web server to increase accuracy, decrease information turn around time, and free business resources for other work."

I can immediately see a number of ways to do this, from using whatever automation exists within the IMS to generate and send that report, or accessing the IMS database directly with a custom report, or using the IMS database programmatic API to write an automated application to pull the data and update the web database, or to make the web database read its data from the IMS database live, or on a daily batch bases, or exposing a web service so the data can be pushed into the web database. I bet you have a few more options.

The take away is that solving the technical problem isn't usually the problem. The problem is solving the correct business problem. Getting bogged down with a huge requirements document doesn't make solving today's problem go any faster and doesn't produce a solution to today's problem any faster.

So, how would I apply this advice to that huge 1000% churn project? Well, the biggest constraint was that we had to produce a commercially shippable product. A goal was to create a single architecture that could be leveraged along the future product line. However, since we were creating new technology and a new command and control system, we set the bar too high.

The immediate business story was to operate the machine and provide a sufficiently flexible control system that would allow for the scientific invention to continue without requiring huge software re-writes. Today, I would drop the goal that we create an architecture that could handle future demands. The project was already very challenging without that goal and we ended up pulling out all the infrastructure that was built to handle that goal anyway, just to get the first product to ship.

We made the classic mistake of solving tomorrow's problem before solving today's problem. You can avoid that mistake by focusing on the business stories that you have today and not creating that huge, over-arching requirements document that will doubtlessly saddle you to a very sizable ball and chain.