How to Get Real in Corporate Development, Part VI

This is the sixth part of a series of essays on how to leverage the concepts of Agile software product development from Getting Real from 37signals in a corporate development organization. Part I introduces the concepts of Getting Real, part II looks at the idea of building less, part III is on focusing on one idea, part IV is on getting something functional out quickly and part V talks about focusing on how your users will use the application.

Now it's time to look at:

Focus on fulfilling business stories, not big specification documents.

In my 20+ years of technology experience, I have seen this as being the biggest and most pervasive problem. And I understand that it comes from a very valid perspective from the engineering community.

Engineers can not easily manage huge amounts of change to requirements. Imagine building a bridge if the length of the span changes weekly, the material under the footings shift regularly and the sponsor wants the bridge in steel; no, titanium; no, wood; no, aluminum; no, steel -- you get the point. Therefore, engineers seek to define the requirements and get agreement and sign-off on those documents so they can build something once. Plus, we all know that change gets more expensive as a project progresses, especially if you have a waterfall or spiral approach. All that inflexible foundation work is very expensive to change. (Another reason to stay light and agile!)

Because of the cost of constant change, engineers push to define requirements up front so that they know what to build and where to start. They require definitions, use cases, scenarios, storyboards, and so forth. From this collected material, they define the grand scheme, break it down into delivery phases and get agreement and sign-off. Again, they want to build something once.

The flaw with this approach is that the requirements cannot be entirely known in advance. Therefore, just as soon as the project starts, the huge requirements specification is out of date. I once worked on a project with over one thousand distinct requirements. Because we were inventing new science and technology while producing a commercial product, the requirements churn was huge. We touched every single requirement dozens of times; the requirements churn topped 1000%!! That's like writing over 10,000 requirements! It was nuts!

Here is the fix: Focus on business stories. Start with the most important story. It works like this.

The business problem is, "We have to generate an inventory spreadsheet daily by running a report in our inventory management software (IMS), copying the date and transforming it into Excel, then cleaning up the data by re-arranging the columns and saving it as a semi-colon delimited file so it can be emailed to the web services group so they can update the inventory counts on the website. Errors and delays in the process can mean displaying products as in-stock when they are currently back-ordered, which annoys customers."

The business story becomes, "We want to automate the daily inventory reporting from our IMS to our web server to increase accuracy, decrease information turn around time, and free business resources for other work."

I can immediately see a number of ways to do this, from using whatever automation exists within the IMS to generate and send that report, or accessing the IMS database directly with a custom report, or using the IMS database programmatic API to write an automated application to pull the data and update the web database, or to make the web database read its data from the IMS database live, or on a daily batch bases, or exposing a web service so the data can be pushed into the web database. I bet you have a few more options.

The take away is that solving the technical problem isn't usually the problem. The problem is solving the correct business problem. Getting bogged down with a huge requirements document doesn't make solving today's problem go any faster and doesn't produce a solution to today's problem any faster.

So, how would I apply this advice to that huge 1000% churn project? Well, the biggest constraint was that we had to produce a commercially shippable product. A goal was to create a single architecture that could be leveraged along the future product line. However, since we were creating new technology and a new command and control system, we set the bar too high.

The immediate business story was to operate the machine and provide a sufficiently flexible control system that would allow for the scientific invention to continue without requiring huge software re-writes. Today, I would drop the goal that we create an architecture that could handle future demands. The project was already very challenging without that goal and we ended up pulling out all the infrastructure that was built to handle that goal anyway, just to get the first product to ship.

We made the classic mistake of solving tomorrow's problem before solving today's problem. You can avoid that mistake by focusing on the business stories that you have today and not creating that huge, over-arching requirements document that will doubtlessly saddle you to a very sizable ball and chain.

Weebly does web sites wite! Uh, Right!

I came across Weebly while reading some blog posting and recalled a conversation that I had recently with Jennifer Lee, the owner of A Doggone Good Life. Jen was a great program manager with a client of mine years back and is now a very happy entrepreneur pursuing her dream.

Jen and I were talking about some web-business ideas things, so when I read about Weebly and the professed ease with which one is able to create new websites, I had to give it a shot. After all, I've used web-site builders over the years. My general take-away is that they all sucked, sucked, sucked.

I tried Word to HTML generation (puke). I tried PDF to web-site generators (retch). I tried WYSIWYG generators (pain). I tried 1and1.com website builder (oh, boring ick). I tried NetObjects (workable, but still a PITA) and ColdFusion (there's a tool, but you need to be a serious web-head to make something worth while). The best was still coding a site by hand (slow and painful to revise) until one added some code generation (PHP, which apparently means "Powerful; Help Please") and Ruby on Rails (rocks).

The technically-light web entreprenuer does not want to learn coding paradigms, so out go PHP, ROR, ColdFusion, etc. What's left is for them to hire a company ($$) and pay them to maintain the site ($$$) or hire some full time staff ($$$$).

Nope, no more. Weebly makes very attractive and highly functional websites. SEO goodness, tags, sub-pages, custom HTML (for your easy shopping-cart functionality with PayPal). Also available are two-column layouts, YouTube, Flickr, Google Maps, forums, audio, a scheduler for your service-based organization, Google Adsense and Analytics, and an API to build custom widgets which your users can use on their Weebly sites.

There's enough serious good stuff that doesn't require a deep engineering background to grock to make me say, "Weebly is a great start".

Plus, Weebly will give you a FREE subdomain, or let you purchase a domain from them, or even let you use your existing domain (They even have instructions to point your domain from GoDaddy or 1and1 to their servers).

Oh, and let's mention that you can get started with a real website, e-commerce capable, with a blog and everything for FREE. Yes. Free. As in "you do not have to pay anything ever". They make their money on the premium services. And, the upgrade to Pro wasn't expensive. Something like $5 per month. Seriously, that's way cheep.

Check it out. I fast forwarded myself many years into the future and built a fantasy website called "Doggie Heaven" that I may actually do some day in the far, far future. I added some e-commerce functionality (don't actually buy anything, OK?) and a blog in only a few minutes.

There are a bunch of very servicable templates and I love the way elements are dragged and dropped onto the page for layout.

Very impressive, congratulations Weebly.

How to Get Real in Corporate Development, Part V

This is the fifth part of a series of essays on how to leverage the concepts of Agile software product development from Getting Real from 37signals in a corporate development organization. Part I introduces the concepts of Getting Real, part II looks at the idea of building less, part III was on focusing on one idea and part IV was on getting something functional out quickly.

In this essay we look at:

Start with the UI design and focus on how your users will use your application.

Lots of experienced engineers like to start with the UI. A new project brings a geyser of ideas and concepts that make the fingers itch to start writing code. So, mockups, prototypes and proofs of concepts are often the first things to come out.

Good! Almost.

Don't let your engineers close themselves in a room for the four weeks that they have and create something they love. The rest of the sentence is the key -- "focus on ... your users ..."

In "Getting Real", the folks at 37signals assume they are writing an appliation for the outside to use. They don't have an opportunity to sit with their most important users when creating something new. But you do! This is where the corporate environment make Getting Real better than the original. You're making software for your business, you and your users are the domain experts. You don't need to go outside. Lucky you!

Get your engineering team (2 really good engineers and a kick-ass business analyst) in the same room as the key stakeholders -- the folks who will use this new system. Understand the business problem (Focus on One Idea) by talking to the users and examining the problem they need solved.

Have the team work through what they need to do. What tasks are grouped to gether logically? What actions are done most of the time? What actions are done only a small fraction of the time? Make the application fix the problem elegantly. That way, the team has to Build Less and will be able to Get Something Functional Deployed Quickly.

Start light. Whiteboard and sketches. Wireframes and light-HTML. Test the process. Can the users do their important tasks quickly and easily? Can they see how they get to the additional features that they don't use every day?

Put the most important information right in the middle. Make sure that new state, the populated state and the error state are considered. Ensure the engineers make the application behave appropriately when it encounters something unexpected. Leave out what doesn't belong to do the task (who cares if it's a "standard" menu item?). Put in everything important. Leave out everything else.

Match and support what the users need to do, then go and make it happen. Start with the UI. Not the UI and the database, just the UI. Not the UI and the object model, just the UI.

The engineers will have a clearer idea of what's important. They will have proven the work flow that they are about to implement. The users will have increased confidence that what will be produced will really help them.

Can you just imagine the response when the first version comes out a few weeks later? Hey, please write and let me know! I don't want to do this in a vacuum!

Thanks again for reading! In the next essay, we'll talk about how to use business stories.

How to Get Real in Corporate Development, Part IV

This is the fourth part of a series of essays on how to leverage the concepts of Agile software product development from Getting Real from 37signals in a corporate development organization. Part I introduces the concepts of Getting Real, part II looks at the idea of building less and part III was on focusing on one idea.

Next, we talk about structuring the project to quickly produce working code. So here it is:

Get something functional and usable deployed quickly. Plan on your next release right away. Make a choice and go with it.

Constraints are what drive a project. If you had unlimited time and money, you would probably never produce anything, as ironic as that may sound. You need limits to focus you on getting something done. My triathlon training buddy and I like to say that we "train to race and race to train" because without the race date, it's too easy to sleep in instead of hitting the pool at 5:00 AM. Constraints drive us.

So, pick one of your projects and give it to a small group of good engineers. Tell them they have four weeks to produce the first version that the business can use, but then they'll have four more weeks to produce the next release. And close the door.

They will focus on building less and implementing one idea. It's all that they'll have time for. Meanwhile, let the business know that the first iteration will be ready for use in four weeks and that the engineers will need their feedback to produce the second iteration.

Your job is to get out of the way and make sure the engineers have the tools, access and permissions they need to do their work. Herein lies additional opportunity.

Perhaps there are procedure and process documents in your organization. Perhaps there is a PMO and a review committee. Perhaps you have strict regression testing requirements for deployments. Perhaps, under normal circumstances it would take your small, Agile group of engineers four weeks just to get permission to connect to the database they need to fulfill this project.

Your opportunity is to fix this. Yes, security and access control is critically important. But, we're in the 21st Century, aren't we? Network directories with access control list are mature technologies and are probably in your organization. Are your Oracle databases set up to take advantage of your LDAP infrastructure? They should be. And no one better exists than you to make it happen.

Your organization should have four environments:

• Development -- open and flexible for the developers to do anything. Developers should work in both Debug and Non-Debug environments. Don't accept "Well, it worked on my machine."
• Test -- mirrors production, but allows developers to see everything in the system.
• Staging -- mirrors production as closely as possible
• Production -- full security and access controls.

Using virtualization, farms, and similar techniques can keep this from being a horrible amount of overhead. The key is that you need to unlock your most productive engineers. Now is a good time to note that not all of your engineers are your most productive ones. Make sure your less productive engineers are fully supporting your most productive ones. There are opportunities for training and mentoring, which build employee satisfaction and loyalty, but let's get back to getting functional software out.

How many times have you already read that a skilled and senior engineer is many times -- 10 times, 15 times, 20 times -- more productive than a junior engineer? So, we don't need to say it again. Nope. Not going to say it. Na-uh.

So then, just how will this work? Well, let's start off by looking at the typical process. The business has an urgent need. Something like, "they spend a huge amount of time running end of month reports and small errors in the data cause regular heroics."

In the typical process they fill out an Application Enhancement Request or New Application Request or some such like that. This gets reviewed at the next Application Review Committee meeting and goes back to the requester for more details. Two weeks go by for the out and back trip and the ARC approves the request. The next week a project manager and business analyst are assigned to it. They take two weeks to gather requirements, another week to create a Statement of Work and one more to have that approved by the stakeholders. The following week, a project team is assembled while the analyst writes the Functional Specification. The stakeholders don't even bother reading that.

We're up to, um, nine weeks. The team takes two months to write the application, three weeks to test it, one week to slot it into deployment and, lo and behold! More than five (5!!) months later, the application is first released. The users try it and file a new Application Enhancement Request to get it to do all the things that they need and eliminate the stupid things that crept into the application. Time to happy users: 9 months.

Here's how we want it to run:

First, you've addressed the infrastructure, you have the governance in place to enable rapid development and you've now paired your best engineers with the business users we just talked about. They meet for one day and use up another afternoon or two. In four weeks the first release is out and in use. In four more weeks the next version comes out and fixes the big problems with the first release and adds a few more features. One more four week period and the business has something that really works for them. Time to happy users: 3 months.

Even if you give this team another three iterations -- that would be a total of six (6!) releases, it would still be 66% of the duration of the Same Old Way, which would have produced 2 releases. Which application do you think would be better? The one that had six tight, focused releases, or the other one? There is no doubt in my mind.

You need to have really great developers to do this. But then again, nothing outstanding was built by mediocre engineers. You need an infrastructure that enables these really great engineers. You still need to have proper governance, SOPs, checks and balances to put software into production, but you can get the work done much, much faster by Getting Real. And this will generate real value for your company.

Thanks again for reading! In the next essay, we'll look at starting with the UI.