Sandy Metz on Design

This is excellent: http://vimeo.com/26330100

Sandy talks about OO design and brings us through the thought process of how to get to loosely coupled, independent objects.  Highly recommended.

Don't Go There

A few years ago, I worked for a company that had a pool table. It was cool and different. A lot of edgy companies had Foosball tables and Ping Pong tables, but a pool table is a different beast altogether. This baby had three or four huge slabs of slate over an inch thick apiece that were laid out on a bed of stuff -- mud, clay, something -- and carefully leveled over the better part of an entire day. Then the stuff was left to set over night before the felt was applied.

We played pool to ease tension, introduce new employees to the company and, finally, to determine who was the "best". That is, we started a ladder.

As we played, we realized that we regularly getting ourselves into situations that were just tough to work out of. We didn't know what to do. Sometimes we tried to force our way through, sometimes we tried to finesse our way out. Usually, our efforts were too little and too late and our opponent trounced us.

So we got training. We brought in an instructor. A Pool Shark. The movies had nothing on this guy.

He wowed us and explained many aspects and strategies. At the end of his session, the inevitable question came up; "What do you do when you find yourself in a situation like this?" Then we would layout some of the most challenging plays in which we found ourselves.

He looked at the table and the balls laid out on it and a pained expression crossed his face. "I wouldn't let myself get into this situation," was his reply. "Never."

He knew enough about the game to know where the danger areas lay. He had played and practiced enough to know how to keep is game in the area where he can control it.

This story occurred to me as my colleagues and I struggle to stand up a large and complicated release that was pushed into production too soon. We know better. We know where the danger areas lie and how to avoid them. But we were pushed right onto them. A mandate was given and we did our best to march to it. Sometimes you just have to do that. But if and when you do, it is helpful to remember that you chose to ignore the evidence that it probably wouldn't work smoothly.

Just don't go there. Don't set things that aren't highly likely to succeed, especially when lots of dollars are on the line.

Join.me is a winner

We have a contractor who works remotely and I was chatting with him on the phone about the project to which we are both assigned. I was trying to talk him through some of the specifics of our administration tool when he said that this would be a lot easier if he could just see my screen.

First thought was iChat, but I already had Colloquy and Adium running and last time I tried iChat it wouldn't connect to the AIM server. (I've since fixed that by connecting to the AIM server on port 443).

Next, I remembered getting an email from LogMeIn about a new service for free screen sharing. I went to the LogMeIn site and found Join.me and clicked on it. It started up Java, and I downloaded some Java code and a small tool bar appeared at the top of my screen. It didn't seem to take even a minute. I pasted the link provided in the tool bar into my Colloquy chat window and my colleague clicked on it.

He didn't need to down load anything. The Java code runs on my system, pushes my screen data to the server and he got a Flash driven experience, complete with zooming and screen resizing right away. We never had any lag, though he was working from home quite a few states away.

Fast, Easy, Free, Unobtrusive, Highly Functional: Join.me is a winner

Bundler Install issue with Nokigiri on OS/X

I finally have an opportunity to move into the Ruby 1.9.2 and Rails 3.0 world having spent some time in earlier versions while supporting our main applications. Let's note that I know some other folks who are in a similar situation, which may have been predicated by our moving from OS/X 10.5 Leopard to 10.6 Snow Leopard by using the automatic migration rather than by manually re-installing all of our libraries, tools and applications. I had to do some rebuilding and patching just to get my Snow Leopard system up and running. However, this may be an hold-over from that process.

I used RVM to keep my Ruby 1.8.6 and 1.9.2 versions playing nicely. After installing Ruby 1.9.2 (wicked easy), I downloaded Rails 3.0.3, got the code base and ran 'bundler install' to install all the needed gems.

The process failed while building nokogiri. The error was:

libiconv is missing. please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.

Popping over to the supplied URL showed that I probably needed the libxml2 and libxslt libraries. I didn't think so because I was sure nokogiri was working in my other ruby environment, but nonetheless I ran:

sudo port install libxml2 libxsl

After doing so, I tried the bundle install again only to have the same error.

Now I was looking at a crisis. One colleague who has successfully navigated this waterway recommended that I rely on homebrew for package management. However, I've used port and fink, plus homebrew doesn't link to system directories by default, so I knew there would be some pain. I briefly considered uninstalling port and moving entirely to homebrew, but I have nearly 200 installed packages. Didn't want to go there.

Simultaneously, I was working in another terminal when I realized that my subversion was no longer working. It was complaining about a newly installed libiconv, which was installed with the libxml2. This actually was a good thing.

Now cutting to the chase: The Berekly DB needed to be updated in order to make all these things work. So I uninstalled Berekly DB, which involved uninstalling apr-util, python26, subversion, serf, and gone-doc-utils (using port for all of this). Then I did a 'port clean ' on all those packages and reinstalled everything.

Subversion now worked, but the bundle install still failed. Confident that the problem lay elsewhere, I cast around and found Chris Roos' post on Libxml2 Problems when Installing Nokogiri.

The solution I chose was to do a universal install of the libxml and libxlst libraries with

  $ sudo port upgrade --enforce-variants libxml2 +universal
 $ sudo port upgrade libxslt +universal

The bundle install worked after this, Subversion still works and I'm back in business.

Things not to say to the customer

Yes, I know this is irreverent, but I'm a bit punch happy, so here are some ideas about things that one should not say to the customer. Especially one who has been pretty good about getting slow support.

• After banging on this product for over a year, we’ve moved on emotionally, spiritually and strategically. The product is going to phased out soon and we’re really busily focused on doing things that generate a whomp-load more revenue right now. So, for right now, we’re just plugging our ears with coffee grinds and ignoring the problem
• My apologies for the delay. My mom says I shouldn’t talk to you.
• Thanks for following up on this. Unfortunately the work to dig into the product issues has temporarily been put on an indefinite hold.
• My apologies but a traumatic brain injury has wiped out all my memories and knowledge of who you are and what you do. Could we start over?

Clearly, someone in Sales needs to pick up the phone. Hello, Sales? Um, Sales?

Ah. They're out, erm, networking.

Rails alias_method_chain explained with pictures!

I've been meaning to do this for a while because I keep coming across alias_method and alias_method_chain in Rails code. I understood what was going on, but felt that I never grokked it. So, I finally mapped out what happens, drew some pictures, and, voila! Enlightenment!

Since I firmly believe that the best way to learn something is to teach it, I thought I would document what I learned in the hopes that it will be beneficial to someone else and firmly cement my understanding. So without further ado:

It is important to keep in mind that a method name is essentially a constant symbol and points to a block of code. "alias_method" is a Ruby class Module method with the signature:

alias_method(new_name, old_name)

And, "alias_method_chain" looks like this:

alias_method_chain :method_name, :feature

That's the easy part.

For the purpose of this example, I'm examining the behavior of:

alias_method_chain :render, :feature

The alias_method method makes a copy of the method block and assigns the new_name constant to point to the copy of the method block. So, it looks something like this:

Here's the method render and its method block:


After we call alias_method :render_without_feature, :render we have the following:
At this point, if we wanted to, we could redefine the render method, which would replace the original {render} block with new functionality. And, as it is often explained, the original functionality is still available at the render_without_feature method.

However, as it is often explained, what is desired in Rails library code is to redefine an existing method while saving the original functionality under a new name. Therefore, what we see in Rails code was often:

alias_method :render_without_feature, :render
alias_method :render, :render_with_feature

We already looked at the first line. Let's look at the second line.

The second line is assigning the constant "render" to a copy of the "render_with_feature" functionality. It looks like this:
Now, render will call the code block with the new feature and render_without_feature will call the original render block.

This is the key to the entire Rails helper, alias_method_chain, which takes the name of the original method and the name of the feature and does this mapping for us.

Therefore, the two alias_method calls above can be replaced with the following:

alias_method_chain :render, :feature

You, the fine engineer, will define a method render_with_feature and the render symbol will point to it. And, after the alias_method_chain call, you will have a new render_without_feature method created for you.

I find all of this rather elegant and hope this has been helpful

Fast Online Documentation - GotAPI.com

When I'm coding and have an API question, I have one and only one demand from my documentation source -- speed. I want to be able to quickly find the information I seek. "Now. I want it now."

I use PDF versions of my favorite books and the generated Ruby 2.3.4 API documentation locally, which is all good, but I found something that may be even better. GotAPI has a clean interface, is wicked fast and utterly intuitive. It even holds your recent searches.

In addition to lightning fast Ruby and Ruby on Rails look ups, it has tabs for HTML, Javascript DOM and CSS; all the things I need during RoR development!

I've added it to my browser bar so I can get to it quickly and it's rapidly becoming my reference tool of choice. Highly recommended.

Conditional state transitions with AASM

Also part of the same state machine I wrote about in "Chaining state transitions with Acts As State Machine (aasm)" required two different types of behavior for a state transition.

The task reviewer state was optional and would be triggered only if selected. I represented that state as a "use_reviewer" boolean column in the object. If use_reviewer == true, then the object transitions from task_created to "to_reviewer", otherwise it needed to transition to "to_writer"

The solution to this is rather straight forward. Use the :guard option in the aasm_event definition.

Here's the code:

aasm_event :created do
transitions :to => :to_reviewer, :from => [:task_created], :guard => Proc.new {|p| p.use_reviewer == true }
transitions :to => :to_writer, :from => [:task_created]
end

The event causes the object to transition to the "to_reviewer" state if the objects "use_reviewer" is true, otherwise it transitions to "to_writer". Here's the code evidence from script/console:

>> a = Assignment.new
=> (Assignment object)
>> a.use_reviewer = true
=> true
>> a.save
=> true
>> a.created!
=> true
>> a.aasm_current_state
=> :to_reviewer

Now we start over:
>> a = Assignment.new
=>(Assignment object)
>> a.save
=> true
>> a.created!
=> true
>> a.aasm_current_state
=> :to_writer

Chaining state transitions with Acts As State Machine (aasm)

Have you ever had state transition requirements that required automatic chaining of transition changes? I've got a system that transitions to a "to_reviewer" state after it has been created.

In the "to_reviewer" state, a group of reviewers are presented the assignment and they may accept the assignment, or a project manager may assign the assignment to a reviewer: in either case, as soon as there is a reviewer assigned, the assignment transitions to the "reviewer" state.

However, if the system already has a reviewer assigned at the "created" event, then it should cascade the state transition change and go directly to "reviewer" without staying in "to_reviewer"

Having selected the Rubyist AASM gem to implement my state machine in this Ruby on Rails project, I expected that I would be able to use the :after options of the aasm_state statement. I got that idea from the README.rdoc for version 2.1.1, which lists "event:after" in the callback chain.

However, the :after callback is called on an event (aasm_event) and I can't find documentation on how to hook into that callback. Digging into the the code, I found the following syntax works correctly in my Assignment object (derived from ActiveRecord). The solution is to use the :after_event option in the aasm_state definition statement.

This first part defines the states:
aasm_initial_state :task_created
aasm_state :task_created
aasm_state :to_reviewer, :after_enter => :check_reviewer
aasm_state :reviewer

This next part defines the events:
aasm_event :created do
transitions :to => :to_reviewer, :from => [:task_created]
end

aasm_event :task_review do
transitions :to => :reviewer, :from => [:to_reviewer]
end

This last part is the definition of the :after_enter method:
def check_task_reviewer
task_review unless reviewer_id.blank?
end

Now, when the object is created, the aasm_current_state is "task_created". If the object receives the "created" event, it transitions to the "to_reviewer" state. However, after the object enters the "to_reviewer" state, the "check_task_reviewer" method advances the state again if a reviewer as been assigned (indicated by the presence of the reviewer_id).

Checking the code in the console, we get:

>> a = Assignment.new
=> (Assignment object information)
>> a.reviewer_id = 5
=> 5
>> a.save
=> true
>> a.created!
=> true
>> a.aasm_current_state
=> :reviewer
And we can see that the object transitioned through the "to_reviewer" state an on into the "reviewer"state.

Rails Trick: Access an ActiveRecord field in a loop in a view

I came up with a little trick that I want to share, even though I'm sure there are better ways of doing this (perhaps someone will suggest them!)

OK, picture this: We have over half a dozen different types of workers -- Writers, Sr. Editors, Editors, etc. -- and we're going to pay them. So, in the database we have things like writer_payment, sr_editor_payment and editor_payment.

In the edit view, we have something like this (actually, there are many more cells, but I've omitted them for clarity):



And, I fully expect that we will be adding many more workers, so we can look forward to much cutting and pasting.

Well, I wanted to come up with a way of declaring an array, iterating through it and creating each row automagically.

My first thought was this:



But this doesn't work at all. The ActiveRecord attribute can't be built dynamically with string interpolation in the view. Similar efforts failed miserably.

But then I took at look at the ActiveRecord::Base class and noticed the attributes method. This got me a-thinkin' and here's what I came up with. Get the attributes from the ActiveRecord. Build a string with the array value and the "_payment", use straight array access to get the payment value



And, Holy Smakeral -- it works!

Configuring Drupal for McNiff Plumbing & Heating

As mentioned in my last post on Selecting Drupal for McNiff Plumbing & Heating, I wanted to talk about configuring Drupal for a mostly static site.

The requirements are:

• Anonymous Users can view all content
• The site is Page oriented; Home, Services, Comments, Contact Us
• No navigation menu
  
• No login options visible to anonymous users
• No content titles, author information or posting dates on pages
• No user comment links

To start with, I went to Administer >> Site Building >> Blocks and disabled nearly all the blocks. I changed the Navigation block to be visible only to Authenticated Users.

I then created the following pages: Home, Services & Comments and copied some of the content from the current site as a place holder. I also set the Menu Title Links in each of these pages to their base name, so that it could be referenced from a menu.

I used Administer >> Site Building >> Menus >> Primary Links to add a menu item for each page.

Next up was creating the Contact Us form. I enabled the Contact module in the Core - optional section, wen to "Administer By Module" and configured the permissions for the Contact Form and enabled "access site-wide contact form" for all users.

Back to the Menus to add a new Primary Link menu option for Contact by specifying the "contact" page as the target. After saving this new menu item, it was showing up as a new tab with the other menu pages. Great!

Let's check for an anonymous users. I opened up a different browser (I just hate logging out and in and out and in) and checked the development site. No joy in Mudville -- and no tab either.

Somewhere along the line, I was missing permissions. I checked and double checked that the menu was enabled and that anonymous users could access the contact form. I even typed the path into the other browser and the form displayed. Hmmm; not a permission problem because the anonymous user could see the form. The navigation link was simply not being displayed.

Back to Administer >> Site Building >> Contact Form >> Edit. There is a "Selected" option, which was set to "No". Changing that to "Yes" turned on the tab in the Anonymous User's view.

Yea!

Next up; clean up the pages to show only the pieces of information we want. This might require a custom View.

Selecting Drupal for McNiff Plumbing & Heating

I've been helping a local service provide, James McNiff of McNiff Plumbing & Heating improve his website. When we started, his site was a single page, over 60 keywords, and practically no inbound links. Website Grader gave it a 7 out of 100

My first work was to try and touch as little as possible and clean up the HTML so that the site was more search engine friendly. It was actually less time to re-implement the site in clean HTML with CSS, but it was still one page and still was lacking on the SEM side. Website grader raised it to an 8 out of 100.

Version 2 broke the single page into three pages, Home, Services, Comments and to add a rudimentary Contact form. I like working in Ruby on Rails, but his hosting provide charged extra for that application, so I used ROR to generate the web pages, captured them statically and uploaded the static content to his provider. Website grader raised it to a 12 out of 100.

Adding listings to his services and website to some business directories has helped, raising his score to 15.

However, Mr. McNiff didn't really like to content, which had been carried over from the original site and he wanted to be able to add pictures to his site. Given my interest in Drupal, I thought this was a perfect application for trying Drupal.

The hosting provide had a control panel installer for installing Drupal, which set up the MySQL database and installed Drupal 6.14. Here is a list of the modules that I added to begin configuring the site:

• Admin Menu
• Advanced Help
• CCK
• Devel
• FCKEditor
• Google Analytics
• IMCE (for image upload)
• INT Meta
• Webform (although the Contact module looks sufficient)

Please note, that as of this posting, the Drupal site is hidden. As soon as it's ready, I'll swap Drupal to the HTML root and archive the current site.

Next up, a bit about configuring Drupal for a site with essentially static content.

Drupal 6 with GD on OS/X

In my last post on Apache, PHP & Postgres on OS/X, I left off with the step where I was installing Drupal. Heck, that was easy. I downloaded and unzipped Drupal, placed it into my Sites directory, set up a symbolic link and hit it with http://localhost/drupal. I had to adjust some permissions with a chmod -R a+w *, but the installer ran fine, found Postgres and finished nicely.

But Drupal's administrative page reported that I had problems because I was missing GD, which is necessary for image handling. This required a recompile of PHP. The documentation indicated that by adding the --with-gd, that PHP would compile with the built in GD distribution. Unfortunately, I needed other libraries, JPEG, PNG & ZLIB in particular. This is with PHP 5.2.10, please recall. Older versions used TIFF also, so it seems.

I ended up intalling FINK to make life easy. I ran the following commands:

sudo fink install libjpeg
sudo fink install libpng3

(use fink list libpng to find out which version you need to install fink reports that the package can't be found)

I then modified my PHP config command to be:

./configure --prefix=/usr/local/php5 --mandir=/usr/share/man --infodir=/usr/share/info --sysconfdir=/etc --with-zlib --with-zlib-dir=/usr --with-openssl --without-iconv --enable-exif --enable-ftp --enable-mbstring --enable-mbregex --enable-sockets --with-pgsql --with-apxs2=/usr/sbin/apxs --with-jpeg-dir=/sw --with-png-dir=/sw --with-gd --with-zlib-dir=/usr/local

Note that when using the --with-jpeg-dir=DIR commands, the DIR needs to be the directory in which the /lib directory that contains the .a or .so exists. That's why I used --with-jpeg-dir=/sw and not --with-jpeg-dir=/sw/lib.

Next is make and sudo make install. Then recycle the Web Sharing service and Drupal is reporting GD installed. Whoot!

Supressing Object#id deprecation warnings

I picked up some Ruby on Rails code up which I was working a few months ago to complete the project. While adding and running my rspec tests, I started getting the following warning during test execution:

"Object#id will be deprecated; use Object#object_id"

The warning was coming from the super_inplace_controls plug in that I was using. I really like these controls, and wanted to clean up the warning.

So, I popped open vendor/plugins/super_inplace_controls/lib/super_inplace_controls.rb and checked out line 234, from which one of the warning was eminating.

Sure enough, there was, in the method id_string_for, the line:

"#{object_name}_#{method_name}_#{object.id}"

No problems, I'll just change "object.id" to "object.object_id". I did this here and on line 260 in the form_for_inplace_display method.

Of course, this broke the code and the in-place object no longer received its updates after changing a value. The ID part was necessary to retrieve the updated record and display the correct value after update. So I backed out my changes.

Some more searching and I came across a way to suppress warnings around a small block of code, which is exactly what I wanted. I didn't want to suppress all warnings, but I knew that the object.id had to be there (truth be told, I should have tried using just "object"); I just didn't want to hear about it.

To suppress the warning, I wrapped the offending line in Kernel::silence_warnings() {}. So the id_string_for method become:

def id_string_for(object_name, method_name, object)
     Kernel::silence_warnings {
        "#{object_name}_#{method_name}_#{object.id}"
    {
end

I tried the same thing for the form_for_inplace_display method and ended up putting the whole method into the silence_warnings block due to the scoping problems the block introduced.

The danger to this whole method is that, should this code base advance into a new version of Ruby or Rails where the object.id call causes an error, code that seems to work will suddenly break. On the plus side, this won't be a complex logic bomb to solve, but should come up right away in a standard testing cycle.

How to Get Real in Corporate Development, Part VI

This is the sixth part of a series of essays on how to leverage the concepts of Agile software product development from Getting Real from 37signals in a corporate development organization. Part I introduces the concepts of Getting Real, part II looks at the idea of building less, part III is on focusing on one idea, part IV is on getting something functional out quickly and part V talks about focusing on how your users will use the application.

Now it's time to look at:

Focus on fulfilling business stories, not big specification documents.

In my 20+ years of technology experience, I have seen this as being the biggest and most pervasive problem. And I understand that it comes from a very valid perspective from the engineering community.

Engineers can not easily manage huge amounts of change to requirements. Imagine building a bridge if the length of the span changes weekly, the material under the footings shift regularly and the sponsor wants the bridge in steel; no, titanium; no, wood; no, aluminum; no, steel -- you get the point. Therefore, engineers seek to define the requirements and get agreement and sign-off on those documents so they can build something once. Plus, we all know that change gets more expensive as a project progresses, especially if you have a waterfall or spiral approach. All that inflexible foundation work is very expensive to change. (Another reason to stay light and agile!)

Because of the cost of constant change, engineers push to define requirements up front so that they know what to build and where to start. They require definitions, use cases, scenarios, storyboards, and so forth. From this collected material, they define the grand scheme, break it down into delivery phases and get agreement and sign-off. Again, they want to build something once.

The flaw with this approach is that the requirements cannot be entirely known in advance. Therefore, just as soon as the project starts, the huge requirements specification is out of date. I once worked on a project with over one thousand distinct requirements. Because we were inventing new science and technology while producing a commercial product, the requirements churn was huge. We touched every single requirement dozens of times; the requirements churn topped 1000%!! That's like writing over 10,000 requirements! It was nuts!

Here is the fix: Focus on business stories. Start with the most important story. It works like this.

The business problem is, "We have to generate an inventory spreadsheet daily by running a report in our inventory management software (IMS), copying the date and transforming it into Excel, then cleaning up the data by re-arranging the columns and saving it as a semi-colon delimited file so it can be emailed to the web services group so they can update the inventory counts on the website. Errors and delays in the process can mean displaying products as in-stock when they are currently back-ordered, which annoys customers."

The business story becomes, "We want to automate the daily inventory reporting from our IMS to our web server to increase accuracy, decrease information turn around time, and free business resources for other work."

I can immediately see a number of ways to do this, from using whatever automation exists within the IMS to generate and send that report, or accessing the IMS database directly with a custom report, or using the IMS database programmatic API to write an automated application to pull the data and update the web database, or to make the web database read its data from the IMS database live, or on a daily batch bases, or exposing a web service so the data can be pushed into the web database. I bet you have a few more options.

The take away is that solving the technical problem isn't usually the problem. The problem is solving the correct business problem. Getting bogged down with a huge requirements document doesn't make solving today's problem go any faster and doesn't produce a solution to today's problem any faster.

So, how would I apply this advice to that huge 1000% churn project? Well, the biggest constraint was that we had to produce a commercially shippable product. A goal was to create a single architecture that could be leveraged along the future product line. However, since we were creating new technology and a new command and control system, we set the bar too high.

The immediate business story was to operate the machine and provide a sufficiently flexible control system that would allow for the scientific invention to continue without requiring huge software re-writes. Today, I would drop the goal that we create an architecture that could handle future demands. The project was already very challenging without that goal and we ended up pulling out all the infrastructure that was built to handle that goal anyway, just to get the first product to ship.

We made the classic mistake of solving tomorrow's problem before solving today's problem. You can avoid that mistake by focusing on the business stories that you have today and not creating that huge, over-arching requirements document that will doubtlessly saddle you to a very sizable ball and chain.

How to Get Real in Corporate Development, Part V

This is the fifth part of a series of essays on how to leverage the concepts of Agile software product development from Getting Real from 37signals in a corporate development organization. Part I introduces the concepts of Getting Real, part II looks at the idea of building less, part III was on focusing on one idea and part IV was on getting something functional out quickly.

In this essay we look at:

Start with the UI design and focus on how your users will use your application.

Lots of experienced engineers like to start with the UI. A new project brings a geyser of ideas and concepts that make the fingers itch to start writing code. So, mockups, prototypes and proofs of concepts are often the first things to come out.

Good! Almost.

Don't let your engineers close themselves in a room for the four weeks that they have and create something they love. The rest of the sentence is the key -- "focus on ... your users ..."

In "Getting Real", the folks at 37signals assume they are writing an appliation for the outside to use. They don't have an opportunity to sit with their most important users when creating something new. But you do! This is where the corporate environment make Getting Real better than the original. You're making software for your business, you and your users are the domain experts. You don't need to go outside. Lucky you!

Get your engineering team (2 really good engineers and a kick-ass business analyst) in the same room as the key stakeholders -- the folks who will use this new system. Understand the business problem (Focus on One Idea) by talking to the users and examining the problem they need solved.

Have the team work through what they need to do. What tasks are grouped to gether logically? What actions are done most of the time? What actions are done only a small fraction of the time? Make the application fix the problem elegantly. That way, the team has to Build Less and will be able to Get Something Functional Deployed Quickly.

Start light. Whiteboard and sketches. Wireframes and light-HTML. Test the process. Can the users do their important tasks quickly and easily? Can they see how they get to the additional features that they don't use every day?

Put the most important information right in the middle. Make sure that new state, the populated state and the error state are considered. Ensure the engineers make the application behave appropriately when it encounters something unexpected. Leave out what doesn't belong to do the task (who cares if it's a "standard" menu item?). Put in everything important. Leave out everything else.

Match and support what the users need to do, then go and make it happen. Start with the UI. Not the UI and the database, just the UI. Not the UI and the object model, just the UI.

The engineers will have a clearer idea of what's important. They will have proven the work flow that they are about to implement. The users will have increased confidence that what will be produced will really help them.

Can you just imagine the response when the first version comes out a few weeks later? Hey, please write and let me know! I don't want to do this in a vacuum!

Thanks again for reading! In the next essay, we'll talk about how to use business stories.

How to Get Real in Corporate Development, Part IV

This is the fourth part of a series of essays on how to leverage the concepts of Agile software product development from Getting Real from 37signals in a corporate development organization. Part I introduces the concepts of Getting Real, part II looks at the idea of building less and part III was on focusing on one idea.

Next, we talk about structuring the project to quickly produce working code. So here it is:

Get something functional and usable deployed quickly. Plan on your next release right away. Make a choice and go with it.

Constraints are what drive a project. If you had unlimited time and money, you would probably never produce anything, as ironic as that may sound. You need limits to focus you on getting something done. My triathlon training buddy and I like to say that we "train to race and race to train" because without the race date, it's too easy to sleep in instead of hitting the pool at 5:00 AM. Constraints drive us.

So, pick one of your projects and give it to a small group of good engineers. Tell them they have four weeks to produce the first version that the business can use, but then they'll have four more weeks to produce the next release. And close the door.

They will focus on building less and implementing one idea. It's all that they'll have time for. Meanwhile, let the business know that the first iteration will be ready for use in four weeks and that the engineers will need their feedback to produce the second iteration.

Your job is to get out of the way and make sure the engineers have the tools, access and permissions they need to do their work. Herein lies additional opportunity.

Perhaps there are procedure and process documents in your organization. Perhaps there is a PMO and a review committee. Perhaps you have strict regression testing requirements for deployments. Perhaps, under normal circumstances it would take your small, Agile group of engineers four weeks just to get permission to connect to the database they need to fulfill this project.

Your opportunity is to fix this. Yes, security and access control is critically important. But, we're in the 21st Century, aren't we? Network directories with access control list are mature technologies and are probably in your organization. Are your Oracle databases set up to take advantage of your LDAP infrastructure? They should be. And no one better exists than you to make it happen.

Your organization should have four environments:

• Development -- open and flexible for the developers to do anything. Developers should work in both Debug and Non-Debug environments. Don't accept "Well, it worked on my machine."
• Test -- mirrors production, but allows developers to see everything in the system.
• Staging -- mirrors production as closely as possible
• Production -- full security and access controls.

Using virtualization, farms, and similar techniques can keep this from being a horrible amount of overhead. The key is that you need to unlock your most productive engineers. Now is a good time to note that not all of your engineers are your most productive ones. Make sure your less productive engineers are fully supporting your most productive ones. There are opportunities for training and mentoring, which build employee satisfaction and loyalty, but let's get back to getting functional software out.

How many times have you already read that a skilled and senior engineer is many times -- 10 times, 15 times, 20 times -- more productive than a junior engineer? So, we don't need to say it again. Nope. Not going to say it. Na-uh.

So then, just how will this work? Well, let's start off by looking at the typical process. The business has an urgent need. Something like, "they spend a huge amount of time running end of month reports and small errors in the data cause regular heroics."

In the typical process they fill out an Application Enhancement Request or New Application Request or some such like that. This gets reviewed at the next Application Review Committee meeting and goes back to the requester for more details. Two weeks go by for the out and back trip and the ARC approves the request. The next week a project manager and business analyst are assigned to it. They take two weeks to gather requirements, another week to create a Statement of Work and one more to have that approved by the stakeholders. The following week, a project team is assembled while the analyst writes the Functional Specification. The stakeholders don't even bother reading that.

We're up to, um, nine weeks. The team takes two months to write the application, three weeks to test it, one week to slot it into deployment and, lo and behold! More than five (5!!) months later, the application is first released. The users try it and file a new Application Enhancement Request to get it to do all the things that they need and eliminate the stupid things that crept into the application. Time to happy users: 9 months.

Here's how we want it to run:

First, you've addressed the infrastructure, you have the governance in place to enable rapid development and you've now paired your best engineers with the business users we just talked about. They meet for one day and use up another afternoon or two. In four weeks the first release is out and in use. In four more weeks the next version comes out and fixes the big problems with the first release and adds a few more features. One more four week period and the business has something that really works for them. Time to happy users: 3 months.

Even if you give this team another three iterations -- that would be a total of six (6!) releases, it would still be 66% of the duration of the Same Old Way, which would have produced 2 releases. Which application do you think would be better? The one that had six tight, focused releases, or the other one? There is no doubt in my mind.

You need to have really great developers to do this. But then again, nothing outstanding was built by mediocre engineers. You need an infrastructure that enables these really great engineers. You still need to have proper governance, SOPs, checks and balances to put software into production, but you can get the work done much, much faster by Getting Real. And this will generate real value for your company.

Thanks again for reading! In the next essay, we'll look at starting with the UI.

How to Get Real in Corporate Development, Part III

This is the third part of an essay on how the concepts of Getting Real from 37signals can benefit development groups in corporate environments. Part I introduces the concepts of Getting Real and part II looks at the idea of building less. To build less, you need to select one problem to solve and focus just on that. Part I introduced this, so let's continue to look at that aspect.

Focus on one idea. Start with the big problem, build the large scale functionality and don’t sweat the details.

One challenge every development effort has is where to stop. That is, where to define the edge of the application. Otherwise stated as, "how far out from the core idea is it reasonable to extend?" For example, should the document handling system not only import PDF's, but generate PDF's and fax them?

One helpful concept is the constraint of time and budget. However, it is not uncommon in corporate development shops to have projects loose the continuity of effort that makes a strict delivery time or budget tracking approach overly challenging. Here is where having one focus pays of immensely.

The thinking is: We are going to solve only the problem at hand (Today's Problem). Building less gets us a solution more quickly. Focusing on the problem helps us meet the goal of solving only the problem at hand.

Engineers with are much easier to keep focused and on-task when they have a delivery due in only a few weeks. Assigning a single, short-term purpose accomplishes this. Part of the standard challenge in corporate development shops is that the an engineer wears many hats as she maintains existing applications, develops one or more concurrent projects, and concurrently designs and plans future applications.

Switching between different types of tasks is a known productivity killer. The well-known blog by Joel on Software lays out the trouble with human task switching. You know it yourself, having your concentrated thought process interrupted is, literally, disruptive. For your team, this means that they need quite, private work spaces, few meetings during sprints, and very long periods of uninterrupted time. It also means you need to keep their plates clear of other tasks.

When you do this, your engineers will be able to stay focused on delivering that one idea.

"Hmmm, good," you say, "but what is the correct one idea?" Good question. After all, "solve global poverty" is one idea, but it's obviously not at the right level.

You're looking for an idea that represents a specific business pain point with a solution that can be produced by a small team in a few quick iterations. Since your team of three engineers can't bang out a solution to global poverty in a few three-week sprints, it's not at the right level. (But you already knew that -- this was an illustration :-)

Let's try some on for size. "Create a portal that allows secure transmission of documents from customers to service representatives." Or how about, "Replace a key paper form with an internal website." Here's what each of these have going for them:

• Each of these have a good change of being produced rapidly in only a few iterations.
• They both address only one problem.
• They require one small application to successfully meet the business need
• They both are probably very well understood by stakeholders and engineers.

Given all this, there is very likely little risk of implementing the wrong software. Even if the end product doesn't work out completely, you have only invested a small amount of time and effort and you're team has more clearly illustrated the business need.

Are you thinking that you don't have any small, simple problems to solve? Are you thinking that you need complex applications to solve complex business problems? Are you thinking that there isn't any opportunity to do something small and elegant like this in your shop?

If you have any of these thoughts, or thoughts similar to them, please try an experiment. Find a key stakeholder at the line level. Perhaps a manager of customer service, or a project manager who executes key services repeatedly. Take them to lunch and ask them about the most challenging aspects of their work and the work their team or group does.

You will probably learn some things you didn't know. And some of what you learn could very well fall right into this camp of creating high-value for the business with a light, flexible, low-risk and low-cost development approach.

The next part is to work with wide brushes and big strokes. You're not looking for a whiz-bang gizmo that does everything. Get the big pieces right first. Now that you know about some business pain points (if you already knew some or knew the ones you talked about over that lunch, then kudos for you!), you're looking to address just that need. Nothing else right now, just that one need.

Yes, the fund accounting deals with mutual funds, 401(k)'s of mutual funds, annuities, closed-end funds, etc. But the bulk of the work is mutual funds. Or the bulk of the time is 401(k)'s. Or the bulk of the revenue is generated by something else. It's up to you to figure out, with your key stakeholders, what the most valuable problem to solve will be, then it's up to you to focus on that one factor.

The mandate you give your team is to fix just that one piece, make it better for the business and don't worry about the details. You will re-work and re-engineer as you dig down into the next set of priorities on upcoming iterations. But you will be providing a high-value solution in very short order, and that is worth more to the business in the long run.

Thanks for staying with me as we continue exploring the approach of Getting Real in your corporate development shop.

How to Get Real in Corporate Development, Part II

This is the second part of an essay on how the concepts of Getting Real from 37signals can benefit development groups in corporate environments. Exploring this mash-up is worthwhile because at first glance, there seems to be very little in common between the web-application, small-team, product bootstrap philosophy in Getting Real and the realities of corporate development teams. These realities include governance, oversight, processes, controlled systems and similar best practices. But pull the covers back a bit and enough similarities appear to make for a compelling argument. Let's dive right in and take a closer look at the key concepts from the first essay.

Build less. Build smaller applications with only the most important features and delivery it quickly.

This is sage advice because the corporate development life cycle is typically beset with serious birthing pains. Life cycles are heavy with process, stages, gate reviews and sign-offs. The corporate process inevitably includes requirements gathering and documentation, which involve meetings, drawings, documents and reviews. The requirements themselves transform from stakeholder statements to business requirements to technical requirements to design documents. After all that, there is finally the build, test, deploy, train, support and maintain functions. All to get one release to the business users.

Some problems with this model is that it takes too darn long and is too darn expensive. Half a year can pass filled with stakeholder interviews, document drafts, reviews, modifications and so forth that will ultimately produce a few dozen pages of specifications and diagrams. One team of stakeholders, business analysis and engineers can easily spend multiple person months or even a person year (or years) producing 60 pages. If you have a blended team salary rate of $90K, then each page costs $1,500. And you haven't built anything yet!

"No" you might be answering resolutely right now, "but we know know what to build! And we have a plan to build it."

True. I've worked this way too and I know that it helps to reduce the risk of development in the long run. You can plan for a three-phase development effort, get the budget lined up and all that. However, please consider an alternative.

Your team knew -- on day one -- what was the most important functionality needed. You, and they, don't need 6o expensive pages for that. Here's how it would work for you when you Get Real.

Two good engineers and a business analyst -- your development team -- meet with the key stakeholders. The stakeholders are asked to identify their top problems within the problem space. Then they have to choose only one. The top problem. Your development team ask enough questions to fully understand that problem and how it relates to the other top problems.

Then, they go away and produce something small to fix that top problem and only that top problem. This takes three weeks. After all, they are focused on only one key important problem.

Next, the key stakeholders begin to use the new application. After one week, the team reconvenes and the development team learns what works, what doesn't and what the next most important problem to solve is.

At this point, you've spent a tiny fraction of the expense and effort of that functional specification we taked about earlier and you have an application that, hopefully, solves an immediate business problem. If it doesn't, it has shone a big, bright light on that problem. How much longer do you think it will really take your development team to get it right? One more iteration? Two? Fine; because at that point three months will have gone by and you will have a demonstrable ROI. Compare that to a functional spec that takes six to nine months to write. You are seriously ahead of the curve.

OK, another valid concern is that when your development team learns about the next problem they will have to throw away a lot of what they've done. And they might. This is why we focus on solving one problem within the context of the larger scope. However, it is critical to solve only one problem at a time. Fix the problem you have today, not the problem you anticipate having tomorrow.

The way to do this is to build only what you need to fix that problem. Build less. What is the smallest, most elegant solution to the business problem?

Keep in mind that the business problem might not be the problem the users report. They may be unhappy with emailing spreadsheets around and want the population, generation and transmission automated. However, what they really need is a way to exchange and store information centrally instead of relying on spreadsheets.

Thanks for reading. We will continue to peel back this onion in the next essay on Getting Real in your corporate development shop.

Using Getting Real Concepts in the Corporate Office

The talented and intelligent folks at 37signals have created some great applications. As major contributors to Ruby on Rails, they also have been fundamental in building and leading a revolution in rapidly developed, high quality web applications. 37signals is successful because they think creatively about the problem that they are trying to solve. Fortunately for all of us, part of that thinking is that they should be open and transparent with many aspects of their process and business. Therefore, they often preach about the practices that have helped them attain success.

Anyone can read their preachings online. For free! It's a great concept, and one that is often encouraged by industry leaders -- think Seth Godin for one.

You can also purchase a high-quality bound copy of their book “Getting Real” from Lulu.com. I have because I like being able to read when not in front of the computer. It’s all laid out in their book (the exact same copy in the web pages, downloadable PDF or book) – their philosophical approach to clients, development, hiring, customer service, and so on.

When I became aware of 37signals , I was developing Ruby on Rails web applications. Because I was creating web apps, which is clearly the focus of the book and philosophy, I found myself agreeing with just about everything they were saying. And hey, I’m a software development veteran.

I’ve done this for years and years and have focused extensively on managing complex projects to successful deliveries. I’ve been through waterfall, spiral, iterative, agile, ad hoc, chaotic, random and panic-driven development models. I agreed with what they were saying because it makes sense and I was able to make it work successfully on my projects without any pain or difficult transitions. There aren’t many “new” development philosophies about which that can be said! For light, flexibile web apps, “Getting Real” made total sense and, more importantly, it worked.

But what about corporate environments with ongoing development efforts? These shops have a different set of starting points and different base assumptions. What if you aren’t developing a web app? What if you aren’t focused on revenue generation with your app? What if your users are waiting for their application and have expectations about how it will help them?

Can your corporate development efforts also Get Real? Yes, I believe that they can. let's look at the components of Getting Real that will help your corporate development teams use the principles from Getting Real to improve productivity and stakeholder satisfaction levels.

Let’s start with some of the key concepts of Getting Real that are directly beneficial for all development efforts. (Links are to the essays on those concepts)

1. Build less. Build smaller applications with only the most important features and deliver it quickly.
2. Focus on one idea. Start with the big problem, build the large scale functionality and don’t sweat the details.
3. Get something functional and useable deployed quickly. Plan on your next release right away. Make a choice and go with it.
4. Start with the UI design and focus on how your users will use your application.
5. Focus on fulfilling business stories, not big specification documents.
6. Keep your developers on the front line of support
7. Be willing to radically change the application to make it better. Resist the urge to make one monolithic tool to solve everyone’s problems at once.

Notice that these nuggets are completely environment, tool, and technology agnostic. There are some radical ideas that will doubtlessly shake up the “way things are done”, but these are the right ideas to grab onto and the right changes to make.

Fortunately, there are some components of Getting Real that an internal corporate development group can safely ignore right off the bat. For example, you’re probably not looking to generate revenue with something like an internal workflow enhancement tool. Not that you can ignore development costs or the value proposition of the development effort, but at least your users won’t be facing the prospect of paying a monthly fee. (Although, there may be something here….)

You can safely ignore the need to market and promote your application extensively. Please note the qualifier. The new whiz-bang tool you’ve developed for Operations is straight sunk costs if end users aren’t using it. Therefore, they do need to know about it and they should be excited about its arrival and feature set. But that should come from good communications and interactive product development and not from promotional marketing campaigns. Trumpeting successes is always allowed, however, so some allowance is made, but we’re not talking at all about the same degree of importance of marketing discussed in the book.

Most importantly, as you – the development lead, manager, director, VP, CTO, Whatever – read Getting Real, you can insert your favorite technology du jour whenever the book talks about web applications. The thrust of web apps in Getting Real is their rapid deployment and instant accessibility to customers. No CD burning, no distribution efforts, no retail operations, etc.

But let’s face it, how long does it really take to deploy a new version of an internally developed application in your shop whether it was written in C++, Java, Lotus Domino, C#, .NET or Ruby on Rails? If you’re thinking “weeks”, then you’ve got another problem, don’t you? You can quickly push out new applications through automated corporate update mechanisms, internal web sites and internal installers with the exact same degree of responsiveness that is the base assumption in the book So don’t sweat it that you’re not making a web app. The underlying philosophies in “Getting Real” remain pertinent, applicable and valid to you and can revolutionize your development centers.

The next essay will look at those seven take-away concepts in greater detail and will focus on how you can leverage these concepts to make high quality software faster.