Friday, November 13, 2009
Event callbacks in AASM
However, when I tried adding a callback to the aasm_event, I ran into trouble. Quite simply, the callback never worked. Here's the first code:
Notice that the callback registration is in the aasm_event code block. It seemed reasonable that this would be part of the transition. When I determined that this was not working, I had a colleague review the code and we ended up both scratching our heads.
So, I cracked open the library and poked around. I saw the successful aasm_state call displaying the callback registration in the options portion of the method, but the options of the aasm_event were blank. Hmm!
Here are the method definitions for your edification:
aasm_state(name, options={})
aasm_event(name, options = {}, &block)
Well, it was pretty clear how to fix the problem; move the callback registration after the name argument and before the block. Here's the corrected code:
That's it! Have fun.
Friday, October 30, 2009
Fast Online Documentation - GotAPI.com
I use PDF versions of my favorite books and the generated Ruby 2.3.4 API documentation locally, which is all good, but I found something that may be even better. GotAPI has a clean interface, is wicked fast and utterly intuitive. It even holds your recent searches.
In addition to lightning fast Ruby and Ruby on Rails look ups, it has tabs for HTML, Javascript DOM and CSS; all the things I need during RoR development!
I've added it to my browser bar so I can get to it quickly and it's rapidly becoming my reference tool of choice. Highly recommended.
Saturday, October 10, 2009
Conditional state transitions with AASM
The task reviewer state was optional and would be triggered only if selected. I represented that state as a "use_reviewer" boolean column in the object. If use_reviewer == true, then the object transitions from task_created to "to_reviewer", otherwise it needed to transition to "to_writer"
The solution to this is rather straight forward. Use the :guard option in the aasm_event definition.
Here's the code:
aasm_event :created do
transitions :to => :to_reviewer, :from => [:task_created], :guard => Proc.new {|p| p.use_reviewer == true }
transitions :to => :to_writer, :from => [:task_created]
end
The event causes the object to transition to the "to_reviewer" state if the objects "use_reviewer" is true, otherwise it transitions to "to_writer". Here's the code evidence from script/console:
>> a = Assignment.new
=> (Assignment object)
>> a.use_reviewer = true
=> true
>> a.save
=> true
>> a.created!
=> true
>> a.aasm_current_state
=> :to_reviewer
Now we start over:
>> a = Assignment.new
=>(Assignment object)
>> a.save
=> true
>> a.created!
=> true
>> a.aasm_current_state
=> :to_writer
Chaining state transitions with Acts As State Machine (aasm)
In the "to_reviewer" state, a group of reviewers are presented the assignment and they may accept the assignment, or a project manager may assign the assignment to a reviewer: in either case, as soon as there is a reviewer assigned, the assignment transitions to the "reviewer" state.
However, if the system already has a reviewer assigned at the "created" event, then it should cascade the state transition change and go directly to "reviewer" without staying in "to_reviewer"
Having selected the Rubyist AASM gem to implement my state machine in this Ruby on Rails project, I expected that I would be able to use the :after options of the aasm_state statement. I got that idea from the README.rdoc for version 2.1.1, which lists "event:after" in the callback chain.
However, the :after callback is called on an event (aasm_event) and I can't find documentation on how to hook into that callback. Digging into the the code, I found the following syntax works correctly in my Assignment object (derived from ActiveRecord). The solution is to use the :after_event option in the aasm_state definition statement.
This first part defines the states:
aasm_initial_state :task_created
aasm_state :task_created
aasm_state :to_reviewer, :after_enter => :check_reviewer
aasm_state :reviewer
This next part defines the events:
aasm_event :created do
transitions :to => :to_reviewer, :from => [:task_created]
end
aasm_event :task_review do
transitions :to => :reviewer, :from => [:to_reviewer]
end
This last part is the definition of the :after_enter method:
def check_task_reviewer
task_review unless reviewer_id.blank?
end
Now, when the object is created, the aasm_current_state is "task_created". If the object receives the "created" event, it transitions to the "to_reviewer" state. However, after the object enters the "to_reviewer" state, the "check_task_reviewer" method advances the state again if a reviewer as been assigned (indicated by the presence of the reviewer_id).
Checking the code in the console, we get:
>> a = Assignment.new
=> (Assignment object information)
>> a.reviewer_id = 5
=> 5
>> a.save
=> true
>> a.created!
=> true
>> a.aasm_current_state
=> :reviewer
And we can see that the object transitioned through the "to_reviewer" state an on into the "reviewer"state.
Thursday, October 08, 2009
The Ruby Toolbox
This delightful site is filled with collections of helpful, nifty plugins and gems that help make the life of any Ruby and/or Rails developer much more pleasant. There are some 75+ different categories that deal with ActiveRecord, Backups, Code Metrics, CSS Frameworks, E-Commerce, Game Libraries, Gem Creation, and on, and on.
Not only are projects, gems & plugins listed, but they are rated in terms of how much attention the project is receiving. The most popular projects are listed at the top with an attractive scale along side to the right to help you compare the popularity of different options.
If you're doing Ruby and/or Rails development and haven't visited this site, then you really deserve to do yourself a favor. I'm headed there now!
Thursday, October 01, 2009
Rails Trick: Access an ActiveRecord field in a loop in a view
OK, picture this: We have over half a dozen different types of workers -- Writers, Sr. Editors, Editors, etc. -- and we're going to pay them. So, in the database we have things like writer_payment, sr_editor_payment and editor_payment.
In the edit view, we have something like this (actually, there are many more cells, but I've omitted them for clarity):
And, I fully expect that we will be adding many more workers, so we can look forward to much cutting and pasting.
Well, I wanted to come up with a way of declaring an array, iterating through it and creating each row automagically.
My first thought was this:
But this doesn't work at all. The ActiveRecord attribute can't be built dynamically with string interpolation in the view. Similar efforts failed miserably.
But then I took at look at the ActiveRecord::Base class and noticed the attributes method. This got me a-thinkin' and here's what I came up with. Get the attributes from the ActiveRecord. Build a string with the array value and the "_payment", use straight array access to get the payment value
And, Holy Smakeral -- it works!
Sunday, September 27, 2009
Great Website for Printing Patents
- US Utility Patent
- US Pre-Grant Publication
- US Design Patent
- US Plant Patent
- US Reissue
- US Statutory Invention Reg.
The site looks up your item, pulls it from the US Patent Office's website and assembles it into a single PDF for download. Sweet.
What I Learned About Garage Door Openers
After opening the cover the problem was obvious. The nylon gear which drove the shaft that drove the sprocket (which is a gear that is driven by or drives a chain), was split in half. I looked up the manufacturer on the 'Net, then the model number and ordered a replacement kit. I was so relieved that it came with a detailed instruction book!
Here's what I learned about this garage door opener:
Limit Switches
The limit switches are very simple. As the drive shaft turns, it drives a screw upon which is mounted a moving contact. As the screw turns, it moves the contact to one end or the other. One screw on each side of the moving contact has the other end of the contact. As soon as the moving contact meets the end contacts -- the door stops. Simple!
Rate Sensor
There is a rate sensor that measures the rate at which the motor is spinning. This was also simple. A cap on the motor shaft had a series of four slots and openings, which passed through a sensor with a tiny emitter and sensor. If the door gets blocked, the motor can't turn. If the motor can't turn and the unit has not hit one of the limit switches, and the door is moving down, the controller reverses direction. If the door is moving up, it stops. Simple!
Retaining Pins
There isn't any other way to get a retaining pin out other than with a hammer. The sprocket drive shaft holds the drive gear in place with two retaining pins. These fit tightly into place. I had to drive the lower pin out in order to slide the new drive gear into place and the only way to do that was with a hammer. As soon as the pin was flush with the shaft, I used a punch to continue driving the pin through the shaft. After the new drive gear was in place, it was hammering once again to get the pin back into position.
Disassembly & Reassembly
Check the orientation of things as you disassemble them. I made two good choice and missed one opportunity. Before unplugging drive motor power connectors, I confirmed that the diagram in the instruction booklet was accurate and took a picture of the setup with my phone. When re-assembling the unit, I had all the documentation that I needed. However, I didn't notice the tabs in the top plate that holds the drive shaft in position and ended up re-installing that plate rotated 1/3 around. I didn't notice the mistake until nearly the very last step of re-installing the protective plastic cap over the sprocket and drive chain. It didn't make sense to disassemble the entire unit at that point, so we're living with my error. Had I taken a picture of that unit before disassembling it, I would have had the necessary reference.
Clean Grease
Clean grease isn't so bad. Dirty grease is wicked icky, but the new clean grease was easy to put on and I didn't mind cleaning up afterwards.
End of the story: the garage door works perfectly now. And I had a lot of fun taking the durn thing apart and putting it together!
Configuring Drupal for McNiff Plumbing & Heating
The requirements are:
- Anonymous Users can view all content
- The site is Page oriented; Home, Services, Comments, Contact Us
- No navigation menu
- No login options visible to anonymous users
- No content titles, author information or posting dates on pages
- No user comment links
I then created the following pages: Home, Services & Comments and copied some of the content from the current site as a place holder. I also set the Menu Title Links in each of these pages to their base name, so that it could be referenced from a menu.
I used Administer >> Site Building >> Menus >> Primary Links to add a menu item for each page.
Next up was creating the Contact Us form. I enabled the Contact module in the Core - optional section, wen to "Administer By Module" and configured the permissions for the Contact Form and enabled "access site-wide contact form" for all users.
Back to the Menus to add a new Primary Link menu option for Contact by specifying the "contact" page as the target. After saving this new menu item, it was showing up as a new tab with the other menu pages. Great!
Let's check for an anonymous users. I opened up a different browser (I just hate logging out and in and out and in) and checked the development site. No joy in Mudville -- and no tab either.
Somewhere along the line, I was missing permissions. I checked and double checked that the menu was enabled and that anonymous users could access the contact form. I even typed the path into the other browser and the form displayed. Hmmm; not a permission problem because the anonymous user could see the form. The navigation link was simply not being displayed.
Back to Administer >> Site Building >> Contact Form >> Edit. There is a "Selected" option, which was set to "No". Changing that to "Yes" turned on the tab in the Anonymous User's view.
Yea!
Next up; clean up the pages to show only the pieces of information we want. This might require a custom View.
Selecting Drupal for McNiff Plumbing & Heating
My first work was to try and touch as little as possible and clean up the HTML so that the site was more search engine friendly. It was actually less time to re-implement the site in clean HTML with CSS, but it was still one page and still was lacking on the SEM side. Website grader raised it to an 8 out of 100.
Version 2 broke the single page into three pages, Home, Services, Comments and to add a rudimentary Contact form. I like working in Ruby on Rails, but his hosting provide charged extra for that application, so I used ROR to generate the web pages, captured them statically and uploaded the static content to his provider. Website grader raised it to a 12 out of 100.
Adding listings to his services and website to some business directories has helped, raising his score to 15.
However, Mr. McNiff didn't really like to content, which had been carried over from the original site and he wanted to be able to add pictures to his site. Given my interest in Drupal, I thought this was a perfect application for trying Drupal.
The hosting provide had a control panel installer for installing Drupal, which set up the MySQL database and installed Drupal 6.14. Here is a list of the modules that I added to begin configuring the site:
- Admin Menu
- Advanced Help
- CCK
- Devel
- FCKEditor
- Google Analytics
- IMCE (for image upload)
- INT Meta
- Webform (although the Contact module looks sufficient)
Next up, a bit about configuring Drupal for a site with essentially static content.
Friday, September 18, 2009
Drupal 6 with GD on OS/X
But Drupal's administrative page reported that I had problems because I was missing GD, which is necessary for image handling. This required a recompile of PHP. The documentation indicated that by adding the --with-gd, that PHP would compile with the built in GD distribution. Unfortunately, I needed other libraries, JPEG, PNG & ZLIB in particular. This is with PHP 5.2.10, please recall. Older versions used TIFF also, so it seems.
I ended up intalling FINK to make life easy. I ran the following commands:
sudo fink install libjpeg
sudo fink install libpng3
(use fink list libpng to find out which version you need to install fink reports that the package can't be found)
I then modified my PHP config command to be:
./configure --prefix=/usr/local/php5 --mandir=/usr/share/man --infodir=/usr/share/info --sysconfdir=/etc --with-zlib --with-zlib-dir=/usr --with-openssl --without-iconv --enable-exif --enable-ftp --enable-mbstring --enable-mbregex --enable-sockets --with-pgsql --with-apxs2=/usr/sbin/apxs --with-jpeg-dir=/sw --with-png-dir=/sw --with-gd --with-zlib-dir=/usr/local
Note that when using the --with-jpeg-dir=DIR commands, the DIR needs to be the directory in which the /lib directory that contains the .a or .so exists. That's why I used --with-jpeg-dir=/sw and not --with-jpeg-dir=/sw/lib.
Next is make and sudo make install. Then recycle the Web Sharing service and Drupal is reporting GD installed. Whoot!
Thursday, September 17, 2009
Apache, PHP & Postgres on OS/X 10.5 (Leopard)
I started looking at the MAMP install, but I don't want MySQL as part of the stack, so I didn't think it would work. However, I found Lullabot's post on Beginning with Drupal 6 and PostgreSQL on OS X 10.5 Leopard and John VanDyk claimed that the MAMP + plain vanilla install gave him the option of choosing which database to use. That didn't work for me -- the Drupal install gave me no such option.
So I tried testing the apache - php - postgres stack to make sure I could hit my long- working postgres database. Running a little php test showed that I wasn't able to connect to my postgres database and the documentation from the PHP site indicates that I needed to recompile php with the postgres option. So I got the latest PHP -- 2.5.10.
However downloading, configuring, makeing & installing, httpd was still using the original PHP 2.5.8 and not the new PHP 2.5.10. So said the phpinfo script after recycling httpd. Httpd uses the php5 shared object library in /usr/libexec/apache2. The libphp5.so had not been updated. Why? Is the problem with Apache?
DIYMacServer.com has an excellent series of articles on installing Apache on OS/X. I'm installing Apache in 32 bits mode on Leopard and chose to use the Darwin layout. So it was configure with the Darwin layout and all the options, make and sudo make install.
The next step was to edit /etc/apache2/httpd.conf and enable index.php, the user under which Apache will run to "_www", and allow the use of the .htaccess file. Next was enabling the use of the users directory in /etc/apache2 (I copied the original users directory into the new /etc/apache2 directory). To do this, I uncommented the Include /etc/apache2/extra/httpd-userdir.conf line in httpd.conf. I chose to not enable Virtual Hosts. I also changed the DocumentRoot to my Sites directory. Because this is a development environment, I also commented out the Deny from all directive in the
With all that done, I fired up the "Web Sharing" from the control panel and hit http://localhost (which is how I configured the ServerName in my users .conf file). The default site came up, so I tried the phpinfo.php script. Nada. Not processing php. Well, there is no LoadModule directive for php5_module in the httpd.conf, so I added it. However, the reference libphp5.so was still from the older PHP installation, the one that was not compiled with support for Postgres.
Here's what worked:
- Moved old /etc/apache2 out of the way
- Made sure all httpd processes were shut down
- Deleted old /usr/libexec/apache2/libphp5.so
- Configured httpd-2.2.13 with the following. The Darwin layout preserved the ability to use the System Preferences to control httpd.
- Did the make and sudo make install.
- Edited the new httpd.conf with the changes described above & copied the user directory from the original apache2 directory to the new directory
- Configured PHP with the following. Note the "--with-pgsql" and "--with-apxs2" commands (From the DIYMacServer site)
- Finished with the make & sudo make install. This time I saw the libphp5.so being copied to the correct directory -- yea!
- Fired up the web service from the System Preference panel and checked my phpinfo file. SUCCESS! We are now looking at PHP 5.2.10.
- Wrote a small postgres test (below), created the table and added two names to it. SUCCESS! Whoot! We are in.
Sunday, September 06, 2009
Installing Spree eCommerce
So, I'm looking for an eCommerce engine for Rails. Yes, I could create the whole shopping cart and payment gateway on my own because there are tons of tutorials and general advice. However, I suspect that if I could find a full-featured engine, then someone else will have thought of -- and solved -- problems that will take me time to recognize and address.
So I came across Spree eCommerce.
However, I haven't updated Rails on my system in a while and Spree requires Rails 2.3.2. So I start by doing a
sudo gem update rails
which gets me to Rails 2.3.4. However, Spree wants 2.3.2 ONLY. So I remove Rails 2.3.4 and install 2.3.2. Then I install Spree according to the instructions. Then I try to create my first Spree application with:
spree mystore
Unfortunately, I got this error:/usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:578:in `report_activate_error': RubyGem version error: hoe(1.5.3 not >= 1.8.0) (Gem::LoadError)
OK. No problem. Need to update hoe~/src:sudo gem update hoe
Updating installed gems
Updating hoe
ERROR: While executing gem ... (Gem::InstallError)
hoe requires RubyGems version >= 1.3.1
OK. No problem. Need to update gem. After all,
~/src:gem --version
1.2.0
So I do the following:~/src:sudo gem update --system
Updating RubyGems
Nothing to update
OK, this may be a problem. Referring to the rubygems documentation, one needs to do the following:gem install rubygems-update
update_rubygems
However, this failed with:~/src:sudo update_rubygems
/usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:578:in `report_activate_error': Could not find RubyGem session (>= 0) (Gem::LoadError)
from /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:134:in `activate'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:158:in `activate'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `each'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `activate'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:49:in `gem'
from /usr/local/bin/update_rubygems:18
Mark Howe has a helpful blog on the matter.In the end, the trick was to uninstall the partly installed rubygems 1.3.5, install rubygems 1.3.4, then do the
update_rubygems
And all was well. I was able to update hoe and then successfully run spree mystore
Friday, August 14, 2009
Fixing the Kenmore washing machine
"Typical", I thought, "the extended warranty expired about a month ago."
The washer wasn't draining water during the spin cycle. But, hey, my friends had this problem with their front loader a year or so ago. The repair man said that they were using way too much soap; that the front loader needs about half the volume of soap the top loader needs. Detergent bottles and measure are still assuming a top-loading washer (I haven't seen any detergent boxes that talk about the volume needed for your front-loader -- post a comment if you have). The extra soap clogs the outlet valve and prevents the water from getting out. So, armed with this completely inadequate knowledge, I grab a screw driver and start looking for screws.
We have a Kenmore 3.1 Cubic Foot Front Loader. There is a lower front panel with two screws holding it in place; so that's where I started. The outlet from the drum as visible on the right side of the machine when facing it. A rubber tube that connected to two drain points from the drum ran to a small impeller pump and then to the outlet line. Everything was held in place with pinch-clamps. So I pull it all apart. Some soap, hair, coins, pebbles and a few tiny toys, but it really didn't look like enough to stop the machine from draining. Put everything back together and test it. Water in; start the spin cycle, water drains, no spin. Damn; I have to pull off the back panel.
That's about 40,000 screws that don't want to come out. Most of them need some Liquid Wrench and a Vise Grip to get them started. Last thing I want are 30,000 stripped screws. Um; you do know that I'm exaggerating here, right? There were only 4,o00 screws.
The drum looks fine. The belt looks fine, but I pull it off to double check. The drum spins without any catching, grinding or stiffness. The belt has no wear at all. I pull off the motor. That spins with a purr. Back-tracking the wires from the motor, I get to the control unit. That's held in with two screws and has two power and control connectors.
I pull out the control unit and, well, damn. The plastic protective sheet that covers the circuit board has a brown spot in the center. Off come two more screws, so I can lift the sheet and sure enough, something burned out some transistors.
Off to Sears.com to look up the model & buy a new control board for $150 + $20 in enhanced shipping.
Slapped that puppy in and we have clean clothes!
You know what? There is no need for that extended warranty. There is nothing in that machine that I can't fix. Darn that feels good.
Monday, June 15, 2009
Replacing a 1st Gen iPod Nano headphone jack
My wife, who is studying voice, relied upon the iPod heavily. She listened to music she was studying, new songs, recordings of her voice lessons, and warm-up routines. She would use the iPod to get her voice work in where ever possible; no small challenge while raising five children. So, when the iPod stopped working reliably, she was at a loss.
The symptoms were that the sound was not coming reliably through both headphone ear buds. I tried swapping headphones, listening to different music and as many other diagnostic comparisons as I could manage. I wish I knew how to put the iPod into diagnostic mode then. (Press and hold the center Activate button + the Menu button until the phone reboots. Then move your finger from the Menu button to the Left Arrow button and press and hold the Activate and Left Arrow button until the diagnostic menu appears. Press Activate + Menu to reboot when done).
Nonetheless, I determined that the headphone jack was defective. I despaired at the thought of trying to pry open the iPod and replace the headphone jack, but since the darn thing didn't work anyway, what did I have to loose?
I googled around for a 1st Generation iPod Nano headphone jack and purchased one for $6 from Binh An at bureenz@gmail.com through eBay. He was very helpful even when I initially purchased the wrong generation headphone jack. I also picked up a replacement battery since I knew I would have the unit open. Mr. Binh included an iPod tool for opening the unit at no cost.
I followed the directions at www.ifixit.com to open the unit and remove the motherboard. The headphone jack is held to the motherboard by a screw and there is a small nipple under the motherboard that fits into a hole in the headphone jack unit. I used a pencil tip on my soldering iron and removed the old headphone jack. I then stripped off as much old solder as possible with some solder braid and cleaned up the entire area by scraping away the solder paste that appeared.
Finally, I put the new headphone jack in place and carefully soldered the four leads to the motherboard. Keep your soldering iron hot and touch the tip to the leads one at a time. Wait a moment before moving to the next lead to let the board cool down.
Then I replaced the battery -- very straight forward compared to soldering the leads of the headphone jack. I used a tiny drop of cryanoacrylate to re-attach the grounding strap before closing up the unit. Make sure the ground strap runs from the headphone jack screw to the back plate.
Check that the little tabs on the metal back plate are all in the proper position before closing up the unit. They're easy to bend when opening the unit.
I charged up the battery and ran the unit through the diagnostics. Everything works perfectly and my wife has her iPod back for less than $20 in materials.
Tuesday, June 09, 2009
Supressing Object#id deprecation warnings
"Object#id will be deprecated; use Object#object_id"
The warning was coming from the super_inplace_controls plug in that I was using. I really like these controls, and wanted to clean up the warning.
So, I popped open vendor/plugins/super_inplace_controls/lib/super_inplace_controls.rb and checked out line 234, from which one of the warning was eminating.
Sure enough, there was, in the method id_string_for, the line:
"#{object_name}_#{method_name}_#{object.id}"
No problems, I'll just change "object.id" to "object.object_id". I did this here and on line 260 in the form_for_inplace_display method.
Of course, this broke the code and the in-place object no longer received its updates after changing a value. The ID part was necessary to retrieve the updated record and display the correct value after update. So I backed out my changes.
Some more searching and I came across a way to suppress warnings around a small block of code, which is exactly what I wanted. I didn't want to suppress all warnings, but I knew that the object.id had to be there (truth be told, I should have tried using just "object"); I just didn't want to hear about it.
To suppress the warning, I wrapped the offending line in Kernel::silence_warnings() {}. So the id_string_for method become:
def id_string_for(object_name, method_name, object)
     Kernel::silence_warnings {
        "#{object_name}_#{method_name}_#{object.id}"
    {
end
I tried the same thing for the form_for_inplace_display method and ended up putting the whole method into the silence_warnings block due to the scoping problems the block introduced.
The danger to this whole method is that, should this code base advance into a new version of Ruby or Rails where the object.id call causes an error, code that seems to work will suddenly break. On the plus side, this won't be a complex logic bomb to solve, but should come up right away in a standard testing cycle.
Friday, May 29, 2009
How to Get Real in Corporate Development, Part VI
Now it's time to look at:
Focus on fulfilling business stories, not big specification documents.
In my 20+ years of technology experience, I have seen this as being the biggest and most pervasive problem. And I understand that it comes from a very valid perspective from the engineering community.
Engineers can not easily manage huge amounts of change to requirements. Imagine building a bridge if the length of the span changes weekly, the material under the footings shift regularly and the sponsor wants the bridge in steel; no, titanium; no, wood; no, aluminum; no, steel -- you get the point. Therefore, engineers seek to define the requirements and get agreement and sign-off on those documents so they can build something once. Plus, we all know that change gets more expensive as a project progresses, especially if you have a waterfall or spiral approach. All that inflexible foundation work is very expensive to change. (Another reason to stay light and agile!)
Because of the cost of constant change, engineers push to define requirements up front so that they know what to build and where to start. They require definitions, use cases, scenarios, storyboards, and so forth. From this collected material, they define the grand scheme, break it down into delivery phases and get agreement and sign-off. Again, they want to build something once.
The flaw with this approach is that the requirements cannot be entirely known in advance. Therefore, just as soon as the project starts, the huge requirements specification is out of date. I once worked on a project with over one thousand distinct requirements. Because we were inventing new science and technology while producing a commercial product, the requirements churn was huge. We touched every single requirement dozens of times; the requirements churn topped 1000%!! That's like writing over 10,000 requirements! It was nuts!
Here is the fix: Focus on business stories. Start with the most important story. It works like this.
The business problem is, "We have to generate an inventory spreadsheet daily by running a report in our inventory management software (IMS), copying the date and transforming it into Excel, then cleaning up the data by re-arranging the columns and saving it as a semi-colon delimited file so it can be emailed to the web services group so they can update the inventory counts on the website. Errors and delays in the process can mean displaying products as in-stock when they are currently back-ordered, which annoys customers."
The business story becomes, "We want to automate the daily inventory reporting from our IMS to our web server to increase accuracy, decrease information turn around time, and free business resources for other work."
I can immediately see a number of ways to do this, from using whatever automation exists within the IMS to generate and send that report, or accessing the IMS database directly with a custom report, or using the IMS database programmatic API to write an automated application to pull the data and update the web database, or to make the web database read its data from the IMS database live, or on a daily batch bases, or exposing a web service so the data can be pushed into the web database. I bet you have a few more options.
The take away is that solving the technical problem isn't usually the problem. The problem is solving the correct business problem. Getting bogged down with a huge requirements document doesn't make solving today's problem go any faster and doesn't produce a solution to today's problem any faster.
So, how would I apply this advice to that huge 1000% churn project? Well, the biggest constraint was that we had to produce a commercially shippable product. A goal was to create a single architecture that could be leveraged along the future product line. However, since we were creating new technology and a new command and control system, we set the bar too high.
The immediate business story was to operate the machine and provide a sufficiently flexible control system that would allow for the scientific invention to continue without requiring huge software re-writes. Today, I would drop the goal that we create an architecture that could handle future demands. The project was already very challenging without that goal and we ended up pulling out all the infrastructure that was built to handle that goal anyway, just to get the first product to ship.
We made the classic mistake of solving tomorrow's problem before solving today's problem. You can avoid that mistake by focusing on the business stories that you have today and not creating that huge, over-arching requirements document that will doubtlessly saddle you to a very sizable ball and chain.
Tuesday, May 19, 2009
Weebly does web sites wite! Uh, Right!
Jen and I were talking about some web-business ideas things, so when I read about Weebly and the professed ease with which one is able to create new websites, I had to give it a shot. After all, I've used web-site builders over the years. My general take-away is that they all sucked, sucked, sucked.
I tried Word to HTML generation (puke). I tried PDF to web-site generators (retch). I tried WYSIWYG generators (pain). I tried 1and1.com website builder (oh, boring ick). I tried NetObjects (workable, but still a PITA) and ColdFusion (there's a tool, but you need to be a serious web-head to make something worth while). The best was still coding a site by hand (slow and painful to revise) until one added some code generation (PHP, which apparently means "Powerful; Help Please") and Ruby on Rails (rocks).
The technically-light web entreprenuer does not want to learn coding paradigms, so out go PHP, ROR, ColdFusion, etc. What's left is for them to hire a company ($$) and pay them to maintain the site ($$$) or hire some full time staff ($$$$).
Nope, no more. Weebly makes very attractive and highly functional websites. SEO goodness, tags, sub-pages, custom HTML (for your easy shopping-cart functionality with PayPal). Also available are two-column layouts, YouTube, Flickr, Google Maps, forums, audio, a scheduler for your service-based organization, Google Adsense and Analytics, and an API to build custom widgets which your users can use on their Weebly sites.
There's enough serious good stuff that doesn't require a deep engineering background to grock to make me say, "Weebly is a great start".
Plus, Weebly will give you a FREE subdomain, or let you purchase a domain from them, or even let you use your existing domain (They even have instructions to point your domain from GoDaddy or 1and1 to their servers).
Oh, and let's mention that you can get started with a real website, e-commerce capable, with a blog and everything for FREE. Yes. Free. As in "you do not have to pay anything ever". They make their money on the premium services. And, the upgrade to Pro wasn't expensive. Something like $5 per month. Seriously, that's way cheep.
Check it out. I fast forwarded myself many years into the future and built a fantasy website called "Doggie Heaven" that I may actually do some day in the far, far future. I added some e-commerce functionality (don't actually buy anything, OK?) and a blog in only a few minutes.
There are a bunch of very servicable templates and I love the way elements are dragged and dropped onto the page for layout.
Very impressive, congratulations Weebly.
Thursday, May 14, 2009
How to Get Real in Corporate Development, Part V
In this essay we look at:
Start with the UI design and focus on how your users will use your application.
Lots of experienced engineers like to start with the UI. A new project brings a geyser of ideas and concepts that make the fingers itch to start writing code. So, mockups, prototypes and proofs of concepts are often the first things to come out.
Good! Almost.
Don't let your engineers close themselves in a room for the four weeks that they have and create something they love. The rest of the sentence is the key -- "focus on ... your users ..."
In "Getting Real", the folks at 37signals assume they are writing an appliation for the outside to use. They don't have an opportunity to sit with their most important users when creating something new. But you do! This is where the corporate environment make Getting Real better than the original. You're making software for your business, you and your users are the domain experts. You don't need to go outside. Lucky you!
Get your engineering team (2 really good engineers and a kick-ass business analyst) in the same room as the key stakeholders -- the folks who will use this new system. Understand the business problem (Focus on One Idea) by talking to the users and examining the problem they need solved.
Have the team work through what they need to do. What tasks are grouped to gether logically? What actions are done most of the time? What actions are done only a small fraction of the time? Make the application fix the problem elegantly. That way, the team has to Build Less and will be able to Get Something Functional Deployed Quickly.
Start light. Whiteboard and sketches. Wireframes and light-HTML. Test the process. Can the users do their important tasks quickly and easily? Can they see how they get to the additional features that they don't use every day?
Put the most important information right in the middle. Make sure that new state, the populated state and the error state are considered. Ensure the engineers make the application behave appropriately when it encounters something unexpected. Leave out what doesn't belong to do the task (who cares if it's a "standard" menu item?). Put in everything important. Leave out everything else.
Match and support what the users need to do, then go and make it happen. Start with the UI. Not the UI and the database, just the UI. Not the UI and the object model, just the UI.
The engineers will have a clearer idea of what's important. They will have proven the work flow that they are about to implement. The users will have increased confidence that what will be produced will really help them.
Can you just imagine the response when the first version comes out a few weeks later? Hey, please write and let me know! I don't want to do this in a vacuum!
Thanks again for reading! In the next essay, we'll talk about how to use business stories.
Tuesday, May 12, 2009
How to Get Real in Corporate Development, Part IV
Next, we talk about structuring the project to quickly produce working code. So here it is:
Get something functional and usable deployed quickly. Plan on your next release right away. Make a choice and go with it.
Constraints are what drive a project. If you had unlimited time and money, you would probably never produce anything, as ironic as that may sound. You need limits to focus you on getting something done. My triathlon training buddy and I like to say that we "train to race and race to train" because without the race date, it's too easy to sleep in instead of hitting the pool at 5:00 AM. Constraints drive us.
So, pick one of your projects and give it to a small group of good engineers. Tell them they have four weeks to produce the first version that the business can use, but then they'll have four more weeks to produce the next release. And close the door.
They will focus on building less and implementing one idea. It's all that they'll have time for. Meanwhile, let the business know that the first iteration will be ready for use in four weeks and that the engineers will need their feedback to produce the second iteration.
Your job is to get out of the way and make sure the engineers have the tools, access and permissions they need to do their work. Herein lies additional opportunity.
Perhaps there are procedure and process documents in your organization. Perhaps there is a PMO and a review committee. Perhaps you have strict regression testing requirements for deployments. Perhaps, under normal circumstances it would take your small, Agile group of engineers four weeks just to get permission to connect to the database they need to fulfill this project.
Your opportunity is to fix this. Yes, security and access control is critically important. But, we're in the 21st Century, aren't we? Network directories with access control list are mature technologies and are probably in your organization. Are your Oracle databases set up to take advantage of your LDAP infrastructure? They should be. And no one better exists than you to make it happen.
Your organization should have four environments:
- Development -- open and flexible for the developers to do anything. Developers should work in both Debug and Non-Debug environments. Don't accept "Well, it worked on my machine."
- Test -- mirrors production, but allows developers to see everything in the system.
- Staging -- mirrors production as closely as possible
- Production -- full security and access controls.
Using virtualization, farms, and similar techniques can keep this from being a horrible amount of overhead. The key is that you need to unlock your most productive engineers. Now is a good time to note that not all of your engineers are your most productive ones. Make sure your less productive engineers are fully supporting your most productive ones. There are opportunities for training and mentoring, which build employee satisfaction and loyalty, but let's get back to getting functional software out.
How many times have you already read that a skilled and senior engineer is many times -- 10 times, 15 times, 20 times -- more productive than a junior engineer? So, we don't need to say it again. Nope. Not going to say it. Na-uh.
So then, just how will this work? Well, let's start off by looking at the typical process. The business has an urgent need. Something like, "they spend a huge amount of time running end of month reports and small errors in the data cause regular heroics."
In the typical process they fill out an Application Enhancement Request or New Application Request or some such like that. This gets reviewed at the next Application Review Committee meeting and goes back to the requester for more details. Two weeks go by for the out and back trip and the ARC approves the request. The next week a project manager and business analyst are assigned to it. They take two weeks to gather requirements, another week to create a Statement of Work and one more to have that approved by the stakeholders. The following week, a project team is assembled while the analyst writes the Functional Specification. The stakeholders don't even bother reading that.
We're up to, um, nine weeks. The team takes two months to write the application, three weeks to test it, one week to slot it into deployment and, lo and behold! More than five (5!!) months later, the application is first released. The users try it and file a new Application Enhancement Request to get it to do all the things that they need and eliminate the stupid things that crept into the application. Time to happy users: 9 months.
Here's how we want it to run:
First, you've addressed the infrastructure, you have the governance in place to enable rapid development and you've now paired your best engineers with the business users we just talked about. They meet for one day and use up another afternoon or two. In four weeks the first release is out and in use. In four more weeks the next version comes out and fixes the big problems with the first release and adds a few more features. One more four week period and the business has something that really works for them. Time to happy users: 3 months.
Even if you give this team another three iterations -- that would be a total of six (6!) releases, it would still be 66% of the duration of the Same Old Way, which would have produced 2 releases. Which application do you think would be better? The one that had six tight, focused releases, or the other one? There is no doubt in my mind.
You need to have really great developers to do this. But then again, nothing outstanding was built by mediocre engineers. You need an infrastructure that enables these really great engineers. You still need to have proper governance, SOPs, checks and balances to put software into production, but you can get the work done much, much faster by Getting Real. And this will generate real value for your company.
Thanks again for reading! In the next essay, we'll look at starting with the UI.
Tuesday, April 28, 2009
How to Get Real in Corporate Development, Part III
Focus on one idea. Start with the big problem, build the large scale functionality and don’t sweat the details.
One challenge every development effort has is where to stop. That is, where to define the edge of the application. Otherwise stated as, "how far out from the core idea is it reasonable to extend?" For example, should the document handling system not only import PDF's, but generate PDF's and fax them?
One helpful concept is the constraint of time and budget. However, it is not uncommon in corporate development shops to have projects loose the continuity of effort that makes a strict delivery time or budget tracking approach overly challenging. Here is where having one focus pays of immensely.
The thinking is: We are going to solve only the problem at hand (Today's Problem). Building less gets us a solution more quickly. Focusing on the problem helps us meet the goal of solving only the problem at hand.
Engineers with are much easier to keep focused and on-task when they have a delivery due in only a few weeks. Assigning a single, short-term purpose accomplishes this. Part of the standard challenge in corporate development shops is that the an engineer wears many hats as she maintains existing applications, develops one or more concurrent projects, and concurrently designs and plans future applications.
Switching between different types of tasks is a known productivity killer. The well-known blog by Joel on Software lays out the trouble with human task switching. You know it yourself, having your concentrated thought process interrupted is, literally, disruptive. For your team, this means that they need quite, private work spaces, few meetings during sprints, and very long periods of uninterrupted time. It also means you need to keep their plates clear of other tasks.
When you do this, your engineers will be able to stay focused on delivering that one idea.
"Hmmm, good," you say, "but what is the correct one idea?" Good question. After all, "solve global poverty" is one idea, but it's obviously not at the right level.
You're looking for an idea that represents a specific business pain point with a solution that can be produced by a small team in a few quick iterations. Since your team of three engineers can't bang out a solution to global poverty in a few three-week sprints, it's not at the right level. (But you already knew that -- this was an illustration :-)
Let's try some on for size. "Create a portal that allows secure transmission of documents from customers to service representatives." Or how about, "Replace a key paper form with an internal website." Here's what each of these have going for them:
- Each of these have a good change of being produced rapidly in only a few iterations.
- They both address only one problem.
- They require one small application to successfully meet the business need
- They both are probably very well understood by stakeholders and engineers.
Given all this, there is very likely little risk of implementing the wrong software. Even if the end product doesn't work out completely, you have only invested a small amount of time and effort and you're team has more clearly illustrated the business need.
Are you thinking that you don't have any small, simple problems to solve? Are you thinking that you need complex applications to solve complex business problems? Are you thinking that there isn't any opportunity to do something small and elegant like this in your shop?If you have any of these thoughts, or thoughts similar to them, please try an experiment. Find a key stakeholder at the line level. Perhaps a manager of customer service, or a project manager who executes key services repeatedly. Take them to lunch and ask them about the most challenging aspects of their work and the work their team or group does.
You will probably learn some things you didn't know. And some of what you learn could very well fall right into this camp of creating high-value for the business with a light, flexible, low-risk and low-cost development approach.
The next part is to work with wide brushes and big strokes. You're not looking for a whiz-bang gizmo that does everything. Get the big pieces right first. Now that you know about some business pain points (if you already knew some or knew the ones you talked about over that lunch, then kudos for you!), you're looking to address just that need. Nothing else right now, just that one need.
Yes, the fund accounting deals with mutual funds, 401(k)'s of mutual funds, annuities, closed-end funds, etc. But the bulk of the work is mutual funds. Or the bulk of the time is 401(k)'s. Or the bulk of the revenue is generated by something else. It's up to you to figure out, with your key stakeholders, what the most valuable problem to solve will be, then it's up to you to focus on that one factor.
The mandate you give your team is to fix just that one piece, make it better for the business and don't worry about the details. You will re-work and re-engineer as you dig down into the next set of priorities on upcoming iterations. But you will be providing a high-value solution in very short order, and that is worth more to the business in the long run.
Thanks for staying with me as we continue exploring the approach of Getting Real in your corporate development shop.
Monday, April 27, 2009
How to Get Real in Corporate Development, Part II
Build less. Build smaller applications with only the most important features and delivery it quickly.
This is sage advice because the corporate development life cycle is typically beset with serious birthing pains. Life cycles are heavy with process, stages, gate reviews and sign-offs. The corporate process inevitably includes requirements gathering and documentation, which involve meetings, drawings, documents and reviews. The requirements themselves transform from stakeholder statements to business requirements to technical requirements to design documents. After all that, there is finally the build, test, deploy, train, support and maintain functions. All to get one release to the business users.
Some problems with this model is that it takes too darn long and is too darn expensive. Half a year can pass filled with stakeholder interviews, document drafts, reviews, modifications and so forth that will ultimately produce a few dozen pages of specifications and diagrams. One team of stakeholders, business analysis and engineers can easily spend multiple person months or even a person year (or years) producing 60 pages. If you have a blended team salary rate of $90K, then each page costs $1,500. And you haven't built anything yet!
"No" you might be answering resolutely right now, "but we know know what to build! And we have a plan to build it."
True. I've worked this way too and I know that it helps to reduce the risk of development in the long run. You can plan for a three-phase development effort, get the budget lined up and all that. However, please consider an alternative.
Your team knew -- on day one -- what was the most important functionality needed. You, and they, don't need 6o expensive pages for that. Here's how it would work for you when you Get Real.
Two good engineers and a business analyst -- your development team -- meet with the key stakeholders. The stakeholders are asked to identify their top problems within the problem space. Then they have to choose only one. The top problem. Your development team ask enough questions to fully understand that problem and how it relates to the other top problems.
Then, they go away and produce something small to fix that top problem and only that top problem. This takes three weeks. After all, they are focused on only one key important problem.
Next, the key stakeholders begin to use the new application. After one week, the team reconvenes and the development team learns what works, what doesn't and what the next most important problem to solve is.
At this point, you've spent a tiny fraction of the expense and effort of that functional specification we taked about earlier and you have an application that, hopefully, solves an immediate business problem. If it doesn't, it has shone a big, bright light on that problem. How much longer do you think it will really take your development team to get it right? One more iteration? Two? Fine; because at that point three months will have gone by and you will have a demonstrable ROI. Compare that to a functional spec that takes six to nine months to write. You are seriously ahead of the curve.
OK, another valid concern is that when your development team learns about the next problem they will have to throw away a lot of what they've done. And they might. This is why we focus on solving one problem within the context of the larger scope. However, it is critical to solve only one problem at a time. Fix the problem you have today, not the problem you anticipate having tomorrow.
The way to do this is to build only what you need to fix that problem. Build less. What is the smallest, most elegant solution to the business problem?
Keep in mind that the business problem might not be the problem the users report. They may be unhappy with emailing spreadsheets around and want the population, generation and transmission automated. However, what they really need is a way to exchange and store information centrally instead of relying on spreadsheets.
Thanks for reading. We will continue to peel back this onion in the next essay on Getting Real in your corporate development shop.
Thursday, April 23, 2009
Using Getting Real Concepts in the Corporate Office
The talented and intelligent folks at 37signals have created some great applications. As major contributors to Ruby on Rails, they also have been fundamental in building and leading a revolution in rapidly developed, high quality web applications. 37signals is successful because they think creatively about the problem that they are trying to solve. Fortunately for all of us, part of that thinking is that they should be open and transparent with many aspects of their process and business. Therefore, they often preach about the practices that have helped them attain success.
Anyone can read their preachings online. For free! It's a great concept, and one that is often encouraged by industry leaders -- think Seth Godin for one.
You can also purchase a high-quality bound copy of their book “Getting Real” from Lulu.com. I have because I like being able to read when not in front of the computer. It’s all laid out in their book (the exact same copy in the web pages, downloadable PDF or book) – their philosophical approach to clients, development, hiring, customer service, and so on.
When I became aware of 37signals , I was developing Ruby on Rails web applications. Because I was creating web apps, which is clearly the focus of the book and philosophy, I found myself agreeing with just about everything they were saying. And hey, I’m a software development veteran.
I’ve done this for years and years and have focused extensively on managing complex projects to successful deliveries. I’ve been through waterfall, spiral, iterative, agile, ad hoc, chaotic, random and panic-driven development models. I agreed with what they were saying because it makes sense and I was able to make it work successfully on my projects without any pain or difficult transitions. There aren’t many “new” development philosophies about which that can be said! For light, flexibile web apps, “Getting Real” made total sense and, more importantly, it worked.
But what about corporate environments with ongoing development efforts? These shops have a different set of starting points and different base assumptions. What if you aren’t developing a web app? What if you aren’t focused on revenue generation with your app? What if your users are waiting for their application and have expectations about how it will help them?
Can your corporate development efforts also Get Real? Yes, I believe that they can. let's look at the components of Getting Real that will help your corporate development teams use the principles from Getting Real to improve productivity and stakeholder satisfaction levels.
Let’s start with some of the key concepts of Getting Real that are directly beneficial for all development efforts. (Links are to the essays on those concepts)
- Build less. Build smaller applications with only the most important features and deliver it quickly.
- Focus on one idea. Start with the big problem, build the large scale functionality and don’t sweat the details.
- Get something functional and useable deployed quickly. Plan on your next release right away. Make a choice and go with it.
- Start with the UI design and focus on how your users will use your application.
- Focus on fulfilling business stories, not big specification documents.
- Keep your developers on the front line of support
- Be willing to radically change the application to make it better. Resist the urge to make one monolithic tool to solve everyone’s problems at once.
Notice that these nuggets are completely environment, tool, and technology agnostic. There are some radical ideas that will doubtlessly shake up the “way things are done”, but these are the right ideas to grab onto and the right changes to make.
Fortunately, there are some components of Getting Real that an internal corporate development group can safely ignore right off the bat. For example, you’re probably not looking to generate revenue with something like an internal workflow enhancement tool. Not that you can ignore development costs or the value proposition of the development effort, but at least your users won’t be facing the prospect of paying a monthly fee. (Although, there may be something here….)
You can safely ignore the need to market and promote your application extensively. Please note the qualifier. The new whiz-bang tool you’ve developed for Operations is straight sunk costs if end users aren’t using it. Therefore, they do need to know about it and they should be excited about its arrival and feature set. But that should come from good communications and interactive product development and not from promotional marketing campaigns. Trumpeting successes is always allowed, however, so some allowance is made, but we’re not talking at all about the same degree of importance of marketing discussed in the book.
Most importantly, as you – the development lead, manager, director, VP, CTO, Whatever – read Getting Real, you can insert your favorite technology du jour whenever the book talks about web applications. The thrust of web apps in Getting Real is their rapid deployment and instant accessibility to customers. No CD burning, no distribution efforts, no retail operations, etc.
But let’s face it, how long does it really take to deploy a new version of an internally developed application in your shop whether it was written in C++, Java, Lotus Domino, C#, .NET or Ruby on Rails? If you’re thinking “weeks”, then you’ve got another problem, don’t you? You can quickly push out new applications through automated corporate update mechanisms, internal web sites and internal installers with the exact same degree of responsiveness that is the base assumption in the book So don’t sweat it that you’re not making a web app. The underlying philosophies in “Getting Real” remain pertinent, applicable and valid to you and can revolutionize your development centers.
The next essay will look at those seven take-away concepts in greater detail and will focus on how you can leverage these concepts to make high quality software faster.
Friday, March 27, 2009
SEO & SEM For C-Level Executives
- Have a domain name that matches your industry and purpose.
- Have a clear focus for your site. Make that focus permeate through your site.
- Ensure that your pages (or posts) have the name of your site in a text form that can be read by spiders.
- Ensure that your keywords and meta description match your message.
- Ensure that your pages have great titles. A great title is clear and describes the content well.
- Obtain links by creating unique, distinct and high-value content. Then, participate as a good citizen by reading and commenting on blogs, posting articles that can define you as an industry and thought leader. Your bios on other sites should link back to your site.
- Each page or post should have a clear and consistent focus. Along with your descriptive title and heading, you should use consistent language in your pages and posts.
- Be conservative with your links out and cross links within your site.
- Use available tools to analyze and improve your site. DAO is an ongoing process, not a one time event.
- Active Pay Per Click (PPC) campaigns across major search engines
- Active ad campaigns across major search engines
- Active monitoring of campaign returns, landing page conversions, and website utilization and traffic patterns
Thursday, March 19, 2009
Desperation in the business world
Monday, February 23, 2009
Don't surprise the customer!
Saturday, February 14, 2009
Fixing a Mortise and Tenon Joint
I remembered something my dad, a dentist, did when he was fitting a new bridge or cap onto a patient. He had these nifty pieces of red and blue paper and he would have the patient bite down of the paper. Where red or blue was left was where the new tooth was too high and needed to be ground down. I needed something similar; some way of marking the mortise so that I could see where material needed to be removed.