Joe Poon

Often the most painful and time consuming part of a software project is the Last Mile, getting it deployed and working in production.   Why is this the case?  To understand this, it is useful to look at the underlying issues such as not having decent source control, lack of automated tests, excessive branching (and eventual merging), dedicated teams and infrequent releases.  These impede our productivity on a daily basis and when left unresolved, become magnified during the Last Mile.

Without shortening the Last Mile, our deployments are delayed and unpredictable events.  As a result, we are unable to quickly respond to changing business needs.

So how do we shorten the Last Mile? Enter continuous integration.  The beauty of continuous integration is that it is the cornerstone for resolving these issues - it forces us to get source control under control, brings visibility to the health of our code base, challenges us to think of multiple environments and configurations and encourages us to automate repetitive tasks.  True to it’s name, it encourages us to integrate early and often. With CI, we’re already resolving integration issues from day 1 thereby making deployment to production thankfully, a non-event.

Troy Gould and I presented The Last Mile at the Agile Vancouver 2009 Conference this week.  The presentation looks at the problems faced by teams trying to get software released, recommends good CI practices, introduces build pipelining with Cruise as an overall structure to manage the build-deploy-test-release process and encourages us not to stop with CI as a build server, but leverage it to automate deployments to environments including production.  Thanks to those who attended.  It was exciting to see all those actively using CI on their projects.  The slides can be downloaded here (17MB).

Thanks to Owen Rogers and the team of organizers for an excellent event and lining up a great set of speakers including Eric Evans, Michael Feathers and Martin Fowler.

A few weeks ago, Jonathan McCracken and I were at the Calgary .NET User Group presenting on jQuery.  We started with the fundamentals of jQuery and then proceeded to build a simple application with jQuery and ASP.NET MVC to manage a grocery list.   True to it’s word, with jQuery and it’s community of plugins, we created a rich grocery list with minimal javascript.

Thanks to all who attended.  As promised, the presentation and code can be downloaded here: Grocery List

Release early and often. On projects, we strive for small incremental changes deployed frequently to production. To achieve this, we do things such as:

• test-drive our code, build, check-in, repeat
• run continuous integration
• automate deployments

However, despite our best intentions, it all comes to a grinding halt if the business doesn’t want the code in production just yet.

Fear and Freeze
Often called a ‘code-freeze’, the reason for not deploying to production usually boils down to fear that the new code will introduce bugs and that any potential downtime is intolerable.

Delaying deployment out of fear causes problems and increases risk.

• End-users suffer: requested bug fixes/features are not seen for months.
• Ceremonious process: deployment to production becomes a “one-shot” deal.
• Golden window: developers, analysts and end-users feel the pressure to make this deployment perfect.
• Loss of focus: by the time deployment rolls around, developers may be working on other features or get pulled into other projects.
• Assumptions change: over time, requirements may change - your undeployed code is already obsolete. Or, perhaps the production infrastructure has changed.
• Loss ROI: why not capitalize on your software investment sooner and reap the value?
• Big bang: with infrequent deployments, the team will tend to adjust by accumulating code changes into larger deployments.

Deployment and the break fix
Ironically, after deployment to production finally occurs, if something goes awry, the business needs to fix it immediately. Usually this will be achieved by a last minute “break fix”, where the earlier qualms to deployment are temporarily forgotten. The business demands a quick code turnaround, the justification being that it is a “fixing a bug” and “not adding functionality.” It would make sense if this type of responsive mentality was the norm throughout the development cycle and not adopted only for emergencies.

Easing business qualms
As developers we take pride in iterative, well tested code and automated deployments. However, we also need to work with the business and understand why code can’t be pushed frequently into production, and minimize the delays where possible. Sure, there are applications where the nature is truly mission critical or it’s a legacy application where granular deployments are far from trivial. However, for all the other scenarios (which I suspect are the norm) where business fear is the driving force, we can work with the business to ease these qualms.

Quick and predictable
To do so, we need to demonstrate to the business that deployments to production can be a pleasant experience for both project managers and end users. Deployments should be:

• quick and predictable
• a graceful and seamless upgrade to the end user
• responsive and understanding to any errors/oversights that arise

The more frequently we deploy, the better we become at making it a good experience for the business.
In the end, if doesn’t matter if you can go from dev to staging in 60 seconds, if your code is left sitting in the garage.

Blogger
I’ve used Blogger since 2005 and it has served me exceptionally well, not to mention that it’s completely free.  My reasons for trying WordPress were simply the myriad of themes and plugins offered by the community.  Over the last week, I’ve moved my blog from Blogger, to WordPress.com, to WordPress running on a shared host.

WordPress.com
So, I first moved over to WordPress.com - a great free blogging platform.  The import utility worked like a charm and to keep the domain name as I had on Blogger (blog.joepoon.com), I purchased the domain mapping upgrade.  I then noticed ads on the bottom of my blog posts - the ads are unintrusive and understandable, as somebody needs to pay to keep the free service going.  Moreover, one can opt out of ads for a small upgrade fee.  Even though Blogger doesn’t charge for domain mapping nor does it place ads, optional upgrade fees for an otherwise free and managed WordPress.com blog is still a great deal.  However, this gave me the nudge to take the goodness of the WordPress engine and host it on a shared host.  I’ve been meaning to get my own host anyways, there will be more maintenance, but it will be a good exercise and allows for unlimited WordPress customizations.

Self-hosted WordPress
So, after a short few days on WordPress.com, I’ve moved to WordPress on HostGator.  So far, so good.  With the power of a shared host, I decided to move my blog from a subdomain blog.joepoon.com to a directory joepoon.com/blog (you cannot map to directories with Blogger and WordPress.com).  There is a whirlwind of discussion as to where you should install your blog - subdomain, blog directory or root.  Each alternative offers trade-offs in regards to theories about page rank, site organization, site uniformity and extensibility.  Right now, joepoon.com simply redirects to /blog, leaving the root for future considerations.  More importantly however, by changing my blog location, I’ve broken links.

To redirect blog.joepoon.com links to joepoon.com/blog, I added an .htaccess file in the subdomain folder.

1. Redirect old Blogger links
   Blogger permalinks have the format: /%year%/%monthnum%/%postname%.html and exhibit the following behavior:
   • concatenates the postname to a maximum of 40 characters
   • absence of day number
   • suffixed with .html
   • excludes article words (the, as, a)

   Take the post name: 'a book titled the cat in the hat by an author named dr. seuss'

   Permalink in Blogger
   2009/03/book-titled-cat-in-hat-by-author-named.html

   Permalink in WordPress
   2009/03/31/a-book-titled-the-cat-in-the-hat-by-an-author-named-dr-seuss

   I’m using the day and name permalink setting in WordPress.  I could change it to use a custom format to append .html and exclude the day number, but the exclusion of article words and limit of 40 characters makes a direct mapping of Blogger permalinks to WordPress permalinks tricky to regex.   So, given the few posts I need to redirect, I manually redirect each link in the .htaccess file.

   # redirect the blogger feed
   RewriteRule ^feeds/posts/default$ http://joepoon.com/blog/feed/ [R=301,NC,L]
    
   # redirect the individual blogger posts
   RewriteRule ^2007/07/information-card-ruby-at-burton-group.html$ http://joepoon.com/blog/2007/07/05/information-card-ruby-at-burton-group-catalyst-conference [R=301,NC,L]
   # ... etc ...

2. Redirect old WordPress.com links
   In the transition phase, I had my posts on WordPress.com. To my amazement, within those 3-days, the Google bots indexed my blog.  So, I redirect these links as well.  As the links on WordPress.com map directly to my self-hosted WordPress blog, a wildcard replacement will do the trick.  In retrospect, experimenting with WordPress on a local install or changing the privacy settings on WordPress.com to private may have been a good idea.

   RewriteRule ^/?(.*)$ http://joepoon.com/blog/$1 [R=301,NC,L]

   It’s useful to note that I had my Blogger and WordPress.com blogs mapped to my own subdomain.  This allowed me to setup the above server-side redirects.  If it was on the .blogspot or .wordpress domains, I imagine that I would have had to resort to meta redirects or utilize one of the Blogger to WordPress conversion plugins.

Whew, so after all that, I’m settling into my new home at joepoon.com and joepoon.com/blog.
Please update your links and let me know if I’ve broken anything.

Three years ago today, a group of eager new ThoughtWorkers from six countries embarked on a four-month journey in Bangalore, India - the inaugural ThoughtWorks University.

As I sit here and reflect, it’s hard to believe that three years have passed. The name may have changed from ThoughtWorks Global Boot Camp to ThoughtWorks University (TWU), but the fond memories of being part of this technically, culturally and socially engaging experience have not.

The foundation for the practices and principles that I utilize daily on software projects today, were set, strengthened and molded through this experience. Some of these concepts, which I view as indispensable today, include TDD, pair programming, continuous integration, short iterations, stand ups, story cards with stickies, retrospectives and refactoring. TWU was also socially and culturally rewarding, spawning a wonderful network of colleagues and friends and getting immersed in the colorful Indian culture.

Throughout TWU, I kept a blog to keep track of my learnings and experiences. Below is an excerpt from the final blog entry I wrote back in 2005, recapping my TWU experience:

Bangalore was amazing - the paradoxical skyscraper next to a hut serving chai, women wearing sarees talking on cell-phones in rickshaws and business men grabbing a bite at the local darshini. It was a city in transition and I am awed by the potential that lies in India - it’s population coupled with the number of bright graduates per year is astounding. I will miss the sights and sounds of India - the hustle and bustle (with some honking) of a big city and the simplicity and serenity of cows wandering the streets.

Us ThoughtWorks Boot Campers quickly bonded and became a close unit. Together, we learned, worked, ate and travelled - I am impressed how well we gelled and synergized as a group. We all left with a global spanning network of connections and friends. We met so many ThoughtWorkers from different countries in the process - this will go a long way in establishing relationships and recognizing familiar faces on future projects.

I learned a wealth of knowledge about Agile methodologies, XP, consulting and software development during the boot camp. The subsequent project work was extremely helpful in putting a practical perspective to the lessons we learned in class. True to the old TW slogan “it’s like college with money,” it was truly an enlightening experience. I am amazed and thankful for the amount of effort that went into preparing the boot camp and the hospitality of the hosting ThoughtWorks India office. With this new knowledge and knowing that there’s always more to learn, I am looking forward to contributing to a project at home.

Since TWU, I have continued the learning trend, growing and evolving as a software developer. ThoughtWorks continues to offer ThoughtWorks University twice a year - if this piques your interest, let us know. We’re always looking for the next graduating class.

A big shout goes out to my fellow TWU’ers, trainers, and the TW Indian office!

While working on a piece of code, we’ve all come across the situation where we feel that the code should be refactored because in the back of our minds, we know that the code will soon need to do more. The conversation with your pair may go something like this - “..the system will eventually need to do XYZ and since we’re dabbling in the area, let’s refactor the code such that it is easier to incorporate feature XYZ in the future…”

At first, it may seem that to code with such foresight is not only courteous to your teammates but downright responsible programming. Moreover, the programmer in us loves to massage and improve code. However, despite these good intentions, premature refactoring of code can be a dangerous trap.

One of the problems of premature code restructuring is that it convolutes the intent and introduces complexities into the code base before they are truly necessary. Symptoms of up-front design begin to creep in, code is abstracted in overly generic and complex ways. Furthermore, we all know that like the weather, requirements change. When change happens, we sadly discover that the premature refactoring of code in anticipation of that no longer imminent feature was done under wrong assumptions and motivations. Our prefactored code has suddenly become obsolete before it was given a chance to shine.

Refactoring is improving the design of existing code and an incredibly worthtwhile practice. Perhaps prefactoring (premature refactoring) could be refactoring’s untimely evil cousin, improving the design in anticipation of non-existent code.

At the Burton Group Catalyst Conference in San Francisco last week, I had the pleasure of tending a booth in Microsoft’s Hospitality suite to spread the good word about Information Card Ruby. The booths centered on the theme of identity, with great representation from companies and leaders in this space. The excitement and activity around information cards was quite inspiring. For instance, I had an interesting discussion with a participant from the insurance industry whose clients are primarily senior citizens. The intuitive nature of the information card experience is of interest, as providing an information card from your identity selector at a website closely parallels the real world scenario of providing a piece of ID from your wallet at the supermarket. Moreover, one less username + password to remember not only appeals to seniors, but myself included! There was also interest from a wide spectrum of organizations, including a college administrator who was interested in the applicability of information card authentication for his students and staff. Many others wanted to understand more fully managed card scenarios and wished to see major sites adopting information cards. Understandably so, when more major sites adopt information cards it will provide the confidence, knowledge and momentum for other interested parties to follow. Having said that, there are impressive sandbox/demo sites available to give users a feel for the information card experience, including the nicely done Fabrikam Friends. And finally, I was impressed by Gemalto’s booth which showcased a modified Windows CardSpace client to store your information cards on a smart card. This allows you to take your information cards with you wherever you go. Kim Cameron, which I had the pleasure of meeting at Burton, nicely summarizes Gemalto’s offering here. Overall, the activity and curiosity around information cards was great to see – definitely interesting times in the identity space.

We are pleased to release Information Card Ruby 0.1.0. Information Card Ruby is an open source project which provides a plugin and gem for integrating information card authentication to your Ruby on Rails web application. The gem does the heavy lifting of decrypting, processing and authenticating information cards, while the plugin supplements your rails application with useful methods to start accepting information cards.

So, Information Card Ruby enables your Rails website to authenticate users with information cards. Sounds great you say, but you may be wondering “what is an information card?”, “where do users get information cards come from?” and “how do users select and provide information cards to my website?” Good questions. An information card is essentially a digital identity which contains claims that relate to you. For instance, I have an information card that claims my first name is Joe and that I live in Canada. As for who issues cards, there are two types of cards, namely personal and managed. Personal cards are created by you and are stored on your local machine. For instance, on Windows, you may use Windows CardSpace to create a personal information card with details such as your name and e-mail. Managed cards are obtained from Identity Providers, which are third parties whom can truly attest to your identity, such as your bank or employer. As Information Card Ruby 0.1.0 only supports personal cards at the moment, we’ll delve into managed cards another day. Alright, at this point we know what an information card is and how we can create our own personal cards. Now, it’s time to find a website to put this card to use. Conveniently, we have a demo site which has integrated the popular forum Beast with Information Card Ruby. If you visit our demo site, you can sign up and login with your information card.

On the demo site, if you click on the shiny red card icon, it will invoke an Identity Selector. For you Windows Vista (or XP with .NET 3.0 runtime) users, this will trigger Windows CardSpace. Not on a Windows machine? No worries, there are non-Windows and non-Internet Explorer specific selectors available for download. The Identity Selector retrieves the policy (ex. the required claims) of the website and pops up a GUI showing you a list of your information cards that would work for the given policy. For example, the demo site has been setup to require the claims {given name, surname, email}. Only your cards which provide a subset of these claims can be used for authentication at the demo site. With the Identity Selector, simply select one of your qualifying cards to be sent to the website for authentication. And that’s it, you’re done!

Well, that’s a brief overview of where Information Card Ruby fits in within the information card metasystem. Excited? Great. From the project home page, you can read the documentation, try the demo and download the code from rubyforge. To learn more about Windows CardSpace, visit cardspace fx. The Information Card Ruby team (Jason Sallis, Vijay Rajagopalan and Joe Poon) truly welcomes your feedback.