Configuration Management is Man’s Best Friend
Another article from the IBM Rational days, written between 1998 and 2001, co-authored with Darren Pulsipher, fellow startup traveler and friend from business school. You can find the reprint (after IBM gobbled up Rational) here. It’s a long one, but was highly rated – and landed us many other paid writing gigs. Enjoy.
Summary: What do Configuration Management and washing a dog have in common? Christian Buckley and Darren Pulsipher take a humorous look at tracking the changes within your project. Get to know the magic words
for a squeaky-clean project: parallel processes. Sit, Ubu, Sit. Good boy
We thought we’d write a nice little article on configuration management, but then something happened … we decided we had a lot to say on the subject. And since the friendly folks at Rational Developer Network thought our articles in the now defunct Rose Architect magazine were moderately humorous, we were invited to participate in the launch of their new portal. So we’re writing a series of articles on the subject. Between the day jobs, the startup projects, family responsibilities, and the daily dose of sleep deprivation we both cherish so much, some people may find it bizarre that we spend so much time writing. I guess we need something to do with our copious spare time.
Well … Mom always said that nothing worth doing is easy. On that note – you won’t find any award winning photographs or cutting-edge digital artwork in this article to divert your eyes from the text, nor will you find humorous quotations from famous philosophers converging timeless anecdotes to broad technical themes as a segue between lectures on various technical disciplines. Maybe that makes this article a tough read for some, but we’re more interested in presenting you – the reader – with the kinds of mind-bending ideas your die-hard software development enthusiast friends and their thick-skulled management counterparts want to hear about.
In short, we hope to make this series of articles worth your time. So, having alienated the thick-skulled readers (most of whom cause all of the problems in the vast majority of projects, according to all of our engineering friends), let’s proceed:
We thought we’d start things out simple by walking through some concepts of configuration management and how it can help manage the complex development efforts we face each day. To illustrate our point, let’s analyze a common problem – a dog that needs a bath. So many ideas start out as "simple" solutions to everyday problems, but then there’s that little issue of "implementation." The premise is that my dog is dirty. We recognize that soap and water clean things. Therefore, adding the dog to soap and water will clean the dog. Easy, right?
Not so fast, Mr. Software Developer. It’s so easy to jump in there without thinking about all of the factors that go into washing a dog.
Planning for the event surfaces some issues that you might not have considered: Where do we wash the dog? How much water is needed? What kind of soap do we use? How do we dry the dog once he’s clean?
Scheduling the big event reveals more issues: We can’t use the good towels, and the beach towels are still in the clothes hamper. The downstairs bathroom is off-limits because the in-laws are visiting, and the spouse would not be thrilled if we messed up the master bathroom.
Implementation uncovers more unseen problems: There is no way to tie down the dog to keep him in the tub while cleaning him. The dog doesn’t seem to like the water. The dog won’t hold still while we try to wash him. The dog is having an allergic reaction to the liquid soap. Every time we try to rinse the dog, he shakes his fur and sprays water and soap all over the bathroom. The dog has escaped and is running through the house while soaking wet.
A simple idea just got complicated.
So what does any of this have to do with configuration management? Well, for one thing, it’s all about managing the increasing complexity of a project. The above example illustrates how a series of tasks without management can quickly get out of hand. We’ve poked fun at the unforeseen complexities of an otherwise simple, everyday issue like washing a dog. But now let’s take a look at something a bit more complicated – your current development project. How do you manage the complexities of an ever growing, always expanding list of customer demands, enhancements, and features?
In the world of software development, far too many engineers, analysts and managers are tackling problems in much the same way as the aforementioned dog enthusiasts – they don’t understand the scope of the project in front of them, and yet they jump right in. And it doesn’t help that we’re bombarded with books and articles and training classes eschewing implementation as the key differentiator between projects and project teams that succeed and those that fail. They’re only partly right. We’re not saying that an artful and accurate (and speedy) execution is not critical to success (because it is), but we do insist that you cannot proceed down this path without a thoughtful and detailed design in place. Understanding this, how do you manage the complexities of your design, allowing your team to stay apprised of changes and enhancements, focus on the tasks at hand, and to execute flawlessly?
Welcome to the world of configuration management.
So what is configuration management, anyway? Why is it important to have some kind of source code management system on your project? Well, that’s what we plan to talk about across the next few articles. The underlying premise behind this series is how configuration management fits into your company, and how the mechanics of version control, structured development, and parallel and distributed development should be adopted by your entire organization.
And we’ll try our best to make these articles interesting. The world can do without one more techno-speak article without a sense of humor. We’re here to educate as well as entertain. The following is an outline of the topics we will address over the coming months. We plan to introduce some ideas that are solidly backed by industry experts, and some that may be a little on the fringe. We welcome your feedback, and plan to incorporate the most constructive and insightful of these comments into future articles. We want to hear your gripes, your horror stories, your most complicated questions, and – of course – your successes. We’d like to make this as interactive as possible, and will answer all of your questions and comments directly.
So here’s a basic outline of the topics we plan to address:
Configuration Management Maturity
The last ten years has seen an increase in the use of source code management systems, but there still remain numerous barriers to the widespread use of these products and systems. More and more companies are beginning to understand the need to version control their code and protect through process their intellectual property. With this increased use of source code control, the Configuration Manager has emerged. And because over the past few years we have seen an increase in functionality and complexity of these tools, Configuration Managers (especially those with ClearCase experience) are in high demand, and are demanding top dollar.
This article will specifically address many of the different solutions available today, and how the software development industry has changed to accept these concepts.
Standards, and How CM Can Help
Because software quality has been poor in the past, several different companies have started developing their own standards to help with product development – everything from ISO 9000 to SEI’s CMM. Companies take on these initiatives without a clear understanding of how much time and effort these projects will take to fully implement, much less reap the benefits that these standards can provide. Many times these efforts fall short because there isn’t a clear "captain of the ship" – without an evangelist for these standards, most of these implementations fail. We’ve seen it time and time again.
Configuration Management is in a unique position because these systems are typically the glue between Engineering and the rest of the product development organization. The product cannot move unless the CM team coordinates with the Build and Release team and manufacturing. Since most groups depend on the CM team for information and guidance, they are the natural leader to help the teams follow the process. In fact, most configuration management tools have some kind of mechanism that allows for automation of software development processes. In ClearCase, these mechanisms are called Triggers. You will also find that most configuration management teams are handling the defect and enhancement tools, as well. These tools are designed for customized workflow and process management.
Quality Starts with CM – Not the QA/QC Testing Process
It is a misconception that you can test quality into a product. The manufacturing industry has known for years that testing is an important aspect of product development, but it does not improve your quality – it only quantifies your quality measures. Configuration management can play an important role in improving the quality of your product through process improvement, and, hand-in-hand with the software architects’ enforcement, improve your design and implementation practices. Many of these enforcements can be put into place by a good set of tools that prevent bad things from happening – some even automatically guide software engineers to follow good practices (wizards and workflows).
Most of the quality problems that exist in product and software development are due to problems with process. Our answer to this is simple – if the process is wrong, fix it. Don’t fall into the trap that so many companies fall into when they claim they don’t have a process. Every company and development team has a process. You might not have it written down, have a clear idea of the steps involved, or understand management expectations, but there are processes that evolve over time in every organization. Most of the time they are poorly conceived, duplicating much of the work that is done. Rework is the costliest of process errors, and can only be corrected with a carefully enforced development methodology. Every time you get a new employee or an employee leaves the company, the process changes and adapts to the new participants. As you can imagine, this is not very productive, and quality suffers greatly.
We have spent a lot of time on this topic in particular, and will have much to say on the subject. In fact, our company was launched with the release of a product that addresses process. (More on that in the coming weeks)
Parallel and Distributed Development (Decrease Time-to-Market)
What manufacturing has been doing for almost 30 years, the software industry has just begun in earnest – off-shore or multiple-site development. The hope is to get cheaper, qualified employees into technology markets where it is virtually impossible to find the right people with the right skills. In the manufacturing industry, the problems with multiple locations have been identified and resolved for the most part, but the software industry presents a different set of problems. Since the development schedules for software products have become extremely short, you cannot afford to waste any time sending information back and forth between locations. The time adds up quickly. On an average project, entire weeks can be wasted.
While configuration management tools have made some leaps forward in allowing for replication of information to multiple locations, a solid plan still needs to be developed and followed to use these tools effectively. Some of the complexities of multi-site development can be solved through division of effort. If your code base is divided effectively, the dependencies between locations can be minimized and the "sync-ups" between sites can be minimized. As a result, groups can work more independently. Since the CM team is typically responsible for the multi-site coordination and integration, it is important that they have some input into the architecture of the product.
Another of the issues that need to be addressed are the branching strategies that are used between the sites, and, potentially, at each site. "Consistency is King" in multi-site development. Develop a branching strategy that does not get in the way of the developers, or they will not use it – believe us, they will subvert it. Once you have developed the branching strategy, you need to flat-out stick with it. When you have multiple locations in every hemisphere of the world, it becomes a logistical nightmare to train everyone each time you have an epiphany and modify your branching philosophy. And continued education will be an important tool that you will need to use often to keep teams from stepping all over each other.
In future articles we will be discussing some of the strategies that we have used and found successful for managing multiple locations.
Some of the first questions that we ask companies when working on their CM strategies surround their choice for defect enhancement tool, and how their system coordinates with their source code control. Many of the commercialized defect tracking tools have hooks into source code control tools. These hooks are typically defect-to-code check-in hooks. In future articles, we will be addressing the use of defect/enhancement tools to aid in the estimation of effort for new development, selective feature builds, and full artifact traceability from requirements to code, and then to the delivered product.
One of the key responsibilities of most CM teams is build and release of the product. They typically are the gatekeeper between engineering QA/QC and Manufacturing. Builds can become a nightmare if the build systems are not designed and maintained throughout the development process. Many of the issues that CM teams need to worry about include: frequency of builds, what gets built, are the builds repeatable, and which platforms need to be built. The key to all of this is that the planning of these activities needs to be done up front, before the software engineers start coding. Many times we have walked into software development companies that have had ad hoc build strategies and were disappointed when they could not integrate all of their code in a repeatable manner (what a shocker). These companies typically waste several months trying to recover from the disaster that they let happen.
You’ve completed your build and integrated all of your code – let’s not overlook the need to actually release and distribute your product effectively. It does not do your company or your product any good if you cannot get it to your customers. This part of the process includes not only your product, but also the supplemental data that goes with your product, such as defect reports, user manuals, installation guides, and runtime requirements. One of the companies that we worked with had spent all of their time building and developing their product, and when the product was finally ready for distribution to their customers, it would not run on their customer’s platform. Because the product did not include the appropriate documentation, nobody caught until it was too late the fact that the customer had moved to a new version of the operating system. Ouch.
We will be covering different types of distribution including over the Internet, labeling your code and executables so there is traceability for customer support, defining runtime requirements, and several other internal release strategies.
Third-party Libraries and Reuse
One of the traps that far too many companies fall into is the NIH syndrome (Not Invented Here). This can cause long delays in product releases, and is generally acknowledged as a waste of valuable engineering time. The alternative is not without effort. Third party tools and libraries should be part of the development strategies that your CM team evaluates. A set of tests should be developed by your company to validate the functionality of these third party libraries before they are released to your engineering teams. There is nothing worse than finding three months into development that a major part of your third party tool does not work as advertised, and the previous version does not have the same API that your team has been using.
When using third party products in the development of your product, you should treat these products as part of your code – they need to be versioned and labeled. Having to go back to a previous release of your product to fix problems can be nearly impossible if the version of a compiler or some other third party tool has changed.
In future articles we will be discussing the different mechanisms for versioning third party tools, third party tool evaluation, and reusing components internal to your company.
And that’s the long and short of it. Once again, we welcome your feedback and ideas. That’s half the fun about a Web site such as this – it can be very interactive. Sort of like the dog washing experience. Except you don’t necessarily end up out of breath and soaking wet.
But if you pay a premium, I’m sure the folks at Rational Developer Domain can accommodate you…
