Much has been written about process improvement in the software industry. The SEI has developed the CMM (now known as CMMI) for about twenty years, yet according to a survey of 2000 government and commercial organizations about 50% are still CMM Level 2 or lower . This is not surprising considering estimates of project failures typically run very high (some estimate over half). The higher levels of CMM are designed to insure repeatability of successes as well as identification of and improvement on failures, so we might assume that organizations operating at a higher CMM level have fewer failures. This leads to an interesting question. Why would a company not improve their process by means of application of CMM KPA’s or other forms of process improvement?
To answer that question we need to look at the evolution of a software business. The industry is filled with stories of two guys starting out in a garage hacking together something they think is cool. The size of the product starts small, the enthusiasm of the people high, and the number of people on the development team low. This sounds like a formula for success so for the sake of argument let’s say it is, the product is a hit, and there are talks about an IPO. It isn’t hard to see that eventually the size of the product that is being developed is much larger, customers demand it because they need it to do more. So we scale up our product, our next release has roughly twice the requirements of the original. We know our team can handle it because this is the team that developed the original version from nothing. Isn’t that right?
No. The complexity of a software system is not well understood early in the development process. Even if all requirements are known (not realistic but perhaps a best case lower bound for unknown requirements) there does not currently exist any cheap and easy way to estimate the development time of the system accurately. There are well known models but they have a well known failing, they work well for systems that are similar to those that have already been developed. Not exactly a revolutionary concept, if you know how long it took to develop a similar system you can make a good estimate. This doesn’t really help our team who is working on extending a completely new product unlike anyone else’s.
Ok, so we have a successful product and team who is now working on the next version of the product that has twice as many requirements as the first. We know the product is more complex but we don’t know by how much. Now we return to our original question, why don’t we engage in a process improvement process? I believe the winning answer is human nature. Our experience includes the previous success, but we learn more from failure. Why? We are not in the habit of asking what we could have done better unless we see failure. So every software development project should include a lessons learned.
What does this have to do with process improvement? It’s simple, most of the big software improvement processes are, well they are big. There are metrics and paperwork and a lot of other artifacts generated that require many resources. This doesn’t help the small software development business who can’t justify the expense or doesn’t want to burden the talented engineering staff with what they see as a bullshit management process. So the question becomes how do you move toward a process improvement culture (one that asks what they can do better even when they succeed) with out alienating everyone on your existing successful teams.
I believe the answer is to make lessons learned along with qualitative statements about the state of the process a part of the project. All of the fancy and big process improvement models include quantifiable metrics that are used to evaluate the existing process and compare to the new process when it is implemented. This is necessary because we have to know where we were and where we are in order to measure any change between the two. However, we don’t need to necessarily make the measure quantitatively. We can exchange the precision of a metric for the efficiency of our talented engineers “gut” feeling, which may more accurately reflect reality. We lose the precision of the quantitative models but we gain reduced overhead and acceptance from our most important resource (the engineering team), while introducing the much needed process improvement concepts into the company culture. With a company culture that is primed for process improvement we will have an easier time introducing full blown process improvement when we do have the resources available to implement it.
 R. N. Charette. Why software fails. IEEE Spectrum, September 2005.