A Fable: There once lived an Emperor who commissioned a famous tailor to create a magnificent set of new royal robes.  When the new outfit was created the Emperor could not see it.  The tailor, however, made a great fuss about how well it looked on the Emperor.  Since the tailor should know, the Emperor pretended he could see the new robes.  The Emperor asked his court what they thought.  They all said it was magnificent.  They also could not see it.  After all they were not expert tailors and they were afraid to say that the Emperor was apparently insane and seeing things.  So the Emperor walked down the main street of the Capitol City of his Empire wearing his new robes.  Everyone commented on how good he looked.  Everyone except a certain little girl who said very loudly, "Why is the Emperor walking in his underwear." Everyone told her to be quiet, but it was too late, the Emperor had heard.  A great hush came over the crowd as the Emperor walked up to the little girl and said,... 

Over the years I have learned that many of the professional fads in the business and technology worlds are developed and promoted by consultants who have the need to sell their wares.  Thus they actively create new ones all of the time.  This is not all together bad.  Occasionally something turns up that has some merit and thus tends to be adopted for the long term.  The rush to embrace process methodology is one of those fads that does not have enough merit for the long term.

My simulations indicate that at best a methodology can have a small effect on efficiency, which for a high achieving organization is largely irrelevant.  At worst a proscribed methodology wastes resources and takes focus away from doing those things that lead to optimal achievement.

Of course, one cannot run an organization or a project without process.  On the other hand this is like saying one cannot write without pencil and paper, a typewriter, or a computer, etc.  However, an efficient word processor does not a writer make.  Of course, if you are going to write, you might as well have a good word processor.  But, I for one have certainly not seen any increase in the quality of writing now that word processors are so prevalent.  Documents do look nicer, and there are more of them, but the quality of intelligent content if anything has deteriorated.  Likewise despite the prevalence of PMI, IEEE, ISO, SEI, 6 Sigma and many other methodologies, I have not seen any improvement in the quality of project or organization management.  I believe these things have taken attention away from what really matters and thereby have indirectly caused deterioration in the quality of work product. 

I am not against methodology any more than I am against using a word processor.  What I am against is the blind application of methodology combined with the belief that it will eliminate incompetence and lead to a successful outcome.  For example, more than one methodology claims that the organization will benefit because results will be repeatable and not dependent upon the hero.  Apparently by re-labeling the highly competent as a hero we are to be tricked into believing such individuals are undesirable in an organization.  Indeed I have seen methodology reduce the effectiveness of such individuals and thus do give more consistent results independent of who is assigned to a project.  I, however, do not believe consistently poor results are desirable. 

But this is all rhetoric.  If I am to adhere to the intense intellectually honest discipline, which I promote in The Optimal Change Agent, then I must offer some solid evidence that the value of process methodology is greatly overstated.  To do this I turn to my computer-based simulations of organizations; some real organizations and some hypothetical organizations.  To fully understand these simulations one will need to read the Section A Theory of Society However, I believe one can get the gist of the analysis without getting into that level of detail. 

The obvious approach would be to take each process of each methodology and measure the effect of implementing the process in a simulation and compare the outcome difference.  This is how I started, however after a few cases a pattern developed from which the opportunity for a more general result became apparent.  But let us start with specific examples. 

Requirements’ tracking is a major feature of several methodologies.  Requirements’ tracking is a process used when there is a group (or person) that has agreed to accomplish a task for another group (or person).  These can be subgroups of a larger organization or they can be customer and contractor.  Let's use the term client and developer to refer to both situations.  Requirements’ tracking is a component of a project management methodology.  Requirements is that part of the negotiated agreement between client and developer that translates the client's wants into a relatively specific set of actionable items to which the developer can provide an estimate.  The requirements describe a deliverable.  We have a continuum of specificity from some minimal general statement of wants to an ever increasing specificity of requirement details.   It takes more and more effort to increase the specificity of the requirements.  There is a law of diminishing return on the effort that from a practical point of view implies that requirements can not precisely match the final result.  There will always be some level of ambiguity that will be left for interpretation. 

It is generally maintained that high specificity is in the best interest of both the client and the developer since it allows a better estimate of cost.  It is certainly in the best interest of the developer since it reduces estimation risk.  But, is it in the best interest of the client? As discussed in Optimal Organization Achievement / Understanding Wants there are several reasons why the client cannot be precise in the statement of requirements.  Insisting on too much precision will decrease the value of the project to the client.  It may however increase the value of the project to the developer. 

Given this general background we can now ask how are client's wants better satisfied with a requirements process. 

First we need to understand if a change in requirements benefits the developer.  This will depend on the nature of the agreement between the client and the developer. 

In the contracting environment a high level of specificity in requirements allows the astute contracting firm to bid what appears to be a very attractive price.  They know that the high level of specificity will result in contract changes for which the client, who is by that time locked in, will pay very dearly.  Of course, the reverse allows that client to take advantage of the contractor.  That is why a compromise is necessary.  Of course, if one is looking for a long term relationship then taking short term advantage of the other will tend to be reduced, but only when it is apparent to both parties. 

In a simulation we can take into account the quality of the requirements process and ask what a high quality requirements process provides.  To do this we must first ask, “What is a quality requirements process?” Since quality is value to someone, we can quickly see that the definition of a quality in requirements process may be different for the client and the developer. 

Imagine that we could rerun a project multiple times, such that the client got the same wants satisfied but with different levels of specificity of requirements and at potentially different costs.  There must exist an optimal specificity such that increasing it would increase the probability of a requirements change and the attendant cost, and decreasing specificity would increase the probability that the developer would waste time doing the wrong thing. 

If the client pays for the developer expenses then any variance from the client optimal specificity yields more cost to the client and greater income for the developer.  Thus either increasing or decreasing the specificity of requirements from the client optimum is beneficial to the developer. 

If the developer has bid a fixed price for the work then it is in the developer's best interest to maximize the specificity, in order to capture additional income as a result of changes in requirements and reduce the risk of developer cost increases as the client refines what is wanted later in the project. 

We have been assuming a fixed wants satisfaction and a variable specificity and a variable cost.  What if we fix the cost and try to maximize client wants?  Again there will be some optimal specificity of requirements for the client.  If the fixed cost is passed to the developer, then the developer is motivated to do a little as possible.  As specificity of requirements decreases there will be an increasing amount of wants discovery that will result in increased work for the developer under the fixed price to the client.  The developer is strongly motivated to see that the requirements are highly specific.  If they are not very specific and therefore open to interpretation, then there is a potential for the client to interpret the requirements in such a way as to get as much work as possible out of the developer. 

No matter how cost and risk are shared between the client and the developer there is always a conflict between what is best for the client and what is best for the developer.  Thus there will be a tendency to work toward a compromise that is a reasonable balance.  Move too far away from the balance and one party or the other will either fail or walk away from an agreement.  Compromise is necessary.  The problem with requirements methodologies is the tendency to move the compromise balance toward the developer without taking into account the value to the client. 

I am not opposed to requirements process.  I am opposed to the attitude that any given requirements process is the right one to use and that optimal achievement requires a requirements process.  It depends.  Some times little to none is best.  Well that is a pretty strong statement.  If it is true, then there should be clear examples where little to no requirements process is appropriate.  Let’s consider some examples. 

In Optimal Org Achievement / Understanding Wants / Getting Past Obstacles / Design Prototype I described building a pirate ship for Northern California Ballet.  The requirements, from my wife, the Artistic Director, were "I want a pirate ship with people on it to move on and off stage." That was it.  So I build a design prototype to help discover requirements in more detail.  Doing a full requirements process from any of the current methodologies would not have been in the least bit useful.  I am quite certain that the pirate ship would never have sailed on to the stage. 

I took a position as the project director for a consortium of agriculture firms with the assignment to develop a precision farming capability.  Precision farming is the ability to manage fields on 5 meter increments, varying such factors as tillage depth, seed depth, seed density, chemical application, and herbicide application.  There was a detailed strategic intent document, which was very effective in describing the wants and expectations of the consortium partners.  Incidentally, such a document is not part of any commonly promoted methodology.  There was no requirements process as defined by the current popular methodologies.  But there could have been such a process.  The expertise in the consortium partners clearly existed such that it would have been possible to institute an industry standard requirements process.  It would have been extensive and intensive.  One of the companies had spent over one million dollars developing just the database design (note, not the database, just the design) that would be needed for precision farming.  Two months into the project I told the Board of Directors that the concept of precision farming would not meet their wants, i.e.  their strategic intent.  But a slight change in the direction of the project toward a farm management planning tool would exceed their expectations.  It would have been very easy to fall into the standard methodology of doing a requirements definition and management process.  Our focus would have been so intent on defining the requirements we likely would have not noticed that it was of no value to pursue developing a precision farming capability. 

There were two pieces of software that had a more dramatic effect on the computing industry than anything other software that has ever been written.  They are the spreadsheet and the word processor.  Some might propose the World Wide Web.  And you may have your own candidates.  But certainly the spreadsheet and the word processor programs made a significant impact.  There were no requirements processes for either.  I can still remember the first time someone showed me a spreadsheet program.  I believe it was Lotus 1-2-3.  My first reaction was, "Well if that is what you wanted, you should have told me.  I could have written that." Today there are creative individuals and teams who are writing "the next hot thing." I am quite certain they are not working from a set of requirements derived by following a methodology. 

You might be thinking but those are exceptions.  Most software development is for more common and mundane applications that are best done with a formal requirements management methodology.  OK, let's consider a welfare system.  I led a team that wrote the prototype of such a system that today runs the welfare program in 35 of the 58 California Counties.  At the time I was working for a large computer manufacturer and contractor who did a lot of business with the State.  The State had a staff of 200 persons who had been working on the design of a statewide system for 2 years.  Tom, the company sales representative to Health and Human Services had told the State that we could do the project in 6 months with our new high productivity application development tool.  One of the senior executives at the State said, “Prove it.  Do a prototype.” When the project was given to me there was only one requirement.  "Build a prototype of a California statewide welfare system that is good enough to win a contract to implement statewide." I felt a little more detail was required so we obtained the services of a welfare supervisor from Napa County.  I learned building a welfare system meant automating the calculations that determine if a person is eligible for welfare and the amount of their benefit.  The rules were contained in a set of manuals that took up 3' 8" of shelf space.  Skipping ahead a couple of decades I can tell you that the system now runs of three of the largest commercial mainframes and 18  large servers to off load some of the calculation work.  The system supports about 20,000 case workers.  The entire system is oriented to collecting and maintaining the data needed to perform the Eligibility Determination and Benefits Calculation (EDBC).  The system handles in excess of 1 million transactions from the case workers per day with an average response time of just under 0.3 seconds.  And yet the average time to do an EDBC is 16.5 seconds.  There are some cases where it takes over 5 minutes.  That is a lot of computing resource.  That it should take so long for the EDBC calculation gives you some idea about the complexity involved. 

One might think that a formal requirements methodology that would translate the nearly 4 feet of rules and regulation manuals would be essential even for the prototype.  It was not.  In fact, since the prototype had to be up and running in 2 months, there was not time to do a requirements definition.  Eventually, with the prototype finished and a contract to develop a production version in place a requirements process was implemented.  It was the most efficient means to track thousands of details, all of which had to be in the final product.  My point is not that one should do away with requirements processes.  They are often an efficient means to an end.  But they have little to do with optimal achievement.  Those characteristics of organizations that are optimally achieving will do what is necessary and requirements methodology may, on occasion, be a useful tool.  To impose a requirements methodology will sometimes take focus away from what is really essential, sometimes create unnecessary work, and sometimes reduce the creativity of the combination of client and developer. 

I will briefly touch on two more processes and then return to the overall question of the value of methodology.  Let us consider risk management. 

The intent of risk management is clearly a part of foresight, my 5th critical factor of optimal organizational achievement.  The approach that every methodology recommends is to ask the experts (people in the organization, working on the project, or some outside consultants) for a list of potential risks, then rank them by probability of occurrence times cost of occurrence.  This becomes the risks list and it is tracked and decisions are made if items are to have a mitigation plan applied.  The problem with this method is the very starting position.  Asking a group, which for what ever reason has been identified as the experts, for opinions produces nothing but unsubstantiated opinion.  The best extended study that this absolutely will not work is Peter Schnaars, "Mega-Mistakes." He simply looked at the track record of the best of the best experts making forecasts.  In his words, "The record is dismal." That it is possible to do much better is a central theme of this work, but that will be discussed much latter.  The current point is that the popular methodologies provide no advice on how to forecast risks correctly. 

In my own personal experience I have seen the many popular and recommended methodologies produce lengthy lists of poorly thought out risk items.  In one project the total list contained over 4,000 items.  Such a project has no risk management at all.  Very frequently, even for shorter lists, I have seen no substantive mitigation planning.  That was actually for the best since the risks listed were not likely real. 

Who benefits from risk management planning that is done poorly?  Clearly not the client.  However the developer, especially in many formal contract situation benefits greatly.  It requires more contractor resources and increases the number of situation where the contractor cannot be held liable for failure.  All the standard methodologies require the contractor to do nothing more than produce a long list of items.  It is difficult to fail at that.  Failure only occurs when someone has to actually perform some real function. 

One last example, then a generalization.  Let's consider Quality Assurance (QA).  This is a very clever semantic ploy.  Quality Control (QC) plays a very clear and specific role in checking that a delivered product meets a certain standard.  It is generally very well defined.  QA is generally presented as the over arching concept that includes the QC process and the watch dog function that checks the process methodology is followed.  Note that it does not access at any point the quality of that process only that the process which also includes QA is followed.  Uhmm, sounds like the proverbial fox watching the hen house.  Of course, we disguise this by hiring an independent consultant (contractor) to perform the QA and blissfully ignore that it is the methodology watching the methodology.  I have seen such QA vendors produce lengthy lists of process failures on the part of the developer.  I have never seen them actually find a fault in the work products that actually do what the client wants done.  This is quite understandable; the QA vendor is too far removed from the intricate detail that arises whenever we undertake a complex task.  They simply do not have the knowledge to know if something is correct or incorrect with the product.  Remember we are not talking about QC.  In QC there is a clear test if the product meets requirements. 

OK, let me ask my usual question? Who benefits from QA? The developer and the QA group benefit.  They both consume more resources, ie make more profit at essentially no risk of process failure.  There may be a few mistakes but they will be minor and will be easily corrected.  A few mistakes corrected make everyone look good.  Does the Client benefit? Only if there is an improved probability of wants satisfaction that is worth the additional costs of the process.  Is there any evidence that all of this methodology actually improve outcome? You may believe that there must be.  I assure you there is none.  The entire basis for the methodologies is a semantic argument that is promoted by consultants and consulting organizations that greatly benefit. 

By way of generalization let me quote from "The Book of Five Rings," by Miyamoto Musashi.  "Military Science involves knowledge of the methods of other schools.  Here in this Wind Book, I have written about the other various schools of martial arts.  Unless you know the ways of other schools, you certainly cannot understand the way of my individual school. ...   In this book I will clearly expose the fact that none of these are the real Way, thus to let it be known what is good and what bad, what true and what false.  The principle of my individual school is something distinctly different.  Other schools become theatrical, dressing up and showing off to make a living, commercializing martial arts, therefore it would seem that they are not the true Way."

Musashi wrote the above more than 350 years ago.  Change just a few words and he could be speaking about the modern day commercialization of process methodology.

ç  Prior Page of Text     Next Page of Text è
(C) 2005-2014 Wayne M. Angel.  All rights reserved.