There is a general uncritical belief that performance measures are good.  I cannot think of any measure that does not have some negative consequence.  Good performance measures will have more positive effects than negative ones and allow the negative aspects to be apparent. 

Here are three examples of performance measures altering organization behavior in unplanned ways.

1. On Time Project Completion: I did some work for a company where project managers received bonuses and promotion based upon their record of completing projects on time.  Consequently the project managers inflated their project estimates until there was no chance of being late.  I noticed several projects that took 2 to 3 times longer in that company than elsewhere.  The problem was that by inflating estimates there was no pressure on the staff to meet challenging commitment dates.  Because of this the project managers had to add more time to their estimates so as not to be late. 

2. Child Support Collections: In an effort to improve the rate of collections in child support, the Federal Government instituted a performance measurement system that affects the allocation of dollars to the States.  The better the performance measure outcome the greater the funding and vice versa.  Sounds like a good idea? Let's see a State that does poorly in collections will get less money to support the program.  Less money to support the system means collections will decline. 

At least one State decided to push the concept out to the counties.  The Counties with the highest income levels were the most successful in collections.  It doesn't take a statistician to figure out why.  The counties with the poorest record and the greatest need got reduced funding and the counties with the most success and least need got increased funding.  Of course the State and Federal position was that the poor performers had bad management.  

One specific measure is the rate of arrears collections.  When a person fails to make a child support payment in a month the amount not paid goes into an account called arrears.  Each year a child support agency is evaluated on the percent of cases for which they collect some amount of arrears.  It does not matter how much.  There is no benefit for collecting any arrears for more than one month.  One State offered an incentive to individuals with outstanding arrears.  If they paid at least $1 in arrears within the year the State forgave 50% of the arrears debt. 

One County modified their driver license revocation program to increase arrears performance.  The State can refuse to renew a person's driver's license if they have an arrears balance due.  When a person goes to renew their license they are informed they have to go to the Child Support Office before the renewal can be processed.  The County previously insisted that a plan be instituted to catch up on the arrears.  The County staff was told to be difficult but not unreasonable.  The goal was to collect as many arrears dollars as seemed feasible.  In order to improve the arrears performance measure the policy was changed.  If a person said they cannot pay anything on arrears the staff was to ask if they have any pocket change and accept that as an arrears payment and release the hold on the license.  The arrears performance measure increased dramatically.  Total dollar collections dropped.

3. EDBC Response Time: Within the Welfare System there is a process that is known as Eligibility Determination and Benefits Calculation (EDBC).  The Welfare rules are so complex and the calculations of benefits so extended that there is no choice but to automate EDBC.  One of the important performance measures of a welfare automation system is how long it takes the computer system to respond to a user requested to perform an EDBC.  I worked on one such system where the work load for EDBC was so heavy that 90% of the calculations were sent from the mainframe to a server farm (a network of several smaller computers that were dedicated to doing EDBCs).  This had been implemented several years before I worked on the system.  During that time the EDBC response time measure had been divided into mainframe response time and server response time.  The more complicated EDBCs that needed a relatively larger number of records from the database were left on the mainframe.  The others were considered more portable and could be transferred in milliseconds for calculation to the servers (at least that was the belief).  For at least 3 years the average response time had been reported at varying between 6 and 7 seconds from month to month and the mainframe measure between 20 and 60 seconds per month.  There were known cases that took up to 15 minutes.  There were many user complaints.  We instituted a mainframe system wide performance enhancement that solved the complaints.  I then took a close look at the performance measure data collection.  I discovered several errors in the performance measures.  I redid the measures and found that the average mainframe time was 15 seconds and the average server time was 17 seconds.  A great deal of time and money had been expended to move the EDBCs to the servers based on the belief in very faulty performance measure5. 

It is extremely difficult to devise the right performance measure without too many adverse effects and very often it is difficult to get the measure correct.  Yet, I find there is a very strong tendency to believe in faulty measures.  One needs to be very careful.

ç  Prior Page of Text     Next Page of Text è
(C) 2005-2014 Wayne M. Angel.  All rights reserved.