The benefits of past performance have been oversold
The jewel in the crown of contracting reinvention is the widespread use of past performance information in vendor selection. Some say that past is prologue; the reinventors agree. They believe that rating the quality of performance for completed contracts will predict the quality of future performance by the same contractor.
The jewel in the crown of contracting reinvention is the widespread use of past
performance information in vendor selection. Some say that past is prologue; the
reinventors agree. They believe that rating the quality of performance for completed
contracts will predict the quality of future performance by the same contractor.
The government used to look at a contractor's experience--what it had done. It was rare
for the government to also evaluate how well--or poorly--the contractor did in its prior
efforts and use that information for source selection.
It makes little sense to give a contractor the same credit whether it performs well or
poorly, and thus the past performance movement was born.
What started as a commonsense reform, however, has now ballooned into a much more
ambitious project. Virtually all contracts are supposed to be rated, and the grades are a
mandatory part of the evaluation process whenever the government spends more than a
pittance.
Some extremists even want to shrink the evaluation process to only two factors--cost
and past performance.
Constructing a rating system for the past performance system has been controversial.
The most striking debate takes place over what Steve Schooner of the Office of Federal
Procurement Policy calls the expectation chasm.
The contractors being rated and the government officials who will rate them focus on
totally different problems.
Government policy-makers tend to obsess over grade inflation. They worry that so many
contractors will get the highest rankings that the distinction will become insignificant.
But contractors fear they will be the victims of unjustly low grades. Potential causes
range from personal grudges to the need to find a scapegoat for an ill-conceived program.
Some even fear retaliation against vendors that exercise their protest rights.
These contradictory concerns collide again and again as the debate continues, but no
known system satisfies both the private and public sectors.
The best example of how incompatible these interests can be is found in the ultimate
sanction when the government terminates a contract for default.
Assume that the contractor appeals the decision, as it can under the Contract Disputes
Act. Assume further that the contractor's defense is that the real cause of the breach was
the government's defective specifications in the contract.
While the matter is pending before the court or the board, what rating does the
contractor get?
These cases can linger for years. If the termination is ignored until the final
decision, the result will be very stale news. But if the contractor is exonerated, it will
have suffered, without cause, years of a bad reputation. The good news might then be
delivered to the trustee in bankruptcy.
Efforts to find a quick way to allocate blame usually come up against the same
obstacles that tend to prolong these cases. Often, there are no easy answers, and the
truth lies in untangling a long and complex history, often intertwined with technical
issues.
The first step out of this conundrum is to stop striving for goals that are as
meaningless as they are impractical.
The vast bulk of contracts fall into a middle ground wherein performance is neither
outstanding nor inadequate. Contractors may occasionally distinguish themselves, for good
or for ill, but these are isolated instances in an otherwise ordinary record. This is not
a condemnation of the quality of contractors; it is the mathematical principle of standard
distribution.
Therefore, no ranking system can truly sort out the bulk of contractors who occupy the
vast middle range of ratings.
To be sure, the ratings may seem to express a degree of difference, especially if the
system is set up to exaggerate fine distinctions. But for most contractors, most of the
time, such differences will be meaningless.
Past performance isn't a cure-all, but neither is it useless. It has been oversold.
Patterns can still be revealing. When a contractor repeatedly gets consistently high or
low ratings, it is probably significant. Predictive information, to the extent it exists
here, is at either end of the bell curve. Past performance rankings are accurate and
useful when their strength and consistency clearly identify outliers--the really excellent
or the abysmally substandard.
But using the ratings to choose among contractors in the middle range is to pursue too
fine an objective with too blunt an instrument.
Joseph J. Petrillo is an attorney with the Washington law firm of Petrillo &
Powell, PLLC. E-mail him at jpetrillo@counsel.com.
NEXT STORY: GSA: Contract out IT work if it makes sense