Performance Evaluations That Can't Deal with Ambiguity, Can't Deal with Reality
Don’t think you can’t be fired for doing the right thing. This week, San Francisco Weekly reported on the case of Frank Lee, a San Francisco cop who may lose his job because he didn’t execute a search warrant fast enough.
The catch? The search warrant turned out to be illegal and Lee, a homicide investigator, delayed implementing it because he was trying to explain to his superiors that it was illegal.
According to article, Lee was told: “You're in homicide. This is the way we do things. I want that evidence.”
Now, a good cop may be tossed out of the force for knowing and trying to obey the law. Meanwhile hundreds of cases are being tossed out in San Francisco because police have made illegal searches—and not a single officer is being disciplined for it.
If only this were an unusual circumstance.
But reports out of the Washington D.C. school system shows that many of the testing gains made under Chancellor Michelle Rhee were caused by cheating. By someone in the schools (teachers? principals?) erasing wrong answers and replacing them with right ones. According to reports, they're under enormous pressure.
So far no one who cheated on the tests has been disciplined, let alone fired, but Rhee fired nearly 45 percent of the teachers and principals in the system who correctly told her that they couldn’t promise major gains every year without actually getting to work with the kids first.
What we’re seeing is an epidemic of systems where the desired results are twisting the process and ruining what really matters. A school system gets so focused on test scores that it rewards teachers willing to cheat and fires teachers who aren’t; a police department gets so focused on obtaining evidence that it tries to fire the offer who won’t commit an illegal search. If this reminds you of the housing bubble—where pursuit of short-term profits led to the creation of enormously destructive financial instruments—then you’re paying attention.
The fact that this is happening in so many sectors—public and private, academic and blue collar—indicates that it is more than a few bad apples or an industry gone wrong. It is a systemic problem with the way we think about measurement and results.
In an article on Governing.com, Ken Miller makes the reasonable case that the problem is “management by fear.” And he's partly right.
But the underlying problem is that management has become reductionist: We’re looking for the easiest result to measure and applying ourselves to that. Test scores, short term profits, the number of searches conducted—there’s nothing ambiguous about these measurements. They’re easy to put on a graph or chart and, when compared from year-to-year, they either go up or down.
As a result, our systems bend around them. Problem is they’re not the same as what we really care about: children learning, economic health, a just justice system. These things—to the extent that they can be measured at all—can only be measured by indicators that are ambiguous at best. “Intangibles” count. Holistic analyses get you a more accurate picture than reducing complex systems down to single indicators.
Obviously our management culture, as it exists, isn’t comfortable with that. The idea that there is no single easy measurement by which we can track results doesn’t fit well with the culture we have.
But we’d better get used to it because, for most of the things we care about, measuring performance by an easily trackable result is synonymous with shooting ourselves in the foot.
A result we can easily count, by the way.