Managing to the Metrics

Since my team runs the operational metrics for my organization, I’ve had some interesting conversations within my team and with some of my peers around the nature of operational metrics, and the uncanny ability for our engineering teams to figure out ways to “game” the system. Not to say that they are abusing the system — it’s just human nature to understand the parameters around which we are being measured and to do everything possible to “optimize” our performance within those boundaries. While reading Joel Spolsky’s book Smart & Gets Things Done about how to find, hire, and retain the best engineering team, one section really jumped out at me. Joel was discussing a book and study conducted by Robert D. Austin from Harvard Business School called Measuring and Managing Performance in Organizations, and stated:

“People are not chemistry experiments, because they are self-aware, and when you try to measure things about them, they’re aware of this, and they have brains they can use to get the measurement to look the way you want it to look.

“(Austin) shows that whenever you institute a new metric in a knowledge organization, that is, any organization with workers who need to do something more complicated than screwing caps on toothpaste tubes, at first you see genuine improvement of the thing you want to measure. The programmers do, actually, try to write more code every day. But very soon what you see is that the workers figure out shortcuts, so that metrics start to go through the roof, while the actual performance actually declines, because programmers start spending more time optimizing for metrics, which comes at the cost of the quality of work that they do.

More importantly, this is not just because you haven’t figured out the perfect metric. It’s the very nature of knowledge work.”