Once you create a standardized test for anything, people stop optimizing for the outcome the test was supposed to measure and start optimizing for the test itself.
2
Apple's iPhone Duo exists because foldable phones are a benchmark achievement in consumer electronics — a measurable innovation milestone that proves capability — even though most people with actual phones don't want them. The prosecutor's case against Timothy Busfield looked procedurally sound on paper until it met reality. Stanley Tucci was cast in a mediocre heist film because his name is a testable asset on a spreadsheet.
3
In a world where everything gets measured, the unmeasurable — taste, judgment, fairness, actual human need — stops mattering inside organizations even though it's the only thing that matters outside them. The winner won't be whoever builds the best benchmark; it will be whoever builds the thing people actually want and is brave enough to admit their measurements didn't predict it.
W66
The Signal
The proxy problem
Observation
We measure what's easy instead of what matters.
Moritz Hardt knows AI benchmarking is broken because it has become theater for stakeholders instead of evidence for builders. The same pattern shows up everywhere now — from how streaming services greenlight shows (Miranda Hart's emotional honesty beats actual storytelling) to how phones get designed (two displays instead of one better one) to how justice systems fail (a prosecutor's metrics look fine until they don't, and then a judge has to throw it out). We've built systems that optimize for the testable thing, not the true thing.
The organizations that will win the next decade are the ones willing to look foolish by ignoring their own metrics long enough to ask whether they're measuring the right thing.