Two interesting reviews today: David
Leonhardt on Ian Ayres, and Daniel
Davies on Steven Levitt and Roland Fryer. There’s a meme catching hold –
think Freakonomics and Moneyball – which says that rigorous empirical
analysis can reveal otherwise-unobtainable insights. But these reviews, along
with Davies’s latest
Freakonomics review (part four of a long, wonderful, and hilariously delayed
series) constitute a rather interesting countermeme.
The problem is that attempts at rigorous empirical analysis have long since
eclipsed the number of empirically-rigorous datasets available. And so you end
up with Levitt and Fryer running regression analyses on a KKK dataset which
is dominated by Pennsylvania and Ohio, or with Ayres gushing about call-center
call times on the basis of some gushing coverage in a back-issue of Fast Company.
As Davies points out in his Freakonomics review, insofar as economics works
it’s largely because a lot of very smart people spent a great deal of time working
out how to generate reliable statistics; and also because whenever someone does
any kind of economics research, you can be sure that they’re using the same
statistics as everybody else. Similarly, insofar as Moneyball worked, that was
probably due to the fact that baseball is one of the few other places where
you can find a set of universally-accepted and exhaustive statistics. But when
you start looking at phone-call durations or KKK membership, you’ve moved a
very long way into much squishier territory.