The Limits of Empiricism

Two interesting reviews today: David

Leonhardt on Ian Ayres, and Daniel

Davies on Steven Levitt and Roland Fryer. There’s a meme catching hold –

think Freakonomics and Moneyball – which says that rigorous empirical

analysis can reveal otherwise-unobtainable insights. But these reviews, along

with Davies’s latest

Freakonomics review (part four of a long, wonderful, and hilariously delayed

series) constitute a rather interesting countermeme.

The problem is that attempts at rigorous empirical analysis have long since

eclipsed the number of empirically-rigorous datasets available. And so you end

up with Levitt and Fryer running regression analyses on a KKK dataset which

is dominated by Pennsylvania and Ohio, or with Ayres gushing about call-center

call times on the basis of some gushing coverage in a back-issue of Fast Company.

As Davies points out in his Freakonomics review, insofar as economics works

it’s largely because a lot of very smart people spent a great deal of time working

out how to generate reliable statistics; and also because whenever someone does

any kind of economics research, you can be sure that they’re using the same

statistics as everybody else. Similarly, insofar as Moneyball worked, that was

probably due to the fact that baseball is one of the few other places where

you can find a set of universally-accepted and exhaustive statistics. But when

you start looking at phone-call durations or KKK membership, you’ve moved a

very long way into much squishier territory.

This entry was posted in economics, statistics. Bookmark the permalink.