« Calipari and rendition | Main | Wolfowitz »

Freakonomics

Last year, I said some nice things about Steven Levitt and Stephen Dubner, or at least about an article they wrote in the New York Times Magazine:

The best piece of all in the magazine, however, doesn't look big, it looks small. Stephen Dubner and Steven Levitt have found an absolute gold mine in Paul F., a trained economist who now runs a bagels-and-doughnuts service for local offices. His business runs on the honour system: throw a buck in the box for a bagel, or 50 cents for a doughnut. And, of course, he's kept detailed data on delinquency rates, which go up when the weather is bad, or Christmas is nigh, or even when the office exceeds a certain size. There are some wonderful results...

The two Steves are very different individuals. Stephen Dubner is a journalist and regular contributor to the New York Times Magazine who wrote an article about Steven Levitt in 2003. From that article first sprang the bagel story, and now a fully-fledged book, Freakonomics. (In the book we get the bagel article all over again, except for now Paul F. is graced with his full name: Paul Feldman.) The publishers liked my blog about the article so much that they sent me an advance copy of the book; I liked the article so much that I very much looked forward to reading it. A match made in heaven, really – or at least in the blogosphere.

Steven Levitt is much better known: mention his name to anybody even peripherally associated with economics, and they have a tendency to get extremely excited. Levitt is not a macroeconomist at all; rather, he's an expert at microeconomics, where he tries to find statistically interesting or important patterns in everyday things. Just scrolling down his list of recent publications is mouthwatering: whose interest wouldn't be piqued by a peer-reviewed article entitled "A Test of Mixed Strategy Equilibria: Penalty Kicks in Soccer"?

In fact, Levitt has a strong interest in both sports and sports betting: he's shown that simply by betting on home underdogs in NFL games, people can break even, and he has a secret plan (really!) to make money betting on horses. Unfortunately, none of that material makes it into the book.

Rather, the aim of the book is to popularize Levitt's work more generally, and to introduce his brand of thinking to the general public in a format which was slightly longer lasting than an article for the New York Times magazine. Levitt would provide the ideas, and Dubner would provide the prose.

The book also, quite explicitly, aims to capitalize on the success of Malcom Gladwell's The Tipping Point, or Oliver Sacks's The Man Who Mistook His Wife For A Hat: books where anecdotal evidence is corralled in support of broader, more empirical, and yet extremely original, ways of looking at the world.

There's a problem, however: it turns out that while Levitt does have a very interesting way of looking at the world, he has very little in the way of broader theses which he might wish to support. The authors make no bones about this:

Most books put forth a single theme, crisply expressed in a sentence or two, and then tell the entire story of that theme... This book boasts no such unifying theme. We did consider, for about six minutes, writing a book that would revolve around a single theme – the theory and practice of applied microeconomics, anyone? – but opted instead for a sort of treasure-hunt approach.

What that means, in practice, is a series of disjointed chapters, some of which are much better than others. In fact, it's the earlier ones which are better than the later ones, which means that readers are likely to come away disappointed, as though they were promised something which was never delivered. And due to its disjointedness, if someone is asked what the book is about, they're not really going to be able to come up with an answer. The subtitle of the book is "A Rogue Economist Explores the Hidden Side of Everything", but that doesn't really help us all that much – especially since Levitt, winner in 2003 of the John Bates Clark Medal (previous winners: everyone from Milton Friedman to Paul Krugman and Larry Summers), is no one's idea of a "rogue economist".

In fact, Levitt is entirely unthreatening to the economic establishment, which is one reason why he and his co-author have to create a straw man, in the form of something called "the conventional wisdom", against which Levitt can valiantly fight. Yet after telling us in their introduction that "the conventional wisdom is often wrong", the authors have no problem relying on it whenever they feel like it.

The most egregious form of conventional wisdom is probably the urban myth: something which millions of people believe, but which is entirely fictional. Kidney harvesting, that sort of thing. Once you know about urban myths, you generally know them when you see them, and when my friend Paul told me a couple of years ago about a friend of his who worked at a hospital and who witnessed an African-American woman naming her daughter Shithead (pronounced "Shateed"), I didn't think it was the literal truth.

But here's Levitt and Dubner:

Roland G Fryer Jr, while discussing his names research on a radio show, took a call from a black woman who was upset with the name just given to her baby niece. It was pronounced shuh-TEED but was in fact spelled "Shithead"... OrangeJello, LemonJello and Shithead have yet to catch on among the masses, but other names do... It would be an overstatement to suggest that all parents are looking – whether consciously or not – for a "smart" name or a "high-end" name. But they are all trying to signal something with a name, whether the name is Winner or Loser, Madison or Amber, Shithead or Sander.

In other words, when it suits them, Levitt and Dubner lap up the conventional wisdom to the point at which they perpetuate something which has all the markings of an urban myth. (Phone-in talk shows on the radio are one of the prime propagating vectors of these things.) Most likely, the woman who phoned in genuinely believed that at least one black woman (a "friend of a friend", no doubt) had named her daugher Shithead, and she simply personalized the anecdote for the sake of making it more immediate to the radio audience. That's the way that urban myths work: it's always a friend of a friend, never a friend of a friend of a friend. But Roland Fryer was simply laughing, so he didn't check her claim out, and Levitt and Dubner have now given a semblance of academic legitimacy to what is essentially a racist stereotype.

In fact, there's more than a little editorialising going on throughout the book, often for no particular reason. In what I expect is going to be the jacket copy on the hardback, we're given a list of "provocative questions" that Levitt asks, like, um, "Why do we spend more on chewing gum than on campaign contributions?" In the event, that question is never really asked, let alone answered. Rather, we get a section of the introduction which begins like this:

Of all the truisms about politics, one is held to be truer than the rest: money buys elections.

It ends like this:

What about the other half of the election truism – that the amount of money spent on campaign finance is obscenely huge? In a typical election period that includes campaigns for the presidency, the Senate, and the House of Representatives, about $1 billion is spent per year – which sounds like a lot of money, unless you care to measure it against something seemingly less important than democratic elections.
It is the same amount, for instance, that Americans spend every year on chewing gum.

Now it might well be true that "money buys elections" is conventional wisdom. But I don't believe that most people think that it is "truer" than all other bits of political conventional wisdom, such as, say, the "divided nation" hypothesis. And as for the chewing-gum factoid, a close reading reveals that what the authors are doing is taking a whole election cycle – four years, I would assume – and working out the annual spending per cycle. In a presidential election year like 2004, it's probably safe to say that Americans spent much more on political donations than they did on chewing gum.

Normally, I would be much more forgiving of such rhetorical excesses than I am here. But this is a book which claims to be solidly rooted in empirical and indubitable data. The running theme of the book is "you might think X, but in fact, Y". So lazy assumptions are more egregious here than they are normally.

Levitt and Dubner like to get holier-than-thou when others make mistakes. At one point, they eviscerate a homeless advocate named Mitch Snyder, for saying that there were 3 million homeless Americans:

When Snyder was pressed on his figure of 3 million homeless, he admitted that it was a fabrication... It may be sad but not surprising to learn that experts like Snyder can be self-interested to the point of deceit. But they cannot deceive on their own. Journalists need experts as much badly as experts need journalists... Working together, journalists and experts are the architects of much conventional wisdom.

So what happens when Dubner and Levitt – a classic pairing of a journalist and an expert – get together? It may be sad but not surprising to learn that even they can come up with decidedly dodgy numbers. Here's one:

Economists have a curious habit of affixing numbers to complicated transactions. Consider the effort to save the northern spotted owl from extinction. One economic study found that in order to protect roughly five thousand owls, the opportunity costs – that is, the income surrendered by the logging industry and others – would be $46 billion, or just over $9 million per owl.

When alarmist figures in the billions start getting quoted, I immediately start getting suspicious. So I went to the footnotes, which cited a paper by Jason Shogren from which, I believe, this is extracted. Here's what Shogren actually writes:

Opportunity costs have been estimated for a few high-profile, regional ESA conflicts such as the northern spotted owl. One study estimated that an owl recovery plan that increased the survival odds to 91 percent for a population of about 1,600 to 2,400 owl pairs would decrease economic welfare by $33 billion (1990 dollars), with a disproportionate share of the losses borne by the regional producers of intermediate wood products, a relatively small segment of the population (Montgomery et al. 1994). If the recovery plan tried to push a goal of 95 percent survival odds, costs increased to $46 billion. Another study estimated the short-run and long-run opportunity costs to Washington and Oregon of owl protection at $1.2 billion and $450 million (Rubin et al. 1991).

In other words, Levitt and Dubner have taken the very highest estimate from Shogren's paper, one which Shogren didn't even come up with himself, and used it uncritically. They could have used the $1.2 billion and $450 million estimates instead, of course, but chose not to for reasons we can only guess at.

I also sent Shogren an email, asking him what he thought of this use of his number. He said that the $46 billion was "an outside estimate," and added:

We used the number to illustrate what little we do know about costs of the Endangered Species Act. Other numbers we cite in the paper say that the ESA is more about transfers of wealth (from agricultural to recreation) than about the loss of wealth. 

The really weird thing is that the factoid aboout the spotted owl seems to have been dropped into the book utterly randomly: it's there only to illustrate the broader point that economists try to measure all manner of different things. But why would Levitt and Dubner concentrate only on the costs of saving the spotted owl, while ignoring the benefits? And why would they pick a number which seems designed to shock, rather than a much more reliable number which is less shocking, like the cost per life saved of installing various safety features on roads or subways? The broader context, after all, is that of abortion, and whether it's possible to quantify the costs and benefits of abortion, after taking into account its role in lowering the crime rate. Saved lives, in this context, seem far more germane than saved owls.

But throughout the book, the authors veer wildly between presenting Levitt's research and presenting striking information (Shithead! $46 billion! Chewing gum!) seemingly just for the sake of showing readers how crazy this world is.

When Levitt and Dubner venture into the anecdotal, then, this is a weak book. I'm sure that the decision to do so was based on conventional wisdom within the publishing world: that if you want to write for a mass audience about empirical ideas, then you have to boil everything down to anecdote. So much for ignoring conventional wisdom.

The empirical parts of the book are much stronger. Chapter 1, for instance, entitled "What Do Schoolteachers and Sumo Wrestlers Have in Common?", gives a number of good examples of when people cheat, and how statisticians like Levitt can catch them doing so. First are Chicago schoolteachers: by looking at raw test results, Levitt could see where they had altered their pupils' scores before handing the tests in. Then there are the sumo wrestlers: it turns out that pairs of them often engage in deals where one wrestler will win one match and the other will win the next, when such a deal will keep the first wrestler in the top leagues. Again, this is quite clear from the statistical data, if you know where and how to look.

Even in this chapter, however, with three different examples of Levitt's work (here's where we find the bagel guy), the authors aren't happy unless they add in some extra colour. So we get padding, too: apparently another teacher cheated, at the University of Georgia, in 2001. There was no statistical analysis involved in his capture, and his story doesn't help the structure of the chapter at all, but we're told about it anyway: if we're interested in cheating schoolteachers in Chicago, might we not be interested in cheating university teachers in Georgia? I think the inclusion of the story betrays a certain lack of faith in how compelling Levitt's story really is. While Levitt is interested in ways of catching cheats, the chapter's attention keeps wandering from that subject, which means we end up learning about other forms of cheating, as opposed to other forms of detecting cheating.

Elsewhere, we're treated to a mildly diverting treatise on the economics of fear, first introduced in Chapter 2, when the authors discuss how a very small number of lynchings managed to cow an entire race into submission during the 1940s.

There are few incentives more powerful than the fear of random violence – which, in essence, is why terrorism is so effective.

This is expanded upon in Chapter 5:

Fear best thrives in the present tense. That is why experts rely on it; in a world that is increasingly impatient with long-term processes, fear is a potent short-term play. Imagine that you are a government official charged with procuring the funds to fight one of two proven killers: terrorist attacks and heart disease. Which do you think the members of Congress will open up the coffers for? The likelihood of any given person being killed in a terrorist attack are infinitessimally smaller than the likelihood that the same person will clog up his arteries with fatty food and die of heart disease. But...

Let's put to one side the fact that "infinitessimally smaller" actually means "of roughly the same size". This is all prefatory to Levitt's determination to teach parents that swimming pools are more dangerous than guns, and that we ought to be more afraid of everyday killers than we are of rare occurrences like terrorist attacks. The problem here isn't that Levitt is wrong; in fact, he's absolutely right. But he's hardly a lone warrior in this war. Here's Senator Joseph Biden of Delaware, in this week's New Yorker:

I would say to John [Kerry], ‘Let me put it to you this way. The Lord Almighty, or Allah, whoever, if he came to every kitchen table in America and said, “Look, I have a Faustian bargain for you, you choose. I will guarantee to you that I will end all terror threats against the United States within the year, but in return for that there will be no help for education, no help for Social Security, no help for health care.” What do you do?’
“My answer,” Biden said, “is that seventy-five per cent of the American people would buy that bargain.”

So Levitt gets it, and Biden gets it. But Biden puts it in a more arresting way, and he doesn't rely on vaguely inchoate notions like "a world that is increasingly impatient with long-term processes". And he certainly doesn't provide, at the end of a list of "Suggested Interview Questions", this:

14. You say your ultimate aim is to figure out a way to use economic data to fight terrorism. How would you do it?

Levitt, it seems, wants to have his cake and eat it: on the one hand he tells us that we shouldn't be nearly as afraid of terrorism as we are, and then on the other he (or his publicist) says that his "ultimate aim" is to fight terrorism.

And just like the University of Georgia's cheating professors, the stuff on fear serves as extra padding. In the first chapter there are two full pages of alphanumerical data, which we're actually given, rather than simply told about. The final chapter is much worse: it has no fewer than 26 different tables listing various boys' and girls' names, sorted according to popularity, sex, what they indicate about the race of the child, what they indicate about the level of education of the child's parents, and so forth. There's even a table telling us that the mother of the average Jasmine has 12.88 years of education, compared to 11.94 years of education for the average Jazmine's mother and 13.23 years of education for the average Jasmyn's mother.

All of these tables do a great job at taking up space in this very short book, but their dialectical purpose, I have to admit, defeats me. The beginning of the chapter (subtitile: Would A Roshanda By Any Other Name Smell As Sweet?) includes one marginally interesting empirical result: that a black child with a distinctly black name does not suffer quantifiable economic disadvantage in comparison with an identical black child with a distinctly white name. Once we've discovered that, however, we then go on a veritable fun-ride of naming conventions and correlations, to no obvious end. Here's the final conclusion of the final chapter (if you don't include the Epilogue):

What the California names data suggest is that an overwhelming number of parents use a name to signal their own expectations of how successful their children will be. (Bold emphasis mine; italic emphasis the authors.)

Nowhere in the chapter have we been told about the numbers of parents who choose various different types of name, or the numbers who don't. There have been lots of lists of names, but no numbers at all: the whole chapter reads as though if you pile up enough anecdote, you can generate empirical conclusions by sheer mass alone.

It conceivable, although I doubt it, that there's something in the data which supports the authors' conclusion. There's certainly nothing in the book, however. If they know how many parents use a name to signal their own expectations of how successful their children will be, why don't they tell us? And if they don't know what the number is, how do they know that the number is overwhelming?

One of the problems is that by this point in the book, the authors have seemingly run out of Levitt studies to talk about. There were three in the first chapter, you'll recall; there are two in Chapter Two – a study of how estate agents get higher prices for their houses, and another on discrimination in the TV program The Weakest Link. In Chapter Three there's one, on the economics of a gang of crack dealers. Apparently it's much the same as the economics of McDonald's.

Chapter Four brings together Levitt's other studies on crime: how adding policemen brings it down, and how it's related to both the crack epidemic and to abortion rates. Chapter Five barely touches on Levitt research; one study about the black-white test score gap is mentioned. And Chapter Six, as we have seen, starts with Levitt research on the consequences of having a distinctively black name, and then sets off into uncharted territory.

As the amount of empirical research underpinning each chapter slowly ebbs away over the course of the book, the authors are forced to cast their net ever wider for interesting, if dubiously relevant, anecdotes. We're given a potted history of the Ku Klux Klan. We find which words correlate with high prices in real-estate ads. We learn how to get a date from online dating sites. We're asked what crack cocaine has in common with nylon stockings.

And, at the beginning of each chapter, we get a little excerpt from Dubner's original profile of Levitt, including some extremely personal information which felt to me at least rather gratuitous. By the end, we've learned not only that Levitt lost his firstborn son to pneumococcal meningitis, but also that his coauthor on one paper, Roland Fryer, was abandoned by his mother, beaten by his alcoholic sex-criminal father, and became a gun-toting drug dealer before turning his life around and ending up at Harvard.

This isn't economics, and it isn't freakonomics. It's prurient biography masquerading as illuminating anecdote.

Which brings me to the question of what, exactly, these anecdotes are purportedly illuminating. The authors go to great pains to say that "there is no unifying theme" to the book, but I'm not sure that's true. Underneath it all, I definitely detect a right-libertarian stance, which is all the more noticeable for the authors' protestations that it isn't there. Here's an excerpt from the introduction to the abortion chapter:

Conservatives were enraged that abortion could be construed as a crime-fighting tool. Liberals were aghast that poor and black women were singled out. Economists grumbled that Levitt's methodology was not sound. As the media gorged on the abortion-crime story, Levitt came under direct assault. He was called an ideologue (by conservatives and liberals alike), a eugenicist, a racist, and downright evil.
In reality, he seems to be very much none of those.

That, I think, is what is known as setting the bar very low. I'm sure that Levitt is not downright evil, nor is he racist, and nor is he an ideologue. But over the course of the book, the authors do seem to have been spending quite a bit of time in the shadow of Ayn Rand. (Levitt is at Chicago, don't forget.) From reading the book, we learn that:

• Big Government is bad, especially in its more lefty manifestations: look at those cheating Chicago public schoolteachers, or the insane cost of the Endangered Species Act. Left-wingers are really bad: look at the made-up numbers of that homeless advocate. Right-wingers are also bad, but maybe not as bad: the Trent Lott affair is referred to in the table of contents with the question "Is Trent Lott more racist than the average Weakest Link contestant?" (The answer, one assumes from the question, is no.) And if you're worried about the influence of money in politics, don't be: there's barely any.
  
• If you're relying on anybody else to make your life better, your faith is probably misplaced. Doctors, real-estate agents and other experts are basically self-interested. Even your parents might want to make your life better, but in fact they aren't "likely to make a shard of difference," whether they're choosing schools, buying car seats, or choosing names. Our destiny is largely set at birth: who our biological parents are is much more of a determining factor than anything else. If your parents are poor and black, they're very likely to give you a ridiculous name, and you are relatively likely to become a criminal.
  
• Some individuals, however, transcend their destiny. Steven Levitt "is the guy who, in the slapstick scenario, sees all the engineers futzing with a broken machine – and then realizes that no one has thought to plug it in." Stetson Kennedy had "a stroke of brilliance" when he leaked secret Klan information to The Adventures of Superman radio show and became "the single most important factor in preventing a postwar revival of the Ku Klux Klan in the North". Roland Fryer went from gangbanger to Harvard economist.

At heart, Freakonomics is not a book about economics at all: it's a book about a hero who can ask the right questions and uncover the truth. Once you've read it, you'll know lots of interesting facts you didn't know before. But it won't make you stop and think (stopping and thinking is the job of the hero), and it certainly won't "literally redefine the way we view the modern world," as the jacket copy has it. You know how that crack gang was just like McDonald's? Well, this book is too. You finish it off quickly, but end up vaguely dissatisfied: all the added sweeteners and calories serve to mask the fact that there's very little protein or nutritional value.

If you're interested in Steven Levitt the person, then by all means read Freakonomics: you'll learn a decent amount about him and his interests. If you're interested in his work, however, I'd advise waiting for his next book, or maybe trying to track down his original papers. This short and hurried book is not the book you're looking for.

Posted by Felix at 19:36 EST

Comments

Posted by: Matthew at 18:38 EST, March 19, 2005

Posted by: Steve Sailer at 2:29 EST, April 06, 2005

Posted by: Steve Sailer at 1:54 EST, April 23, 2005

Posted by: Michael Flynn at 1:45 EST, April 27, 2005

Posted by: Taos at 21:12 EST, May 13, 2005

Posted by: Taos at 21:18 EST, May 13, 2005

Posted by: fullymubbed at 0:40 EST, July 11, 2005

Posted by: ono at 7:57 EST, August 08, 2005

Posted by: kjw at 2:16 EST, August 23, 2005

Posted by: John at 5:17 EST, December 04, 2005

Posted by: RCL at 13:54 EST, January 25, 2006

Posted by: Eric at 3:35 EST, February 07, 2006

Posted by: Tim Aaronson at 19:03 EST, September 22, 2006

Posted by: Thomas at 9:15 EST, November 23, 2006

Posted by: Daniel at 8:38 EST, December 11, 2006

Posted by: J. at 17:31 EST, December 08, 2007

Post a comment

Name:

Email Address:

URL:
Remember Me? YesNo

Subscribe to This Entry:

Antispam question: How many letters 'o' in the word 'school'? (type the digit)

Comments: (you may use HTML tags for style)

« Calipari and rendition | Main | Wolfowitz »

Search felixsalmon.com:

Felix Salmon: Recent posts

Felix's del.icio.us links

Archives

By month 2007/11 2007/10 2007/09 2007/08 2007/07 2007/06 2007/05 2007/04 2007/03 2007/02 2007/01 2006/12 2006/11 2006/10 2006/09 2006/08 2006/07 2006/06 2006/05 2006/04 2006/03 2006/02 2006/01 2005/12 2005/11 2005/10 2005/09 2005/08 2005/07 2005/06 2005/05 2005/04 2005/03 2005/02 2005/01 2004/12 2004/11 2004/10 2004/09 2004/08 2004/07 2004/06 2004/05 2004/04 2004/03 2004/02 2004/01 2003/12 2003/11 2003/10 2003/09 2003/08 2003/07 2003/06 2003/05 2003/04 2003/03 2003/02 2003/01 2002/12 2002/11 2002/10 2002/09 2002/08 2002/07 2002/06 2002/05 2002/03 2002/02 2002/01 2001/12 2001/11 2001/10 2001/09 2001/08 2001/07 2001/06 2001/05 2001/04 2001/03 2001/02 2001/01 2000/12 2000/09 2000/07 2000/03 1999/07