Can Ratings Agencies be Fixed?

Stanford’s Darrell

Duffie is giving a series

of lectures at Princeton on capital immobility, which will be turned into

a book to be published by Princeton University Press. The Press invited me to

dinner with Duffie last night – delicious, thanks! – and we

got to talking about ratings agencies. Duffie is a member of Moody’s Academic

Research Committee, and his take on the ratings

crisis is basically that there’s nothing there that better modelling and

clearer thinking can’t fix.

What’s the problem, he says, with Moody’s rating structured products like the

notorious CPDOs?

Not that Moody’s got the rating wrong, so much as that the product

should never have been considered a ratable instrument in the first place. Is

there a conflict, he asks, inherent in the fact that issuers, and not investors,

pay the ratings agencies? No: the ratings agencies’ credibility is much more

important to them than any benefit they might get from boosting the ratings

they give, and in any case it’s relative ratings which matter, not

absolute ones.

I wasn’t convinced. For one thing, the ratings agencies clearly and consistently

said that ratings are horizontally comparable, as it were: that a double-A rating

on a sovereign means it has the same default risk as a double-A rated municipal

bond, or corporate bond, or structured product. And it turns out that’s not

the case: structured products, especially, default much more than identically-rated

munis, and they always have – this is not news. It certainly seems as

though the ratings agencies, which made enormous profits from rating structured

products, were "nicer" to those products than they were to other issuers.

But my real reason for skepticism is that I’m in the middle of being extremely

impressed by another Princeton University Press author, Riccardo Rebonato. In

his new

book, Rebonato compellingly skewers the "frequentist" approach

to probabilities employed not only by Moody’s and the other ratings agencies

but more generally across the whole world of finance. Looking backwards at what

happened in the past gives you lots of data, which can be chopped up and examined

in any number of different ways, gives a false sense that future probabilities

can be scientifically determined to three or four significant figures.

But that whole approach is based on the idea that market moves are like coin

flips, or balls in an urn: that the markets might move, but that their

underlying structure never really changes. In fact, the defaults rates that

matter are the ones in the future, not the ones in the past. And to get a grip

on those you need to understand what Rebonato calls "subjective probability"

– which, although it might not have quite the mathematical rigor of frequentist

probability, can actually be much more useful and accurate.

Duffie does understand this. Moody’s had a habit, he says, of tweaking incredibly

complicated models until they matched the past data, and then deciding that

because the models matched the past data, they must give a good idea of what

will happen in the future. Obviously, that’s never going to work very well:

it’s the financial equivalent of shooting an arrow at a barn door, painting

a target around it, and claiming astonishing marksmanship. But I think Duffie

and I differ on the question of whether better models can fix this problem.

He thinks they can; I think they can’t, certainly so long as Bayesian statistics

remains a backwater for Wall Street’s quants.

This entry was posted in bonds and loans. Bookmark the permalink.