Showing posts with label quantitative finance. Show all posts
Showing posts with label quantitative finance. Show all posts

Thursday, June 4, 2009

The optimist in Wilmott thinks that people realize these models don't work. But he's not really an optimist.

TO BE NOTED: From Newsweek Via Marginal Revolution:

Paul Wilmott is out to save Wall Street's soul—one dork at a time.

Matthew Philips
NEWSWEEK
From the magazine issue dated Jun 8, 2009

Jude Edginton / Redux for Newsweek

Imagine an aeronautics engineer designing a state-of-the-art jumbo jet. In order for it to fly, the engineer has to rely on the same aerodynamics equation devised by physicists 150 years ago, which is based on Newton's second law of motion: force equals mass times acceleration. Problem is, the engineer can't reconcile his elegant design with the equation. The plane has too much mass and not enough force. But rather than tweak the design to fit the equation, imagine if the engineer does the opposite, and tweaks the equation to fit the design. The plane still looks awesome, and on paper, it flies. The engineer gets paid, the plane gets built, and soon thousands just like it are packed full of people and sent out onto runways. They fly for a while, but eventually, because of that fatal tweak, they all end up crashing.

In a way, this is what's happened in quantitative finance. The planes are the complex derivatives—like collateralized debt obligations—that now lie smoldering on the balance sheets of banks. The engineers are the "quants": those math and science Ph.D.s who flocked to Wall Street over the past decade and used mathematical models to build these new investment products. These are the people Warren Buffett was talking about when he said, "Beware of geeks bearing formulas" in his letter to shareholders this year. The quants aren't entirely to blame for the financial meltdown; there's plenty of guilt to be shared by regulators, top executives and the investors who bought the instruments the quants created. Yet while aeronautical engineers who willfully designed a faulty plane might be on trial for criminal negligence, Wall Street's math gurus are, for the most part, still employed. Strangely, the banks need quants more than ever right now. If anyone's going to figure out how to price these toxic assets, it's them. Quantitative finance isn't going away, but it is in desperate need of reform. And one man—a math geek himself—thinks he knows where to start.

Paul Wilmott is a 49-year-old Oxford-trained mathematician and arguably the most influential quant today, the brightest star in their insular, nerdy universe. The Financial Times calls him a "cult derivatives lecturer." Nassim Taleb, the mathematician and author of the bestseller The Black Swan, calls him the smartest quant in the world. "He's the only one who truly understands what's going on ... the only quant who uses his own head and has any sense of ethics," says Taleb. Wilmott stands atop a veritable quant empire. His wonk-made-simple textbooks sell for hundreds of dollars. A subscription to his bimonthly magazine, Wilmott, costs $695 a year. His Web site, Wilmott.com, is the nerve center of the quant community, with 65,000 registered users and a chat forum that buzzes over such topics as geodesic stochastic manifolds and swaption vol arbitrage.

Over the past decade, and increasingly since the crash, Wilmott has cultivated a loyal following of truth-seeking converts from the failed school of thought that the entire world can be turned into Greek symbols, plugged into equations, priced and predicted. He's especially critical of the notion that math can forecast human behavior, essentially the basis of finance. "This," says Wilmott, "is absolute rubbish." To rectify the situation, he's started his own training program, the Certificate in Quantitative Finance, or CQF. It's the gem of his empire, the key, he hopes, to saving the quants from themselves, not to mention all of us from their future destruction. In essence, the course is rehab for quants, a six-month, $18,000 program designed to break them of the abstract, theoretical approach to finance they learned in their Ph.D. or financial-engineering programs, and replace it with a more practical set of skills that are actually used on Wall Street. "What banks really need are quants who can translate theoretical math and tell them how it applies," says Wilmott. "Because what good is being fluent in geek if you can't apply it? You might as well stay in university."

Ever since Ed Thorp, the first true quant, left his job teaching math at UC Irvine to start a hedge fund in the 1970s, math and science types have been setting out from academia to try their hand at finance. The watershed came in the mid-1970s when MIT-trained economist Robert Merton along with Myron Scholes, a University of Chicago economist, and Fischer Black of Harvard developed the Black-Scholes equation for pricing options, which eventually garnered a Nobel Prize. Over the past 20 years, quantitative methods gradually spread into commercial and investment banks, fueling a huge demand for math savants. Since the mid-1990s, dozens of master's-degree programs in financial engineering have sprouted up at top universities. (The highest-rated ones are at Carnegie Mellon, Columbia, Stanford and Princeton.) Along with physics Ph.D. programs, these are the primary breeding grounds for the many thousands of quants who have found their way to Wall Street. It's these programs that Wilmott has taken direct aim at with his CQF. "I'm building a new army of quants," he says. His ranks currently stand at 1,273: the number of CQF alumni who have graduated since the program was founded in 2003. Wilmott realizes he's vastly outnumbered, but he's undaunted. "Ever see the movie Zulu?" he asks, referring to the 1964 Michael Caine film about a battalion of 140 British soldiers besieged by 4,000 African warriors in 1879. The Brits won, mostly because they were better-trained and -equipped, just as Wilmott wants his graduates to be.

The CQF is taught at night in London's financial district by Wilmott and a handful of his buddies, most of whom have worked in finance. These instructors are miles away from the stereotypical math or finance professors. Among them is Espen Haug, a Norwegian former JPMorgan options trader who looks like a cross between James Bond and Arnold Schwarzenegger, and often lectures in sunglasses and a tuxedo. One evening this spring, Wilmott is wrapping up the final class on interest rates and fixed income. He stands at a podium, looking out at about 30 students, most of whom have just gotten off work from their jobs at banks: Citigroup, Deutsche Bank, Credit Suisse. Another 100 or so watch the lecture streaming over the Internet. The official topic of the evening is how to decompose the random movements of the forward rate curve into its principal components—i.e., how to predict random fluctuations in interest rates in order to determine bond prices. "There's nothing difficult here," Wilmott says without irony. "If you can keep your head, you'll be fine."

He's just scribbled a handful of equations on a whiteboard, including one called the Heath-Jarrow-Morton model. Developed in the late 1980s, the formula looks horrifically complicated to the layman. But to a mathematician it's elegant, simple—and dangerous. Behind its simplicity lie hidden mistakes, unobservable variables like volatility and correlation that can provide a false sense of security. "With models, you want to see where things go wrong," says Wilmott. "You want to see the dirt. But HJM is actually just a big rug for [mistakes] to be swept under." For the next half hour, Wilmott deconstructs the thing, cautioning students on overreliance. "In the end, we should all like models that wear their faults on their sleeves," he tells the class.

Two hours later, Wilmott is treating his students to beers at a nearby pub. Among them are half a dozen fresh-faced Citigroup employees, all in their mid-20s, all recent physics or engineering Ph.D. graduates, and all being sponsored by Citigroup to take the CQF. (Banks have only recently started sponsoring students to take Wilmott's classes, a tacit acknowledgment that quants need more holistic training.) "There's an arms race going on in quantitative finance," says one of the students. "The CQF gives us another weapon in our arsenal."

On a typical April weekday, Wilmott spends the morning in his office, a small, gray room that's completely nondescript save for the giant blackboard filled with mathematical notation. There he goes over the final touches of an options formula he's getting ready to publish with a former student. He then takes the tube across town to London's posh Grosvenor Square to meet with three former investment bankers who want to hire him as a quant on call for their consultancy startup. They pitch Wilmott while sitting in the modern dining room of Gordon Ramsay's Maze Grill restaurant. Wilmott tells them he'll think about it, then sets off down Oxford Street, a stretch of shops that's swarming with tourists.

Picking his way through, on and off his cell phone, Wilmott talks about the contradictions in his life: he's a car enthusiast who hates to drive; he goes on Swiss ski trips but doesn't ski. Though he's nearly 50, he could pass for 30-something, especially when dressed in skinny jeans, black Chuck Taylors, Gucci eyeglasses and a retro zip-up cardigan. This is his outfit of choice so long as he's not giving a speech somewhere, in which case he might change his shoes and throw on a jacket.

Born in Birkenhead, a small town outside Liverpool, Wilmott studied applied math at Oxford. In his spare time, he dabbled in juggling and competitive ballroom dancing. After earning a Ph.D. in fluid mechanics from Oxford in 1985, he got his start as an applied mathematician, working on jet-engine turbines for Rolls-Royce and calculating detonation sites for an explosives company. In the late 1980s, he started applying math to finance. His first burst of fame came in 1993 when he co-wrote a textbook on derivatives. Soon he'd made a name for himself as a contrarian guru, writing more textbooks, and giving speeches around the world to rooms full of bankers. From 2001 to 2005 he ran a $170 million hedge fund that returned an average of 15 percent a year.

Back on Oxford Street, Wilmott walks by several tube stops, deep in conversation on the topic that gets him most agitated these days: structured credit, the area of finance most at fault in the crash, and where quants inflicted the most damage by applying mathematical models they swore could predict default rates. "A complete lapse of ethics and responsibility," he calls it.

A collateralized debt obligation (CDO) is the most common form of structured credit. Banks build CDOs by putting together a bunch of loans, slicing them into little pieces (tranches) and selling them off to investors. Think of it as disassembling a cow into different cuts of meat—from prime steaks to ground beef—that are priced according to their quality. The first CDO was issued in 1987 by Drexel Burnham Lambert, the same firm that went bust in 1990. After the fall of Drexel, CDOs went away for a while, until the quants came along. In 2000, the CDO market was jump-started by David X. Li, who, while working at JPMorgan, created the Gaussian copula function, a formula for determining the correlation between the default rates of different securities. In theory, the model tells you the odds that, if one CDO goes bad, others will too. The apparent genius of the Gaussian copula is its abstraction. Rather than relying on the immense amount of data used to figure the odds that a CDO might default, Li appeared to have discovered a law of correlation. That is, you didn't need the data; the correlation was just there. Armed with it, quants could price CDOs much faster, and traders could buy and sell them at record speeds. Gaussian was rocket fuel for the CDO market. The global volume of CDO deals went from $157 billion in 2004 to $520 billion in 2006. As more banks got in on the game, the once large profit margins started to shrink. In order for banks to make the same kind of returns, they had to pack more and more loans into a CDO, essentially making bigger bombs. Li was on his way to a Nobel Prize when the world blew up. Wilmott marvels at the carelessness of it all. "They built these things on false assumptions without testing them, and stuffed them full of trillions of dollars. How could anyone have thought that was a good idea?"

To Wilmott, Gaussian is an example of how dangerously abstract quant finance has become. "We need to get back to testing models rather than revering them," he says. "That's hard work, but this idea that there are these great principles governing finance and that correlations can just be plucked out of the air is totally false." Wilmott spends a lot of time with another former student trying to tackle the biggest problem facing quant finance right now: how to price all those CDOs sitting on the balance sheets of banks, the toxic assets we hear so much about. "We don't have the tools yet to truly price them," Wilmott says. "People thought we did, but they were nowhere near robust enough."

The optimist in Wilmott thinks that people realize these models don't work. But he's not really an optimist. "What I think is going to happen is that people will forget and we'll just keep going on the way we have been with nothing really changing," he says. Wilmott is encouraged by President Obama's proposals to tighten regulation of derivatives; he thinks it'll keep quants on a shorter leash. But he's also stunned by the lack of outrage over the financial mess. The violence that erupted at this year's G20 summit wasn't anywhere near what he thought it should've been. "Where the hell was everybody? If people aren't angry now, they'll never be."

Wilmott realizes he's fighting a losing battle, and that changing finance will take a lot more than a few thousand better-prepared quants. As long as banks get paid in the first year for selling a CDO that doesn't mature for 30 years, little will change. Still, he does sense a tidal shift. "I've been helped by recent events, but I don't really take solace in that. I'm not gonna say I told you so."

For now, Wilmott will go about the hard work of retraining his merry band of quant soldiers, hoping his attempts at remedial education can help minimize the odds of a future derivatives-fueled melt-down. Waiting for traffic in the shadow of St. Paul's Cathedral, having walked the three miles from Grosvenor Square, Wilmott realizes he's late for the CQF class. "My God, I'm lecturing in 10 minutes!" For investors who hope to be protected from these "financial weapons of mass destruction," it's not a moment too soon.

URL: http://www.newsweek.com/id/200015"

Saturday, April 25, 2009

“We have a saying in statistics, ‘All ­models are wrong, but some are useful.’”

TO BE NOTED: From the FT:

"
Of couples and copulas

By Sam Jones

Published: April 24 2009 22:04 | Last updated: April 24 2009 22:04

Johnny Cash and June Carter met backstage at the Grand Ole Opry. It was a little like a country song: he was married, she recently divorced, and an affair ensued; both singers had young children, and Cash would have three more with his first wife before she left him in 1966, citing his drinking and carousing. Two years later, he proposed to Carter on stage and, despite having turned him down numerous times before, she accepted. They’d each found a life match.

It ended like a country song, too. In 2003, Carter died in Nashville of complications from heart surgery, and Cash followed her to the grave four months later. The heart complication for him, it seemed, was that it was broken: “It hurts so bad,” he told the audience at the last concert he would give. The pain, he said, tuning his guitar, close to tears, was “the big one. It’s the biggest.”

Cash was speaking for many a bereaved partner – and well before Johnny ever met June, scientists had noticed that cases of spouses dying in rapid succession were not at all unusual. By the 1980s, medical researchers had started writing about “stress cardiomyopathy”, or “apical ballooning syndrome”, the ungainly name for the peculiar condition whereby an individual’s brain, following an intense emotional trauma, would inexplicably release chemicals into the bloodstream that weakened the heart – in some cases, causing it quite literally to break.

The medical community was interested because it offered a chance, potentially, to intervene and prolong life. Another industry was interested in the phenomenon, too – but less to stop it and more to understand it. These were the actuaries working in life assurance. Actuarial science is the study of the statistics surrounding life and death – and the statistics surrounding the broken heart phenomenon were striking. Pages and pages of death records showed the same marked trend: that in human couples, the death of one partner significantly increases the chances of the death of the other. Dying of a broken heart – in the most general sense, not necessarily from stress cardiomyopathy – was not a rare occurrence, but something of a statistical probability. So much so that life assurers, in order to conduct their business, needed to incorporate it into their models. In a March 2008 study, Jaap Spreeuw and Xu Wang of the Cass Business School observed that in the year following a loved one’s death, women were more than twice as likely to die than normal, and men more than six times as likely. “This implies … that joint life annuities [in which payments continue at the same price until both partners die] are underpriced while last survivor annuities [in which payments increase after one partner dies] are overpriced,” concluded the authors.

Even before the definitive Cass study, however, actuaries had begun to incorporate the broken heart trend into their mathematical models calculating the chances of clients dying. How could such an ephemeral relationship be reliably captured? The actuaries, of course, relied on probability. While they could not hope to devise a model that predicted the likelihood of death from a broken heart for a specific couple, they could use statistical science to devise a fairly accurate picture across a group of people.

They borrowed from physics and devised a formula based on something called a Markov chain: a way of expressing a series of statistical events whose outcomes are dependent upon one another. In physics, Markov chain processes underlie our most basic understanding of the world around us, from the way liquid turns to gas to the way a drop of vivid ink might slowly diffuse through a glass of water. If you treated people like atoms, the actuaries reasoned, you could apply the same maths.

. . .

In the autumn of 1987, the man who would become the world’s most influential actuary landed in Canada on a flight from China. Neither Xiang Lin Li nor the handful of fellow junior academics with whom he was travelling – all from the University of Nankai – had ever been abroad before, yet they had come at the behest of the Chinese and Canadian governments to do something most unusual: study capitalism. The small band of mathematicians and statisticians would be taking business degrees at Quebec’s Laval University.

For Li, going to Canada was just the latest in a series of unlikely opportunities that had shaped his life up to then. Decades earlier, his family had suffered at the blunt end of Mao’s cultural revolution: his father, a mid-ranking police official, was precisely the kind of lowly bureaucrat that the red guard mob was intent on re-educating, and the family was uprooted to a small village in southern China. In the countryside, the chances of young Xiang Lin going to school – let alone university – were slim. But he was talented and driven, and made it not only into school, but on to ­Nankai, one of the country’s most prestigious institutions. Li studied economics, and had just passed his master’s examinations when the Canadians came calling. Determined to be among those sent to Quebec, he learnt French in four months – as much at home studying the language, it seemed, as he was crunching numbers.

Li’s drive did not abate abroad. Four years after arriving in Canada, he’d earned his MBA; by this stage, he had no intention of returning home. In the few years he had been away, China’s mini-glasnost period had withered. Hu Yaobang, general secretary of the Communist party and a pro-democratic reformist, had been ousted, and Chinese leader Deng Xiaoping was now wary of the liberalism genie that had been let out of the bottle. In 1989, the world had looked on as students were mowed down in Tiananmen Square. Universities such as Nankai were not exactly the safest places for ambitious young students – especially not ones returning from MBA courses in the west.

As if to make clear the break, Xiang Lin changed his name and became David Li. After graduation from Laval, he enrolled at a new university, Waterloo, near Toronto. He would now be studying actuarial science. And this wasn’t the only change: the move from genteel, francophone Montreal to the more worldly and business-oriented Toronto was profound – and deliberate. According to Jie Dai, a fellow immigrant from China and a classmate at Laval, “I clearly remember [Li] mention that if you are an actuarial guy, you can earn a lot of money.”

. . .

The big money in the 1990s, of course, was not to be found at Waterloo but in Silicon Valley, on Wall Street and in the City of London. The first of these might have been an obvious destination for a talented mathematician, but the latter two were also becoming magnets for the likes of Li. In 1984, Robert Rubin, who a decade later would become US treasury secretary, made a bold decision for his employer at the time, the investment bank Goldman Sachs. Rubin decided to hire Fischer Black, an economist and academic at the Massachusetts Institute of Technology’s Sloan School of Management. Prior to 1983, a few academics had toyed with economics and markets, but as intellectual curiosities; Black was the first of his kind – a serious academic, with publications under his belt and a tenured position to boot – to make the move to Wall Street, putting theory into practice and risking the scorn of his ivory tower colleagues.

Rubin’s bet earned Goldman many multiples of Black’s salary. At the bank, the professor pioneered the use of mathematics in pursuit of money. He was one half of the duo that came up with the Black-Scholes formula, which revolutionised Wall Street by promising to determine a rational price for market risks – a principle that would become the founding doctrine of a new field, quantitative finance. Quantitative finance’s practitioners were trying to outwit the markets, using maths to eliminate risk by first using maths to calculate it. And the numbers of those practitioners grew quickly. With the collapse of the Soviet Union, the end of the arms race and, in 1993, the cancellation by the US congress of the superconducting super collider – intended to be the world’s greatest physics experiment – particle physicists, experts in quantum mechanics and computing engineers were twiddling their thumbs. For the younger generation of newly qualified grads and PhDs, applying their expertise to finance was the obvious alternative to fighting it out for the dwindling number of jobs strictly in their fields.

Emanuel Derman – a particle physicist – was such a convert. He joined Goldman in 1985, working under Black before eventually taking over from his mentor, and recalls members of the “quant” influx being referred to as “POWs” – physicists on Wall Street. Equally accurate was the acronym used by Andrew Lo, another Wall Street quant and now a lecturer at MIT. What Wall Street was really after, said Lo in a recent lecture, was not PhDs, but PSDs: people who were “poor, smart and with a deep desire to get rich”. At Waterloo, Li fitted that description to a T. He was studying for his PhD in actuarial science, but no one expected him to go on to a career in academia. Instead, in 1997, after earning his doctorate, he took a job at one of Canada’s largest banks, Canadian Imperial Bank of Commerce (CIBC).

For graduates such as Li, joining the rough and tumble world of business could be something of a shock, even when armed with MBAs. At best, the mathematicians were semi-disparagingly referred to as quants; if they were lucky, trader colleagues might slap them on the back and call them, half in flattery, “rocket scientists”. Emanuel Derman recalls a time at Goldman Sachs when he and another quant colleague were standing on the trading floor, on either side of a central aisle, and a senior trader passed between them. The trader “winced, clutching his head with both hands as though in excruciating pain, and exclaimed, ‘Aaarrggh-hhh! The force field! It’s too intense! Let me out of the way!’”

And yet by the time Li got to New York, in 1998, the quants had taken over the asylum. In the summer of that year, Long Term Capital Management, a hedge fund run by the finest minds in quantitative finance, required a massive bailout from the federal government. But far from serving as a warning that mathematical models could get investors into serious trouble, LTCM exploded the notion of quantitative finance as a geeky, back-office support task. The fund’s might before its fall – and the fact that its failure might have left a trillion-dollar hole in the financial system – discounted the notion that traders’ instinct and experience counted more than numerical intelligence.

The quants weren’t exactly out on the trading floor, however. The best of them still spent their days writing papers, crunching numbers, applying their academic expertise to the world of business. Li had come to New York to work for a consultancy called the RiskMetrics Group, which had been spun out of JP Morgan, but he was still thinking about life, death and love. In 2000, he published a paper in the prestigious Journal of Fixed Income that gained some serious attention. In it, Li performed a most elegant trick. Borrowing from his work in actuarial science and insurance and his knowledge of the broken-heart syndrome, he attempted to solve one of Wall Street quants’ most intractable problems: default correlation.

Markets do not function in laboratory-like isolation. They are linked, correlated. It isn’t enough for any quant to try and know the probability of each individual company in his bank’s portfolio going bust; he has to know how the bankruptcy of one company – or several – might increase (or decrease) the likelihood that other companies will default. Suppose, for example, that a bank loans money to two outfits – a dairy farm and a dairy. The farm, according to ratings agencies, has a 10 per cent chance of going bust and the dairy a 5 per cent chance. But if the farm does go under, the chances that the dairy will follow will rise above 5 per cent – quickly and steeply – if the farm was its main milk supplier.

And it gets more complicated from there. How correlated are the default probabilities on bonds issued by our Irish dairy farm and those issued by a software company in Malaysia? Not at all, you might think: the businesses not only provide totally different products and services, they’re also geographically remote from each other. Suppose, though, that both companies have been lent money by the same troubled bank that is now calling in its loans.

In fact, this is exactly what sank LTCM. How correlated are Russian government bonds and those in Mexico? Not at all, according to LTCM’s model, which, it should be noted, crunched data going back a hundred years. And yet it turned out for the hedge fund that both markets were dominated by the same few investors. The 1998 financial crisis in Russia, when Boris Yeltsin’s government defaulted on its bonds, caused panic selling in Mexico as investors rushed to de-risk their portfolios.

Li realised that his insight was groundbreaking. Speaking to The Wall Street Journal seven years later, he said: “Suddenly I thought that the problem I was trying to solve [as an actuary] was exactly the problem these guys were trying to solve. Default [on a loan] is like the death of a company.” And if he could apply the broken hearts maths to broken companies, he’d have a way of mathematically modelling the effect that one company’s default would have on the chance of default for others.

. . .

When mathematicians and physicists want to describe the chances of events occurring, they often rely on a curve called a copula. The Latin root is a noun meaning a “link or tie”, and indeed, copulas connect variables in such a way that their interdependence can be plotted. Throughout his PhD at Waterloo, and at CIBC, Li had been interested in how he could use copulas to develop existing actuarial models of the ­broken heart syndrome. The problem with relying on Markov chains was that they painted a far too mechanical, physical – atomic, even – picture of the human lifespan. Li reasoned that with a copula that showed a probable distribution of outcomes, a more accurate, encompassing picture of the broken heart or, for that matter, the broken company, could be devised.

He decided to use a very standard type of curve – the Gaussian copula, which is better known as a bell curve, or normal distribution – to map and determine the correlation on any given portfolio of assets. In the same way that actuaries could tell their employers the chances of Johnny Cash dying soon after June Carter without knowing anything about Cash other than the fact of his recent widowhood, so quants could tell their employers the effect one company defaulting might have on another doing the same – without knowing anything about the companies themselves. From this point on, it really could be, would be, a number-crunching game.

By 2003, Li’s paper had made his name on Wall Street. By now he was director and global head of derivatives research at Citigroup, and on a bright Tuesday morning in November, he arrived at the annual Quant Congress to bask in the glory with a presentation about his work. In front of a room of hundreds of fellow quants (“not a million miles from some kind of science fiction convention”, one person who was there recalls) he ran through his model – the Gaussian copula function for default.

The presentation was a riot of equations, mathematical lemmas, arching curves and matrices of numbers. The questions afterwards were deferential, technical. Li, it seemed, had found the final piece of a risk-management jigsaw that banks had been slowly piecing together since quants arrived on Wall Street.

. . .

By 2001, correlation was a big deal. A new fervour was gripping Wall Street – one almost as revolutionary as that which had struck when the Black-Scholes model brought about the explosion in stock options and derivatives in the early 1980s. This was structured finance, the culmination of two decades of quants on Wall Street. The basic idea was simple: that banks no longer had to hold on to risks. Instead they could value them, using complex maths and modelling, then package and trade them like any other, ordinary security.

Mortgages were the prime example. Rather than make a mortgage loan and gradually collect interest over its lifespan, banks began to bundle the loans together and sell them into specially created off-balance-sheet shell companies. These companies in turn issued bonds to raise cash. And by using the modelling and maths being cranked out by quants, banks were able to tailor the structure of mortgage portfolios to ensure that bonds of varying risks could be issued to investors. The problem, however, was correlation. The one thing any off-balance-sheet securitisation could not properly capture was the interrelatedness of all the ­hundreds of thousands of different mortgage loans they owned. As a consequence, structured finance had remained a niche and highly bespoke practice throughout the 1990s.

On August 10 2004, however, the rating agency Moody’s incorporated Li’s Gaussian copula default function formula into its rating methodology for collateralised debt obligations, the structured finance instruments that subsequently proved the nemesis of so many banks. Previously, Moody’s had insisted that CDOs meet a diversity score – that is, that each should contain different types of assets, such as commercial mortgages, student loans and credit card debts, as well as the popular subprime debt. This was standard investing good practice, where the best way to guard against risk is to avoid putting all your eggs in one basket. But Li’s formula meant Moody’s now had a model that enabled it to gauge the interrelatedness of risks – and that traditional good practice could be thrown out of the window, since risk could be measured with mathematical certainty. No need to spread your eggs across baskets if you knew the exact odds of your one basket being dropped. A week after Moody’s, the world’s other large rating agency, Standard & Poor’s, changed its methodology, too.

CDOs built solely out of subprime mortgage debt became the rage. And using the magic of the Gaussian copula correlation model, and some clever off-balance-sheet architecture, high-risk mortgages were re-packaged into triple-A-rated investor gold. The CDO market exploded. In 2000, the total number of CDOs issued were worth somewhere in the tens of billions of dollars. By 2007, two trillion dollars of CDO bonds had been issued. And with so many investors looking to put their money in debt, that debt became incredibly cheap, fuelling a massive boom in house prices and turbo-charging the world’s economies.

. . .

The unwinding started, as we all now know, in the US subprime housing market. Defaults started to increase in late 2006. The banks weren’t worried at first. Their models assumed that the pinprick default points all over the US were not correlated. But the defaults kept coming. By early 2007, it was clear that the US subprime market had a problem and by that summer, homeowners all over the US were defaulting on their mortgages. The cheap debt made available by the finance revolution was so cheap, in fact, that the loans should never have been made. And the correlation model was still mapping the housing market as it had been in 1990s, not the grossly inflated monster it had become. The development of the model had, ironically, changed the nature of the reality it was modelling.

The losses the banks began to take against their holdings of CDOs were staggering. And as the institutions grew fearful about one another’s solvency, they stopped lending to each other. Global liquidity dried up. The rot spread from asset class to asset class, and the banks’ pain spread to the real economy. Suddenly, everything was highly correlated.

How had Li’s formula failed to anticipate this? The problem was that it assumed events tended to cluster heavily around an average – a “normal” state. In actuarial science, Li’s formula could adequately capture binary outcomes such as life or death, but in the messy world of mortgages and economics, it faltered. The range of possible outcomes here was more complicated, and indeed, random, than those facing an insurance company’s clients. Markets – particularly the mortgage market – were far more prone to extreme correlation scenarios than were insurers. Death from a broken heart, for all its poetic associations, is far easier to predict than the more prosaic, but ultimately unknowable, interrelatedness of markets.

Why did no one notice the formula’s Achilles heel? Some did. Nassim Nicholas Taleb, author of the bestselling The Black Swan – a book about the importance of considering outliers when looking at copulas – was a voluble critic of quantitative finance and Li’s formula. “The thing never worked,” he says. “Anything that relies on correlation is charlatanism.”

In 2007, David Li left Wall Street and moved back to China. He could not be contacted for this story. But two years earlier, before the financial system blew up, he did warn: “Very few people understand the essence of the model.” Harry Panjer, a professor of statistics and actuarial science who was Li’s mentor at Waterloo, strikes a balance between Taleb’s accusations and the stance of Li. Earlier this year, Panjer told The Toronto Star newspaper: “We have a saying in statistics, ‘All ­models are wrong, but some are useful.’” And David Li’s model was, for a period, profoundly useful.

Sam Jones is a reporter for FT Alphaville."