The Undoing Project

It was a clear day in San Fransisco. Bay area rivals faced off in game 3 of the 1989 Baseball World Series pitting the Athletics against the Giants.

ABC had just begun its pregame analysis with Al Michaels and Tim McCarver when cries of fear and terror erupted throughout Candlestick Park. What is now known as the World Series quake rocked the coast causing extensive damage and resulting in almost 4,000 injuries and 63 deaths. The Athletics went on to win the Series. But the aftershocks to the baseball world were just beginning.

Athletic's owner Walter Haas, Jr had spared no expense in acquiring the best talent available to attain his ultimate goal. By 1990, the Athletics payroll was the highest of the 30 teams in both leagues. Sadly, in 1995 Haas died as a result of prostate cancer. A violent aftershock was about to rock the Athletics too.

The A's new owners did not share Mr. Haas's baseball enthusiasm so they dismantled the team roster in an effort to drastically cut costs with a resulting decline in performance. Then Athletic's General Manager Sandy Alderson tasked his young scouting assistant, Billy Beane, to apply some sabermetrics in his scouting reports with an eye toward obtaining undervalued players because of the shear lack of funds. Some may be unfamiliar with the term sabermetrics. Simply, it is empirical analysis of baseball, focusing on statistics that measure in-game activity. The name is derived from the Society of American Baseball Research founded in 1971.

Of course baseball box scores have been around since 1858, but Beane saw some statistical variables that had not been taken into account as he began his quest to get the biggest bang with the smallest buck. With help from Paul dePodesta, a Harvard educated statistician, they changed the trajectory of baseball.

For example, in 2001 the New York Yankees, with a player payroll of $109 million bested the Athletics in the Division Series. However, the A' s only spent $33 million because of Beane's ability to spot under valued top quality players using the sabermetric analysis which includes subtleties of batting,pitching and fielding. Author Michael Lewis highlighted these statistical phenoms in his New York best seller, Moneyball, The Art of Winning an Unfair Game, 2003. It was turned into a mega hit movie with Brad Pitt playing Beane. And the story might end there but for a question that kept nagging Lewis. If so called highly paid professionals in the baseball world who had been using the same strategy to select players since the 1860s had so misunderstood their market, what other industries might be making the same mistake? Therein lies the premise for Lewis's latest novel, The Undoing Game, A Friendship that Changed Our Minds, December, 2016.

At the outset it must be noted that this is not a book on statistics,or economics or even psychology. Rather it is an amalgam of all three with the friendship of two men as the covalent bond.

When the Moneyball novel came out in 2003, a critique cowritten by economist Richard Thaler and lawyer Cass Sunstein from the University of Chicago wondered why Lewis did not delve into the deeper inefficiencies which surreptitiously sprang forth from the human mind such as the ways a baseball expert might misjudge a baseball player or the way any expert's judgements might be skewed by his own mind. The critics referenced a decades old study by two Israeli psychologists that made Lewis realize he had not broken any new ground at all but simply illustrated what had been out there in the public domain for years much like the eyes of an alligator represent but a minuscule part of an incredible power lurking just beneath the surface.

Lewis had not considered the psychological aspects of the decision making process in Moneyball although he was quite aware of the biases that were exposed in the inefficient ways the experts did their jobs. He decided to investigate how the mind worked or didn't work in forming judgements and making decisions. When faced with uncertainty in baseball or any other endeavor, how do we come to a conclusion? What are people doing, including experts, that lead them to misjudgments that could be exploited by others who ignored the experts and relied on data.

The Living Corpse

Danny Khaneman knew exactly what he wanted to be at an early age, an intellectual. As a Parisian child in 1941, he did not have any physical prowess at all. the school boys teased him with the nickname, "the living corpse". Instead he was bookish and precocious with little time for classmates. His maturity may in part have been due to the Nazi occupation that weighed heavily on many Jewish families at the time.

Danny occupied his time by reading novels like Around the World in Eighty Days and listening and watching those around him. His father died in 1944 due to complications of diabetes to which no medical help was available. He was not allowed to attend his father's funeral because it considered to be too dangerous. The remaining family abandoned Europe and resettled in the more hospitable (or so they thought) environs of Jerusalem in 1947. Of course in May of 1948, Israel declared its independence and Danny, 14, was thrust into yet another battlefield. These experiences formed the basis for what shaped the man who was to become one of the foremost psychologist of our time and the ultimate devotee of human error.

Danny excelled at Hebrew University, not by instruction as much as his voracious appetite to learn. At the time B F Skinner's behavioralism was the predominant school of thought. Danny wished to pursue the objectivity of the Gestalt school of psychology which postulated that the mind can override external stimuli to arrive at an incorrect conclusion.

Upon graduation, Danny joined the Israeli Army as is required under law. As you can imagine he was not well suited to Army life but none the less placed in the psychological unit, and at age 20, was the Israeli Defense Forces expert in psychology. The IDF was ramping up its forces with over 700,000 recruits arriving to defend Israel. But up to this point, Israel, which looked more like a fort than a nation, had a poorly sewn together bunch of misfits unaware of their individual strengths or weaknesses.

Interviewers of the recruits fell victim to the "halo effect" which assigned unwarranted credit based on an initial positive assessment, say for appearance.

Danny was assigned the task to sort this menagerie out. He gave the interviewers questionnaires for the recruit to complete and thus Danny designed a specific questionnaire that was less about fact seeking and more designed to disguise the facts being sought about the recruit. Rather than the interviewer go with a gut instinct, remove it, and the interviewers judgments improved dramatically.

At 21 years of age this young psychologist revolutionized the IDF to secure a nation's future. The Kahnamen's Score as it became known proved so successful that the Israeli Army continues to use it today. The US adopted the KS in 1983.

After his stint in the military, Danny began teaching at Hebrew U. He was outstanding. He was a bold genius, weaving statistics courses into the meaning of life. His teachings on everything from enhancing fighter pilot's performance to rabbi's interpretation of the colors of the Talmud gave insights into his perception class that no one had ever heard or seen, all done without notes or syllabi.

However, this genius came with the price of constant insecurity and doubt, not only about himself but the world around him. To calm his nerves he was known to smoke a couple of packs of cigarettes a day. But this couldn't slow his pursuit for answering life's questions. He had a relentless way of converting insight into application, freely flowing from one idea to the next uninhibited by conventional perspectives.

The New Spartan

In 1957 Hebrew University announced it was opening a psychology department. A pale, young fellow named Amos Tversky, a quintessential Israeli, still in his paratrooper's uniform, was one of only 20 accepted under this new program. A few month's before he had won the army's highest award for bravery. However, he seldom mentioned his exploits.

Amos's family had fled the Russian Anti-semitism of the early twenties and were some of the early pioneers of the Zionist nation. His mother excelled as a masterful politician, while his father, a veterinarian, could weave a wondrous tale of his life's experiences. Amos loved that trait and perfected it for himself.

While small in stature, he was quick, brave and agile. He spoke with almost a Catalan lisp, but more often than not, when he spun a tale he would find the high ground, whether a desk or tank and commanded the audience's attention.

He was highly gifted in math and science, but instead pursued the humanities in college rationalizing that he could teach himself numbers and nuclei anytime. While arguably one of the most popular in school , he met and fell in love with a quiet loner named Dahlia, whose father had been killed in the war. They would marry. They both loved poetry. She later would win Israel's top literary prize and become an international figure in her own right. She thought Amos was the smartest person she had ever met.

So did almost everyone else. Of course that lends itself to somewhat quirky behavior. He maintained vampire hours, going to bed at dawn and waking up at happy hour. He had the preternatural gift of doing exactly what he wished to do without regard to social norms.

Amos finished at the top of his class academically and socially. He pursued a graduate degree in the United States wanting to explore how people made decisions.

If people make choices by comparing some ideal in their head and the real world versions, you have to know how people made such judgments.

Similarity judgments typically ask how closely did the two competing objects or ideas or people have to one another.

Amos began to challenge this notion of similarity judgments, resulting in a theory more akin to "features of similarity". He explained that when two things are compared and judged for the similarity, the person was essentially making a list of features. The more the noticeable the features, the more likely there is an apparent similarity. And if the noticeability of the features can be manipulated by the way they are or are not highlighted, the sense of how similar two things are can be manipulated as well. He had turned conventional wisdom on its head.

Upon his return to Israel in 1966 Amos "settled into teaching at Hebrew University. His new wife Barbara Gans had never been out of the United States, she had never been in a war. That soon changed.

Every man was in the Israeli Army, even the professors. In May 1967, Amos went off to fight again after Egypt closed the Straits of Tiran to Israeli ships, the primary trade route for the country. The war started on a Monday and by Friday it was over.

Barbara spent most of her time in a bomb shelter sewing sand bags. Amos camped out at King Hussein's summer palace and briefly was the military governor of Jericho. But, at war's end the professors simply returned to teaching without ceremony or much fanfare.

Unconventional Friends

These two scholars began to meet to discuss the two sides of their respective fields of psychology. They would usually meet alone, behind closed doors for hours on end interchanging their discussions between English and Hebrew.

Almost immediately this unlikely pair became soul mates. Danny the forever pessimist, Amos the eternal optimist. Amos the life of the party. Danny wouldn't go to the party. Amos was the definition of informality while Danny appeared to be the mortician's apprentice in his relationships. He took everything so seriously while Amos made everything into a joke.

Danny was short tempered, but eager to please. On the other hand Amos couldn't figure out why you would want to please anyone. The most striking difference may have been how each office looked.

Danny's was the absent minded professor look with papers, books and cigarette butts everywhere while Amos had nothing, literally nothing in his office but a pencil and an unused writing pad.

However they did share a few things in common. While neither believed in God, both were grandsons of Eastern European rabbis. Both were explicitly interested in how people functioned. Both wanted to search for simple yet powerful truths. And both were blessed with amazingly fertile minds.

The two decided to collaborate on a paper to which would challenge Bayes theory that whenever presented with a problem there would always be a statistically correct answer to support it. The best part of the exercise was that both men had fun doing it. They joked and laughed and made the project seem effortless. Their first paper was called "Belief in the Law of Small Numbers", 1970, and in essence it proffered that people mistook even a very small part of a thing, as whole. Even statisticians made the leap to conclusions from inconclusively small amounts of evidence.

Another profound work called "Subjective Probability: A Judgment of Representativeness" led Danny and Amos to coin the term "heuristics".

The two were investigating how people assign odds to any given situation when one is more or less guessing. They offered a substitute from rules of chance to rules of thumb, or heuristics. For example, if a 6'8' man walked down the street, someone might conclude that odds are likely that he is a NBA basketball player because his hight is representative of NBA basketball players, even though the odds are really low. The two met with international success and is so often the story, that became their undoing.

Lewis has an uncanny ability to not only put flesh to bones but one can actually see the heart beat of these two unlikely friends as the twoevolve into something greater than themselves. Their work represents the best research of social science of the past century. It exposed the folly of a rational mind in the decision making process. Whether in the field of medicine or eating habits or even retirement savings, these two psychologists revolutionized how things are done. But as Kahnamen explained, this information is not dictated by numbers alone. "No one ever made a decision because of a number - they need a story." Nor is it so unfeeling as to eliminate the core emotions we rely on to relate to one another. Amos explained." Because stinginess and generosity are both contagious, and because behaving generously makes you happier, surround yourself with generous people."

The 350 or so page book will sometimes baffle the reader with the plethora of test examples given to not only grad students but economists, doctors, politicians and even the CIA. Most subjects usually reaching the wrong conclusion in a most humorous way. Here are but a handful of tests to whet your appetite.

Memory: Consider the letter K

A typical text was selected and the relative frequency with which various letters of the alphabet appeared in the first and and third positions of the words was recorded. Words of less than three letters were excluded from the count.

You will be given several letters of the alphabet, and you will be asked to judge whether these letters appear more often and the first or in the third position, and to estimate the ratio of the frequency with which they appear in these positions...

Consider the K. Is K more likely to appear in ____ the first position

____ the third position

My estimate for the ratio of these two values is: ____ to 1

If you thought that K was, say, twice as likely to appear as the first letter of an English word then as the third letter, you checked the first box and wrote your estimate as 2:1. This was what the typical person did, as it happens: Danny and Amos replicated the demonstration with other letters ---R, L, N and V. Those letters also appeared more frequently as the third letter in an English word then as the first letter–by a ratio of two to one. Once again peoples judgment was, systematically, very wrong. And it was wrong, Danny and Amos now proposed, because it was distorted by memory. It was simply easier to recall words that start with K then to recall words with K as a third letter. The more easily people can call some scenario to mind –the more available it is to them –the more probable they find it to be.

The Anchor

It is not just that people don't know what they don't know, but that they don't bother to factor their ignorance into their judgments. Another possible heuristic they called "anchoring". They first dramatize its effects by giving a bunch of high school students five seconds to guess the answer to a math question.

The first group was asked to estimate this product: 8x7x6x5x4x3x2x1

A second group to estimate this product: 1x2x3x4x5x6x7x8

Five seconds wasn't long enough to actually do the math: the kids had to guess. The two groups answers should have been at least roughly the same, but they weren't, even roughly. The first groups median answer was 2250. The second group's median answer was 512. (the right answer is 40,320). The reason the kids in the first group guessed a higher number for the first sequence was that they had used 8 as the starting point, while the kids in the second group had used 1. It was almost too easy to dramatize this weird trick of the mind. People could be anchored with information that was totally irrelevant to the problem they were being asked to solve.