book review

Christian and Griffiths’s Algorithms to Live By: The Computer Science of Human Decisions

christianIn a sea of books describing a competition between perfectly rational decision makers and biased humans who make systematic errors in the way they decide, Brian Christian and Tom Griffiths’s Algorithms to Live By: The Computer Science of Human Decisions provides a nice contrast.

Christian and Griffiths’s decision-making benchmarks are the algorithms developed by mathematicians, computer scientists and their friends. In that world, decision making under uncertainty involves major trade-offs between efficiency, accuracy and the types of errors you are willing to accept. As they write:

The solutions to everyday problems that come from computer science tell a different story about the human mind. Life is full of problems that are, quite simply, hard. And the mistakes made by people often say more about the intrinsic difficulties of the problem than about the fallibility of human brains. Thinking algorithmically about the world, learning about the fundamental structures of the problems we face and about the properties of their solutions, can help us see how good we actually are, and better understand the errors that we make.

Even where perfect algorithms haven’t been found, however, the battle between generations of computer scientists and the most intractable real-world problems has yielded a series of insights. These hard-won precepts are at odds with our intuitions about rationality, and they don’t sound anything like the narrow prescriptions of a mathematician trying to force the world into clean, formal lines. They say: Don’t always consider all your options. Don’t necessarily go for the outcome that seems best every time. Make a mess on occasion. Travel light. Let things wait. Trust your instincts and don’t think too long. Relax. Toss a coin. Forgive, but don’t forget. To thine own self be true.

And as they close:

The intuitive standard for rational decision-making is carefully considering all available options and taking the best one. At first glance, computers look like the paragons of this approach, grinding their way through complex computations for as long as it takes to get perfect answers. But as we’ve seen, that is an outdated picture of what computers do: it’s a luxury afforded by an easy problem. In the hard cases, the best algorithms are all about doing what makes the most sense in the least amount of time, which by no means involves giving careful consideration to every factor and pursuing every computation to the end. Life is just too complicated for that.

Here’s a few examples.

Suppose you face a choice between two uncertain options. Those options have an expected value – the most likely result. If your objective is to maximise your outcome, you pick the option with the highest expected value.

But what if you objective is to minimise regret – the feeling of pain when you look back at what you did compared to what you could have done? In that case it may be worth looking at the confidence intervals around that expected value – the plausible ranges in which the actual value could lie. Picking the option which has the highest upper confidence interval – the highest plausible value – is the rational approach, even if it has the lower expected value. It is “optimism” in the way a behavioural scientist might frame it, but for an objective of minimising regret, it is rational.

 

Or consider memory. From a computer science perspective, memory is often not a question of storage but of organisation – particularly in today’s world of cheap storage. How does a computer predict which items it will want from its memory in the future such that they are accessible within a reasonable time? Faced with that problem, it makes sense to forget things. In particular, it is often useful to forget things with time – those items least recently used. The human mind mimics this strategy, as more recently used items are more likely to be used in the future. It is too expensive to maintain access to an unbounded number of items.

One chapter of the book covers the idea of “less is more”, which you may be familiar if you know the work of Gerd Gigerenzer and friends. The idea behind “less is more” it that it is often rational to ignore information in making decisions to prevent “overfitting”. Overfitting is an over-sensitivity to the observed data in developing a model. The inclusion of every detail helps the model match the observed data, but prevents generalisation to new situations and predictions based on new data lack reliability.

To avoid overfitting you might deliberately exclude certain factors, impose penalties for including factors in analysis, or stop the analysis early. There is no shortage of computer science or machine learning applications where these types of strategies, often employed by humans in their decision making, can result in better decisions.

Christian and Griffiths suggest that evolution tends not to overfit as it is constrained by existing infrastructure and time – features of the environment need some degree of persistence before adaptations to that environment spread, preventing major changes in response to short-term phenomena. Preventing overfitting is also a benefit of a conservative bias in society – preventing us getting caught up in the boom and bust of fads.

There are times in the book where Christian and Griffiths jump too far from experiment or algorithm to real world application. As an example, they suggest that analysis of a routing tells us not to try to control traffic congestion using a top down coordinator, as the selfish solution is only 33% worse than best case top down coordination. They give little thought to whether congestion has more dimensions of control than just routing. The prisoner’s dilemma chapter also seemed shallow at points – possibly reflecting that it is the area for which I already had the most understanding.

But those are small niggles about an otherwise excellent book.

Best books I read in 2016

The best books I read in 2016 – generally released in other years – are below (in no particular order). For the non-fiction books, the links lead to my reviews.

henrichJoe Henrich’s The Secret of Our Success: How Culture Is Driving Human Evolution, Domesticating Our Species, and Making Us Smarter (2015): A lot of interesting ideas, but left me with a lot of questions.
jonesGarett Jones’s Hive Mind: How Your Nation’s IQ Matters So Much More Than Your Own (2015): A fantastic exposition of some important but neglected features of the world.
rosenzweigPhil Rosenzweig’s Left Brain, Right Stuff: How Leaders Make Winning Decisions (2014): An entertaining examination of how behavioural economics findings hold up for real world decision making.
EPJPhilip Tetlock’s Expert Political Judgment: How Good Is It? How Can We Know? (2006): I re-read this before reading Tetlock’s also good Superforecasting, but Expert Political Judgement is the superior of the two books.
lastJonathan Last’s What to Expect When No One’s Expecting: America’s Coming Demographic Disaster (2014): Much to disagree or argue with, but entertaining and a lot to like.
Other Peoples MoneyJohn Kay’s Other People’s Money (2015): A fantastic critique of the modern financial system and regulation.
dostoevskyFyodor Dostoevsky’s Crime and Punishment: The classic I most enjoyed.

Levine’s Is Behavioural Economics Doomed?

levineDavid Levine’s Is Behavioural Economics Doomed? is a good but slightly frustrating read. I agree with Levine’s central argument that rationality is underweighted in many applications of behavioural economics, and he provides many good examples of the power of traditional economic thinking. For someone unfamiliar with game theory, this book is in some ways a good introduction (or more particularly, to the concept of Nash equilibrium). And for some of the points, Levine shows a richness in the literature that you don’t often hear about if you only consume pop behavioural economics books.

But the book is also littered with straw man arguments. Levine often gives views to behavioural economists which I am not sure they generally hold, and he often picks strange examples. And when it comes to explaining away behaviour that doesn’t fit so neatly with the rational actor model, Levine is not always convincing.

As an example, Levine provides an overview of the prisoner’s dilemma, a classic game demonstrating why two people might not cooperate, even though cooperation leads to a better outcome than both players defecting. Levine uses it to argue against those who suffer from the fallacy of composition and who wonder why we can have war, crime and poverty if people are so rational. But who are these people that Levine is arguing against? I presume not the majority of the behavioural economics profession who are more than familiar with the prisoner’s dilemma game.

Levine’s introduction to the prisoner’s dilemma is good when he discusses what happens with different strategies or game designs. But when it comes to the players in experiments who don’t conform to the Nash equilibrium – such as those who don’t defect in every period if there is a defined end to the game – he hand waves away their play as “rational and altruistic” rather than seriously exploring whether they made systematic errors.

Similarly, when discussing the ultimatum game, Levine simply describes the failure to maximise income as “modest”. He does make the important point that it is rational for first movers to offer more than the minimum if there is a possibility of rejection (and since they don’t have opportunity to learn, they will get this wrong sometimes). But he seems less concerned about the behaviour of player 2 who rejects a material sum. Yes it might be a Nash equilibrium, but the behavioural view might shed some light on why we end up at that particular Nash equilibrium.

Levine is similarly dismissive of the situations where people do make errors in markets. “Behavioural economics focuses on the irrationality of a few people or with people faced with extraordinary circumstances. Given time economists expect these same people will rationally adjust their behaviour to account for new understandings of reality and not simply repeat the same mistakes over and over again.” But given how many major decisions are one-shot decisions with major consequences (purchasing cars, retirement decisions etc), surely they are worth exploring.

One of the more bizarre examples is where Levine addresses the question of why people vote despite having almost no chance of changing the outcome. Levine gives an example of a voting participation game conducted in the lab where he found that participants acted according to the predicted Nash equilibrium, reflecting their costs of voting, the benefits of winning and the probability of their vote swinging the result. But he doesn’t then grapple with the clear problem that this limited experiment doesn’t translate to the real world. Funnily enough, only pages later he cautions “[B]eware also of social scientist [sic] bearing only laboratory results.”

Levine also  brings out the now classic question of why couldn’t economics predict the economic crisis. He points out that crises must be inherently unpredictable as there is an inherent connection between the forecaster and the forecast. If a model that people believed predicted a collapse in the market of 20% next week, the crash would happen today (Let’s ignore for the moment that there seems to be an economist predicting a crash almost every day).

In defence of the economists, Levine pulls out a series of (well cited) papers that he believes already explained the crisis, such as providing for the possibility of sharp crashes and the effect of fools in the market. Look, the shape of the curve by this random paper is the same! But was that actually what happened? Was that the dominant theory? Levine seems to believe mere existence of literature in which crises are present is an indication that the profession is fine, even if that wasn’t a dominant or even widely believed model.

Having spend most this post complaining about Levine’s angle of attack, there are many good points. His discussion of learning theory is interesting – people don’t know all information before they undertake an action and learn along the way. Selfish rationality with imperfect learning does a pretty good job of explaining much behaviour. Some of this throwaway lines also make important points. For example, if a task is unpleasant, it can be rational to leave it to the last moment. Uncertainty can make the procrastination even more rational.

Some of Levine’s critiques of the experimental evidence are also interesting. One I was not aware of was whether the appearance of the endowment effect in some experiments was due to people misunderstanding the Becker-DeGreeot-Marschak elicitation procedure. (People state their willingness to pay or accept and a random draw of the price is made. If the price is lower than the willingness to pay, they pay it.) Levine points to experiments where, if people are trained to understand the procedure, the endowment effect disappears. As I mentioned in a previous post, Levine also points to some interesting literature on anchoring.

Levine closes with a quote from Loewenstein and Ubel that is worth repeating:

… [behavioral economics] has its limits. As policymakers use it to devise programs, it’s becoming clear that behavioral economics is being asked to solve problems it wasn’t meant to address. Indeed, it seems in some cases that behavioral economics is being used as a political expedient, allowing policymakers to avoid painful but more effective solutions rooted in traditional economics.

Behavioral economics should complement, not substitute for, more substantive economic interventions. If traditional economics suggests that we should have a larger price difference between sugar-free and sugared drinks, behavioral economics could suggest whether consumers would respond better to a subsidy on unsweetened drinks or a tax on sugary drinks.

But that’s the most it can do.

Underneath Levine’s critique you sense this is what is really bugging him. Despite the critiques, traditional economic approaches still have a lot of power. And for some people, that seems to have been forgotten along the way.

Saint-Paul’s The Tyranny of Utility: Behavioral Social Science and the Rise of Paternalism

Saint-PaulThe growth in behavioural science has given a new foundation for paternalistic government interventions. Governments now try to help “biased” humans make better decisions – from nudging them to pay their taxes on time, to constraining the size of the soda they can buy, to making them save for that retirement so far in the future.

There is no shortage of critics of these interventions. Are people actually biased? Do these interventions change behaviour or improve outcomes for the better? Is an also biased government the right agent to fix these problems? Ultimately, do the costs outweigh the benefits of government action?

In The Tyranny of Utility: Behavioral Social Science and the Rise of Paternalism, Gilles Saint-Paul points out the danger in this line of defence. By fighting the utilitarian battle based on costs and benefits, there will almost certainly be circumstances in which the scientific evidence on human behaviour and the effect of the interventions will point in the freedom-reducing direction. Arguing about whether a certain behaviour is rational at best leads to an empirical debate. Similarly, arguments about the irrationality of government can be countered by empirical debate on how particular government interventions change behaviour and outcomes.

As a result, Saint-Paul argues that:

[I]f we want to provide intellectual foundations for limited governments, we cannot do it merely on the basis of instrumental arguments. Instead, we need a system of values that delivers those limits and such a system cannot be utilitarian.

Saint-Paul argues that part of the problem is that the utilitarian approach is the backbone of neoclassical economics – once (and still in some respects) a major source of arguments in favour of freedom. Now that the assumptions about human behaviour underpinning many neoclassical models are seen to no longer hold, you are still left with utility maximisation as the policy objective. As Saint-Paul writes:

It should be emphasized that the drift toward paternalism is entirely consistent with the research program of traditional economics, which supposes that policies should be advocated on the basis of a consequentialist cost-benefit analysis, using some appropriate social welfare function. Paternalism then derives naturally from these premises, by simply adding empirical knowledge about how people actually behave …

When Saint-Paul describes the practical costs of this increased paternalism, his choice of examples often make it hard to share his anger. One of his prime cases of infringed liberty is a five-times public transport molester who is banned from using the train as a court determined he lacked the self-control to travelling on it. On gun control laws he suggests authoritarian governments could rise in the absence of an armed citizenry.

Still, some of the other stories (or even these more extreme examples) lead to an important point. Saint-Paul points out that many of these interventions extend beyond the initial cause of the problem and impose responsibility on people for the failings of others. For example, in many countries you need a pool fence even if don’t have kids. You effectively need to look after other people’s children. Similarly, liquor laws can extend to preventing sales to people who are drunk or likely to drive. Where does the chain of responsibility transfer stop?

One of the more interesting threads in the book concerns what the objective of policy is. Is it consumption? Or happiness? And based on this objective, how far does the utilitarian argument extend. If it is happiness, should we just load everyone up with Prozac? And then what of the flow on costs if everyone decides to check out and be happy?

What if a cardiologist decides that experts and studies are right, that it’s stupid after all to buy a glossy Lamborghini, and dumps a few of his patients in order to take more time off with his family? How is the well-being of the patients affected? What if that entrepreneur who works seventy hours a week to gain market shares calls it a day and closes his factory? In a market society the pursuit of status and material achievement is obtained through voluntary exchange, and must thus benefit somebody else. Owning a Lamborghini is futile, but curing a heart disease is not. The cardiologist may be selfish and alienated; he makes his neighbors feel bad; and he is tired of the Lamborghini. His foolishness, however, has improved the lives of many people, even by the standards of happiness researchers. Competition to achieve status may be unpleasant to my future incarnations and those of my neighbors, but it increases the welfare of those who buy the goods I am producing to achieve this goal.

Saint-Paul’s response to these problems – presented more as suggestions than a manifesto, and thinly summarised in only two pages  at the end of the book – is not to ignore science but to set some limits:

I am not advocating that scientific evidence should be disregarded in the decision-making process. That is obviously a recipe for poor outcomes. Instead, I am pointing out that the increased power and reliability of Science makes it all the more important that strict limits define what is an acceptable government intervention and that it is socially accepted that policies which trespass those limits cannot be implemented regardless of their alleged beneficial outcomes. We are going in the opposite direction from such discipline.

These limits could involve a minimal redistributive state to rule out absolute poverty – allowing some values to supersede freedom – but these values would not include “statistical notions of public health or aggregate happiness”, nor most forms of strong paternalism.

But despite pointing to the dangers of utilitarian arguments against paternalistic interventions, Saint-Paul finds them hard to resist. He regularly refers the biases of government, noting the irony that “the government could well offset such deficiencies with its own policy tools but soon chose not to by having high public deficits and low interest rates.” And when it comes to his picture of his preferred world it has a utilitarian flavour itself.

Being treated by society as responsible and unitary goes a long way toward eliciting responsible and unitary behavior. The incentives to solve my own behavioral problems are much larger if I expect society to hold me responsible for the consequences of my actions.

Ariely’s The Honest Truth About Dishonesty

ArielyI rate the third of Dan Ariely’s books, The Honest Truth About Dishonesty: How We Lie to Everyone – Especially Ourselves, somewhere between his first two books.

One of the strengths of Ariely’s books is that he is largely writing about his own experiments, and not simply scraping through the same barrel as every other pop behavioural science author. The Honest Truth has a smaller back catalogue of experiments to draw from than Predictably Irrational, so it sometimes meanders in the same way as The Upside of Irrationality. But the thread that ties The Honest Truth together – how and why we cheat – and Ariely’s investigations into it gave those extended riffs more substance than the story telling that filled some parts of The Upside.

The basic story of the book is that we like to see ourselves as honest, but are quite willing and able to indulge in a small amount of cheating where we can rationalise it. This amount of cheating is quite flexible based on situational factors, such as what other people are doing, and is not purely the result of a cost-benefit calculation.

The experiment that crops up again and again through the book is a task to find numbers in a series of matrices. People then shred the answers before collecting payment based on how many the completed. Most people cheat a little, possibly because they can rationalise that they could have solved more, or had almost completed the next one. Few cheat to the maximum, even when it is clear they have the opportunity to do so.

For much of the first part of the book, Ariely frames his research against the Simple Model of Rational Crime (or ‘SMORC’) – where people do a rational cost-benefit analysis as to whether to commit the crime. He shows experiments where people don’t cheat to the maximum amount when they have no chance of being caught – almost no-one says that they solved all the puzzles (amusingly, a few say they solved 20 out of 20, but no-one says 18 or 19). And most people do not increase their level of cheating when the potential gains increase.

As Ariely works through the various experiments attempting to isolate parts of the SMORC and show they don’t hold, I never felt fully satisfied. It is always possible to see how people might rationally respond in a way that thwarts the experimental design.

For example, Ariely found that changes in the stake with no change in enforcement did not result in an increase in cheating. But if I am in an environment with more money, I might assume there is more monitoring and enforcement, even if I can’t see it. However, I believe Ariely is right in arguing that the decision is not a pure cost-benefit analysis.

One of the more interesting parts of the book concerned how increasing the degrees of separation from the monetary outcome increases cheating. Having people collect tokens, which could be later exchanged for cash, increased cheating. In that light, a decision to cheat in an area such as financial services, where the ultimate cost is cash but there are many degrees of separation (e.g. manipulating an interest rate benchmark which changes the price I get on a trade which affects my profit and loss which affects the size of my bonus), might not feel like cheating at all.

As is the case when I read any behavioural science book, the part that leaves me slightly cold is that I’m not sure I can trust some of the results. The recent replication failures involving priming and ego depletion – and both phenomena feature in the book – resulted in me taking some of the results with a grain of salt. How many will stand the test of time?

Kay’s Other People’s Money

Other Peoples MoneyJohn Kay’s Other People’s Money is generally an excellent book. Kay argues that the growth in the size of the financial system hasn’t been matched by improvements in the allocation of capital. He proposes that financial services are not as profitable as some headline numbers would suggest. And he suggests that the replacement of  those who are good at meeting clients on the 19th hole with those who were good at solving complex mathematical problems was not always a good thing – sometimes clever people are the problem, particularly in a complex environment.

I highlighted a lot of passages through the book. Here is a small selection.

First, the chapter on risk in excellent – particularly its treatment of rationality. The point in the following paragraph is in some senses obvious, but often ignored:

If you don’t behave ‘rationally’, you can be ‘Dutch-booked’ – an offensive phrase (to the Dutch – the origins of the expression seem lost in the mists of time) which means that others can devise strategies that will make money at your expense. Many economists use this argument to insist that people do behave ‘rationally’ – behaviour that does not conform to the model will be abandoned because those who engage in it lose money. I used this reasoning myself with students. But I now see it differently. People do buy lottery tickets, week after week, and they do so for reasons that seem entirely valid to them. People don’t behave – for both good and bad reasons – in line with the economic model of rationality. In consequence others do devise strategies that make money at their expense. That consequence is critical to an understanding of how financial markets operate today.

One of the most interesting threads in the books is that many of the regulatory mantras are about the financial intermediaries, not the end users. The drives for transparency and liquidity in particular come in for criticism by Kay. First, the demand for transparency is a sign of the problem:

Transparency is the mantra in the modern world of finance. But the demand for transparency in intermediation is a sign that intermediation is working badly, not a means of making it work  well. A happy motorist is one who need never look under the car bonnet. A good lawyer manages our problem; a bad lawyer responds to every issue by asking us what we want to do. When ill, we look for a recommended course of action, not a detailed description of our ailments and a list of references to relevant medical texts.

And is this demand for transparency even desirable?

The primary objective of the Securities and Exchange Commission … was to increase the quality and quantity of information available to the public. The corollary was that trading should take place on the basis of that information alone.

The idea has superficial attractions and fundamental flaws. The framework of thought is frequently described through the sporting metaphor of ‘fairness’: the ‘level playing field’ on which all players compete on equal terms. To achieve fairness, a standard template of information should be provided to everyone, whether director of a company, investment banker or day trader with a home computer. Market participants may deal, and may only deal, on the basis of that information. No trader can have better information than any other, and success depends only on skill in interpreting it – or anticipating the interpretations of others.

Of course, this ‘level playing field’ is not achievable or achieved, and would not be desirable if it were to be achieved. Yet, like the regulators of casinos, the regulators of security markets often describe ‘market integrity’ as their objective; their focus is on the efficient functioning of the market, in a narrow technical sense that is concerned with process rather than outcome. The emphasis on the preoccupations of market participants rather than the interests of market users is deeply embedded in current thinking.

The effectiveness of financial intermediation in promoting efficient capital allocation depends on the quality of the information available to market participants. Regulation whose primary purpose is to encourage trading by ensuring no trader has an informational advantage actually gets in the ways of efficient capital allocation, in principle and in practice. Effective information and monitoring are best achieved – perhaps only achieved  – in the context of a trust relationship.

And on liquidity:

The need for extreme liquidity, the capacity to trade in volume (or, at least, to trade) every millisecond, is not a need transmitted to markets from the demands of the end-users of these markets but a need, or a perceived need, created by financial market participants themselves. People who applaud traders for providing liquidity to markets are often saying little more than that trading facilitates trading – an observation which is true, but of very little general interest.

Kay also has a subtle shot at the ability of governments to use interest rates to achieve policy outcomes:

Is it desirable for government and its agencies – which have sensibly extricated themselves from the business of controlling most prices – to manipulate interest rates, with a view to managing not just the banking system but the economy as a whole? Electricity is an essential element of the national infrastructure, used by every household and business. It is possible to imagine a government trying to manage the economy by controlling the supply and price of electricity – restraining booms by limiting the availability of new power stations and new connections, or by raising the price of electricity, and tackling recessions with low electricity prices and plentiful power.

I suspect most people would share my instinctive reaction that this approach would be an extremely bad idea – that the outcome would be inefficiency in the supply and use of electricity, and instability in economic growth. Is the intuition that seems relevant to electricity not equally relevant to the financial sector?

I think it is.

And finally, on the inevitability of crises:

The organisational sociologist Charles Perrow has studied the robustness and resilience of engineering systems in different contexts, such as nuclear power stations and marine accidents. Robustness and resilience require that individual components of the system are designed to high standards. … More significantly, resilience of individual components is not always necessary, and never sufficient, to achieve system stability. Failures in complex systems are inevitable, and no one can ever be confident of anticipating the full variety of interactions that will be involved.

Engineers responsible for interactively complex systems have learned that stability and resilience requires conscious and systematic simplification, modularity, which enables failures to be contained, and redundancy, which allows failed elements to be by-passed. None of these features – simplification, modularity, redundancy – characterised the financial system as it had developed in 2008. On the contrary, financialisation had greatly increased complexity, interaction and interdependence. Redundancy – as, for example, in holding capital above the regulatory minimum – was everywhere regarded as an indicator of inefficiency, not of strength.

Thiel’s Zero to One

Zero to oneI am sympathetic to many of Peter Thiel’s arguments in Zero to One: Notes on Startups, or How to Build the Future, but this is not a book where the arguments are buttressed with evidence to convince you they are true. Below are some random observations.

Thiel argues that competition is something a company should avoid. Competition erodes profits, so you want to be a monopoly. In some ways Thiel’s argument isn’t about avoiding competition, but a recommendation to compete in different spheres – competing to find the next monopoly, wealth, status, etc. It’s a recommendation to dream big – look for 10x improvements.

Thiel puts Apple and Google in this camp – they do what they do so much better than their competitors that they are effectively monopolies. In contrast, restaurants tend not be monopolies, with profits rapidly competed away. But this brings the exceptions to mind – the restaurant chains that explode in popularity – think Starbucks. Many businesses have done very well in competitive, commodotised markets. Were they 10x improvements? Is this an ex post justification where all extraordinarily successful businesses can be explained as 10x improvements?

A thread with which I have much sympathy is Thiel’s critique of education conformity (in fact, conformity in general). Through high school the ambitious build their CV to get into a top university. They then aspire to go into law, management consulting or finance. Are they happy competing on this treadmill?

I’m close to convinced that it would be a good thing if less bright people went to law, management consulting and finance (on the last, I’m reading John Kay’s Other People’s Money). But from the perspective of the student, I’m not sure the treadmill is a bad choice. It’s a pretty good payoff. And how many people diverge from this path, dream big and fail big? If society has a dysfunctioning structure – whether created by regulation or something else – why not take advantage of it?

This argument against conformity and the recommendation to compete in new ways is also a central thread in Malcolm Gladwell’s David and Goliath.

There are some sections of the book that didn’t work for me. In a chapter “You are not a lottery ticket”, Thiel disagrees with Malcolm Gladwell’s argument that luck is central to success (found in Outliers). Thiel quotes Warren Buffet’s statement that he is a “member of the lucky sperm club”, or Bill Gates’s suggestion that he “was lucky to be born with certain skills”, which at a level must be right. As Gladwell argues, if Gates was born anywhere but the small part of the US where he was born (with certain genes etc), it wouldn’t have panned out the same for him.

The only piece of evidence Thiel throws into the debate is the existence of serial entrepreneurs who have started multiple multimillion or billion dollar businesses. “If success were mostly a matter of luck, these kinds of serial entrepreneurs probably wouldn’t exist.” Thiel effectively ignores Gladwell’s central argument.

Following this, Thiel moves to classify people’s (and whole countries’ and continents’) perspectives on the future along two spectrums – optimism/pessimism and definite/indefinite. For example, a definite optimist (in Thiel’s mind, the US before the 1980s) believes the future will be better and that you can plan how you will get there. An indefinite optimist also believes the future will be better, but doesn’t know how it will be better so they don’t plan. Rather, they try to keep their options open (e.g. the resume building Ivy League generalist).

Thiel argues for definite optimism. Those who plan and don’t believe it is all down to luck will be the ones who create the future – who go from zero to one. Is Thiel arguing for the “growth mindset” – even if much of success is innate talent through genes etc, those who pretend success is wholly due to effort and plan accordingly will do better?

The argument is interesting, but I’m not sure it’s anything more than storytelling. Are Americans today really less “definite” about the future than people in the 1960s (e.g. is there any better evidence than stories about moon landings etc.)? The argument also made me think of this paragraph from Hayek:

This is not a dispute about whether planning is to be done or not. It is a dispute as to whether planning is to be done centrally, by one authority for the whole economic system, or is to be divided among many individuals.

Is it a problem if a nation is “indefinite” and doesn’t plan if its people are “definite” planners?

Kenrick and Griskevicius’s The Rational Animal

The Rational AnimalI am in two minds about Doug Kenrick and Vlad Griskevicius’s The Rational Animal: How Evolution Made Us Smarter Than We Think. As an introduction to evolutionary psychology and the idea that evolutionary psychology could add a lot of value to economics – and behavioural economics in particular – it does a pretty good job.

On the other hand, the occasional straw man discussion of economics and the forced attempt to sex up the book kept distracting me from the central argument, so I never found myself really enjoying the reading experience. Then there is the heavy reliance on priming research – more on that later.

The basic argument of the book is that people are deeply rational. Today’s choices “reflect a deep-seated evolutionary wisdom”. That wisdom sometimes works well, but we can have an impression that behaviour is irrational because we do not understand what people are trying to achieve. And sometimes this wisdom backfires when the environment is different from the one in which we evolved.

Importantly – and I struggle with this point – they argue that humans pursue several different evolutionary goals and the evolutionary goal that is on someone’s mind at a particular moment will affect the decisions they make. Someone will make a different decision if thinking about acquiring a mate as opposed to responding to a threat to their safety.

Kenrick and Griskevicius identify seven sub-selves that relate to specific evolutionary goals – self-protection, disease avoidance, alliance building, status building, mate acquisition, mate retention and care of kin. When thinking of mate acquisition, we will be interested in demonstrating our value over others. When self protection becomes the focus, we will be happier mixing in with the crowd. Most of the book is an examination of how these sub-selves affect our decision making, including how they vary between the sexes and change over our lifespan.

As a neat example (although note my comments below about priming), people watched a clip from one of two movies, The Shining and the romantic Before Sunrise. They then saw a set of commercials in which products were promoted as being popular (e.g. “visited by over a million people a year”) or unique (e.g. “limited edition”). Those who saw the ads after seeing the clip about The Shining preferred popular products (safety in numbers), while those who saw the romantic film preferred unique products (to attract a mate you need to stand out from the crowd). Different films triggered different sub-selves and accordingly, different decisions.

Through the book, here are a few of the random snippets that I bookmarked:

  • A classic behavioural science problem involves framing a choice between two disease treatments. One group has a choice between saving 200 of 600 people, or having a 33% chance of saving all 600. A second group has a choice between 400 of 600 people dying or a 33% chance that no-one will die. Those who hear the first (positive) framing tend to choose the certain treatment, but most choose the uncertain treatment in the second (negative frame). However, those numbers differ from the typical group size of our evolutionary past – around one hundred people. X. T. Wang found that when the same problem was framed with numbers similar to an ancestral band – i.e. 20 of 60 will be saved – the framing effect disappears.
  • When the prisoner’s dilemma is played between brothers, the payoffs from an inclusive fitness perspective encourage cooperation, and that is what we see.
  • When something is coming toward us – say a rock at our head – our brain tells us it will hit sooner than actually will. It’s an error, but making a predictable error in this direction is not a bad thing. There are asymmetric costs to an error in either direction. The propensity to sense that an approaching object will arrive sooner than it will is called auditory looming.

Now to the main issue that gnawed at me through the book. The arguments heavily draw on research in priming, which is not faring particularly well through the failure to replicate many priming studies and evidence of publication bias. I’ve been willing to give some benefit of the doubt to priming research in evolutionary psychology, as there seems to be some basis for it. It feels reasonable that seeing picture of an attractive woman – even if it is just a picture – could result in more mating-related behaviour (well, certainly more of a basis for that than reading words related to the elderly and walking more slowly).

Alas, even the work in this space seems to be falling apart. I’ve cited that work in my published papers, and believe that many of the underlying phenomena are there (for instance, men taking more risk in the presence of attractive women), but it looks like priming is not the way to show this.

Manzi’s Uncontrolled

ManziIn social science, a myriad of factors can affect outcomes. Think of all the factors claimed to affect school achievement – student characteristics such as intelligence, conscientiousness, patience and willingness to work hard, parental characteristics such as income and education, and then there is genetics, socioeconomic status, school peers, teacher quality, class size, local crime and so on.

In assessing the effect of any policy or program, researchers typically attempt to control for these confounding factors. But as James Manzi forcefully argues in Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society, the high “causal density” in these settings nearly always results in the possibility that there is an important factor you have missed or do not understand.

As a result, Manzi advocates the use of randomised field trials (RFTs) to attempt to tease out whether interventions are having the desired effect. If control and treatment groups are randomised, any unidentified factors affecting the outcome should affect each group equally.

The ubiquity of uncontrolled factors and the ability of RFTs to do a better job of capturing them was demonstrated by John Ioannidis in a 2005 paper evaluating the reliability of forty-nine studies. As Manzi reports, 90 per cent of the large randomized experiments had produced results that could be replicated, compared to only 20 per cent of the non-randomized studies.

Manzi notes that RFTs have critics and limitations, and people such as James Heckman have argued that it is possible to achieve the same results as RFTs using non-experimental mathematical techniques. However, as Manzi points out, Heckman and friends’ demonstration that RFT results can be replicated using improved econometric methods after the fact is not the same as defining a set of procedures that can produce the same effect as future RFTs.

Although Manzi is a strong advocate of RFTs, he is clear that RFTs will not lead to a new era where we will understand everything. High causal density will always place limits on the ability to generalise experimental results. Manzi writes:

[I]ncreasing complexity has another pernicious effect: it becomes far harder to generalize the results of experiments. We can run a clinical trial in Norfolk, Virginia, and conclude with tolerable reliability that “Vaccine X prevents disease Y.” We can’t conclude that if literacy program X works in Norfolk, then it will work everywhere. The real predictive rule is usually closer to something like “Literacy program X is effective for children in urban areas, and who have the following range of incomes and prior test scores, when the following alternatives are not available in the school district, and the teachers have the following qualifications, and overall economic conditions in the district are within the following range.” And by the way, even this predictive rule stops working ten years from now, when different background conditions obtain in the society.

Manzi’s critique of the famous jam study is indicative. Can you truly generalise from 10 hours in one store with shoppers randomised into one hour chunks? Taken literally, the result implies that eliminating 75 per cent of products could increase sales by 900 per cent. However, that hasn’t stopped popularisers telescoping “the conclusions derived from one coupon-plus-display promotion in one store on two Saturdays, up through assertions about the impact of product selection for jam for this store, to the impact of product selection for jam for all grocery stores in America, to claims about the impact of product selection for all retail products of any kind in every store, ultimately to fairly grandiose claims about the benefits of choice to society.”

It’s not hard to come up other studies that are generalised in this matter. The Perry Pre-School project that found benefits for disadvantaged African American children in public pre-schools in the 1960s is generalised to promote more intensive early childhood education for everyone, regardless of country, race, socioeconomic status or era. A single Kenyan case study of deworming leads to a plan to deworm the world. And so on.

As a result, succeeding or failing in a single trial doesn’t usually constitute adequate evaluation of a program. Rather, promising ideas need to be subject to iterative evaluation in the relevant contexts.

Manzi’s reluctance to suggest RFTs will lead us to a new era also stems from the results of the few RFTs conducted in social science. Most programs fail replicated, independent, well-designed RFTs, so we should be sceptical of claims about the effectiveness of new programs. As Manzi states, innovative ideas rarely work.

In his review of RFTs in the social sciences, he does suggest one pattern emerges. Programs targeted at improving behaviour or raising skills or consciousness are more likely to fail than changes in incentives or environment. This might be considered a nod to both standard and behavioural economic tools.

At the end of the book, Manzi provides some guidance on how government should consider programs in an environment of high causal density.

First, he recommends that government build strong experimental capability. To keep the foxes out of the henhouse and avoid program advocates influencing results, he recommends a separate organisational entity be established to evaluate programs.

Second, there should be experimentation at the state level, or at the smallest possible competent authority. This might involve state by state deviation from Federal laws or programs on a trial basis.

Manzi recommends a broader scope for experimentation than you might normally hear advocated, with his suggestion that experimentation extend to examining different levels of coercion:

The characteristic error of the contemporary Right and Left in this is enforcing too many social norms on a national basis. All that has varied has been which norms predominate. The characteristic error of liberty-as-goal libertarians has been more subtle but no less severe: the parallel failure to appreciate that a national rule of “no restrictions on non-coercive behavior” (which, admittedly, is something of a cartoon) contravenes a primary rationale for liberty. What if social conservatives are right and the wheels really will come off society in the long run if we don’t legally restrict various sexual behaviors? What if some left-wing economists are right and it is better to have aggressive zoning laws that prohibit big-box retailers? I think both are mistaken, but I might be wrong. What if I’m right for some people at this moment in time but wrong for others, or what if I’m wrong for the same people ten years from now?

The freedom to experiment needs to include freedom to experiment with different governmental (i.e., coercive) rules. So here we have the paradox: a liberty-as-means libertarian ought to support, in many cases, local autonomy to restrict at least some personal freedoms.

To enable experimentation, Manzi uses an evolutionary framing and notes there is a need to encourage variation, cross-pollination of ideas and selection pressure. Encouraging variation requires a willingness to allow failure and deviation from whatever vision of social organisation we believe is best.

Our ignorance demands that we let social evolution operate as the least bad of the alternatives for determining what works. Subsocieties that behave differently on many dimensions are both the raw materials for an evolutionary process that sifts through and hybridizes alternative institutions, and also are analogous to the kind of evolutionary “reserves” of variation that may not be adaptive now but might be in some future changed environment. We want variation in human social arrangements for some of the same reasons that biodiversity can be useful in genetic evolution. This is the standard libertarian insight that the open society is well suited to developing knowledge in the face of a complex and changing environment. As per the first two parts of this book, it remains valid. But if we take our ignorance seriously, the implications of this insight significantly diverge from much of what the modern libertarian movement espouses.

Manzi highlights the importance of selection pressure is his discussion of school vouchers. He considers that “giving choice” to parents does not necessarily provide an environment in which trial-and-error improvement will occur as there may not be alternatives to status quo, the right incentives for market participants or adequate information for parents. Manzi is also sceptical as to whether taxpayer funded vouchers will come with so many controls to render the experiment useless.

Manzi’s proposals to provide selection pressure are not without problems. He suggests a comprehensive national exam for all schools receiving government funding, with those results published. But is the need to do well in this test is a form of control that kills off much of the experimentation, turning the education system into a group of organisations competing for high test scores?

One of Manzi’s more interesting ideas relates to immigration. Manzi supports programs to attract highly skilled immigrants, such as skills-based immigration programs, or offering entry to foreign students upon completing certain degrees. He proposes testing this idea by using a subset of the visas granted through lotteries to run a RFT. Immigrant outcomes could then be tracked.

Ultimately, however, Manzi’s message is one of humility. No matter what our worldview, we should be prepared to allow experimentation with alternatives, as we may well be wrong. And that favourite program you have been promoting? Feel free to experiment, but don’t expect success. And if it works in that context, test and test again, as it may not work somewhere else.

Best books I read in 2014

Continuing my tradition of giving the best books I read in the year – generally released in other years – the best books I read in 2014 are below (albeit from a smaller pool than usual).

CoatesJohn Coates’s (2012)  The Hour Between Dog and Wolf: Risk Taking, Gut Feelings and the Biology of Boom and Bust: The best book I read this year. An innovative consideration of how human biology affects financial decision making.
The Son Also RisesGregory Clark’s (2013) The Son Also Rises: Surnames and the History of Social Mobility: Not the most exciting book to read, but an important examination of social mobility.
Complexity and the Art of Public PolicyDavid Colander and Roland Kupers’s (2014) Complexity and the Art of Public Policy: An important way to think about policy, even though I’m not convinced by many of their proposed applications.
Rationality for MortalsGerd Gigerenzer’s (2010) Rationality for Mortals: How People Cope with Uncertainty: Essays capturing a lot of Gigerenzer’s important work. I reread it following his disappointing Risk Savvy. I didn’t write a review, but posted about parts of the book here, here and here.
ScarcitySendhil Mullainathan and Eldar Shafir’s (2013) Scarcity: Why Having Too Little Means So Much: A novel way of looking at scarcity that extends beyond the typical analysis in economics, but I’m not convinced it presents a coherent new perspective on how the world works.
Young men in spatsP.G. Wodehouse’s (1936) Young Men in Spats: Magic.