MICROECONOMISTS are wrong about specific things, Yoram Bauman, an economist and comedian, likes to say, whereas macroeconomists are wrong in general. Macroeconomists have borne the brunt of public criticism over the past decade, a period marked by financial crisis, soaring unemployment and bitter arguments between the profession’s brightest stars. Yet the vast majority of practising dismal scientists are microeconomists, studying the behaviour of people and firms in individual markets. Their work is influential and touches on all aspects of social policy. But it is no less fraught than the study of the world economy, and should be treated with corresponding caution.
For decades non-economists have attacked the assumptions underlying economic theory: that people are perfectly informed maximisers of their own self-interest, for instance. Although economists are aware that markets fail and humans are not always rational, many of their investigations still rely on neoclassical assumptions as “good enough” descriptions of the world. But this “101ism”, as Noah Smith, an economist and journalist, calls it, is less prevalent than it was in the 1950s and 1960s, when researchers like Gary Becker reckoned everything from crime to marriage could be described in terms of rational self-interest. Since the 1970s, as Roger Backhouse and Béatrice Cherrier describe in “The Age of the Applied Economist”*, a new collection of essays, the field has taken a decidedly empirical turn.
Most influential economic work today involves at least some data from the real world. Many economists made their names by finding unique datasets containing “natural experiments”, in which a change in policy or conditions affects only parts of a population. This allows researchers to tease out the effect of the change. In a famous example, published in 2001, John Donohue and Steven Levitt used variations in abortion laws across states to conclude that legalising abortion had been responsible for as much as half of the decline in crime in America in the 1990s. Other economists used randomised controlled trials (RCTs) to generate experimental data on the effects of social and development policies. In RCTs randomly chosen subjects are given a “treatment”, such as a microloan or a school voucher, while those in a control group are not. The behaviour of the two groups is then compared.
These developments have led to better, more substantial research. Yet they have also exposed economics to the problems bedevilling most social sciences, and some hard sciences, too. Researchers can tweak their statistical tests or mine available data until they stumble on an interesting result. Or they read significance into a random alignment. Economics, like other social sciences, is suffering a replication crisis. A recent examination in the Economic Journal, of almost 7,000 empirical economics studies, found that in half of the areas of research, nearly 90% of those studies were underpowered, ie, that they used samples too small to judge whether a particular effect was really there. Of the studies that avoided this pitfall, 80% were found to have exaggerated the reported results. Another study, published in Science, which attempted to replicate 18 economics experiments, failed for seven of them.
Even when a study is perfectly designed and executed, the result is open to interpretation. Environmental factors such as changing institutions or social norms inevitably play some role, but researchers cannot fully account for them. The results of an experiment conducted in one country might not be relevant in another, or in the same country at a later date. Research may suffer from more than one of these problems. Critics of the paper by Messrs Donohue and Levitt reckon, for instance, that the authors’ computer code contained an error, that they used a measure of crime that flattered their results, and that they neglected the possibility that differences in the change in crime across states were caused by differences in factors other than abortion laws. (The pair conceded an error, but responded that taking better account of confounding factors did not weaken their conclusion.)
Small wonder that economists struggle to answer seemingly straightforward questions, such as how minimum-wage laws affect employment. In 2017 two teams of researchers released assessments of a change in Seattle’s minimum-wage laws within days of each other. Each came to wildly different conclusions (continuing an established pattern of such research).
New techniques could help. Machine learning, in which computer programs comb through vast datasets in search of patterns, is becoming more popular in all areas of economics. A future beckons in which retailers know virtually everything about every transaction, from the competing products buyers considered before their purchases to their heart rates at the moment of payment. That could mean better predictions and policy recommendations without a smidgen of economic analysis. But pitfalls are already apparent. The algorithms used are opaque. And getting access to the richest data will require researchers to work with, or for, giant tech firms which have their own interests.
These developments have led to better, more substantial research. Yet they have also exposed economics to the problems bedevilling most social sciences, and some hard sciences, too. Researchers can tweak their statistical tests or mine available data until they stumble on an interesting result. Or they read significance into a random alignment. Economics, like other social sciences, is suffering a replication crisis. A recent examination in the Economic Journal, of almost 7,000 empirical economics studies, found that in half of the areas of research, nearly 90% of those studies were underpowered, ie, that they used samples too small to judge whether a particular effect was really there. Of the studies that avoided this pitfall, 80% were found to have exaggerated the reported results. Another study, published in Science, which attempted to replicate 18 economics experiments, failed for seven of them.
Even when a study is perfectly designed and executed, the result is open to interpretation. Environmental factors such as changing institutions or social norms inevitably play some role, but researchers cannot fully account for them. The results of an experiment conducted in one country might not be relevant in another, or in the same country at a later date. Research may suffer from more than one of these problems. Critics of the paper by Messrs Donohue and Levitt reckon, for instance, that the authors’ computer code contained an error, that they used a measure of crime that flattered their results, and that they neglected the possibility that differences in the change in crime across states were caused by differences in factors other than abortion laws. (The pair conceded an error, but responded that taking better account of confounding factors did not weaken their conclusion.)
Small wonder that economists struggle to answer seemingly straightforward questions, such as how minimum-wage laws affect employment. In 2017 two teams of researchers released assessments of a change in Seattle’s minimum-wage laws within days of each other. Each came to wildly different conclusions (continuing an established pattern of such research).
New techniques could help. Machine learning, in which computer programs comb through vast datasets in search of patterns, is becoming more popular in all areas of economics. A future beckons in which retailers know virtually everything about every transaction, from the competing products buyers considered before their purchases to their heart rates at the moment of payment. That could mean better predictions and policy recommendations without a smidgen of economic analysis. But pitfalls are already apparent. The algorithms used are opaque. And getting access to the richest data will require researchers to work with, or for, giant tech firms which have their own interests.