Tuesday, May 19, 2015

A Veritable Epidemic of Mathiness

Paul Romer’s eruption against mathiness has been quite a spectacle.  Here you have an iconic name in modern economic theory throwing a fit in public, naming names (some of them also iconic) and denouncing his adversaries as enemies of scientific and ethical norms.  It’s a bit over the top, a bit overdue and a bit underconsidered.

I want to focus on the underconsidered part.  I was alerted to this aspect of Romer’s original paper by his sideswipes at Joan Robinson and the UK faction of the Cambridge capital controversy.  Now, it happens that I take a middle position on this dispute: I think they were both in some sense wrong.  The British Cantabrigians, along with their Italian comrades, were arguing from a model whose equilibrium assumption (equal rates of profit in all processes) is meaningless, in a mathiness sense, in an intertemporal context.  (If you think Lucas rational expectations is a stretch, Sraffa rational expectations is even crazier.)   But the MITers were also defending an aggregation of physical capital and its equivalence to a sum of financial capital that was also shown to be mathy—see here and here.  Romer’s attack on Robinson was signaling that a double standard was at work.

In fact, economics is a veritable empire of mathiness.  I agree with Romer that the use of algebraic entities that have no meaningful correspondence to real world objects and deliberate obfuscation through the use of words with multiple meanings are sins against science, but that is just the beginning.  Here are two more, one theoretical, the other empirical:

1. Equilibrium with mechanisms.  A large part of economic theory takes the form of equilibrium conditions and the comparative statics thereof.  Even theories that describe events transpiring through time usually take the form of traversals: routes mapped from an initial equilibrium state to its successor.  What usually goes missing are the mechanisms, the processes, in principle observable in the real world, by which actors revise their behavior and produce new collective outcomes.  Without such mechanisms the concept of equilibrium is meaningless: you can’t get there from here.  One symptom of this malady is the inability to distinguish between equilibrium conditions and identities—equal signs and identity signs.  The difference is that causal processes apply to the first but not the second, so if your theoretical world lacks processes altogether you won’t know what that extra little line, ≡ vs =, is all about.

2. Misuse of null hypothesis significance testing.  Suppose you have a theory that A causes B.  You can’t observe this directly (or you haven’t bothered to try), but you can infer that, if this is true, a relationship between two measurable variables, an x and a y, will occur.  So you do a study on x and y, run a significance test and conclude you can reject the null hypothesis that x and y are unrelated.  And, if you are like most empirical economists, you will then announce that you have “tested” your theory about A and B and have found that the evidence is “consistent with” it.  But wait!  There are other theories that would also generate expectations on x and y and they may be inconsistent with yours.  If the x-y business actually lends more support to one of these other theories than to yours, the evidence is saying the opposite of what you claim it says.  The reality is that rejecting a null hypothesis says nothing at all about which of the many possible explanations for the non-null is correct.  A conscientious empiricist would put all the potential theories on the table and consider in a systematic manner how the new evidence alters the relative credence we should give them.  The reason utterly implausible theories live on, decade after decade, in economics is that low-bar implications—implications suggested by many theories of greater or lesser plausibility—survive significance testing and are then proclaimed “consistent with” the particular theory to which the researcher is attached.  This too is a kind of mathiness: lots of fancy econometric technique wrapped around a dishonest and thoroughly unscientific core.

So I have mixed feelings about the Romer meltdown.  I definitely understand where he’s coming from and how frustrating it is to see ideologues deploying math to obfuscate rather than clarify.  But the problem is much wider and deeper than he seems to realize.


Thornton Hall said...

The thing that needs explaining is the persistence of economics. Understanding how critiques like Romer's strengthen rather than weaken the edifice is one of the steps needed before the dismantling can begin.

Also, there needs to be an alternate way into the discourse. Under the current selection method, the solution to the persistence of economics problem requires someone to say: "I have a certain set of skills. I base my self esteem on those skills. But those skills are worse than useless in the field of economics. Doing what I do best is making the world a worse place."

Wallfly said...

PD, It seems your "abuse of hyp. testing" is related (if not synonymous) with the abuse of observational data where it is treated as dispositive when in fact it is not in general, and rather experimental data is. (Though I know it near impossible to run experiments in various areas of econ - e.g. macro). Am I missing something?

Peter Dorman said...

Wallfly, no -- these are different points. (1) The issue is not about confounders and whether statistical control takes care of them. (2) It *is* about the difference between rejecting a null (all theories suggesting a relationship between x and y are still on the table) and adducing evidence for one particular theory.

reason said...

I must say, I am a bit confused here:
"The difference is that causal processes apply to the first but not the second, "
is this true. For instance the identity that spending equals income, has a direct causal process - money changes hands. (i.e. A decrease in consumption implies ceteris paribus a fall in total income). Whereas the causal process in clearing markets (failure to sell results in the decision of producers to either reduce quantity produced or reduce prices) is just less direct.

reason said...

Thornton Hall,
I think you must have left a word out somewhere - economics is a field of study. You want to stop human curiosity? Don't you mean some particular economic school of thought or methodology here? Precision please.

Thornton Hall said...

Reason: "Don't you mean some particular economics school of thought or methodology here?"

This is one of the rote responses that enables the persistence of economics. It is powerful because it seems so completely reasonable.

But compare the following claim, "You criticize the entire GOP, but surely it is only a small minority that rejects reality, and extreme liberals are equally bad."

Or consider this: devastating critiques of this or that subset of economics have been advanced for at least a century. The result? Behavioral economics has been co-opted to produce Neoclassical results and other would be critics have become New Keynesians, satisfied with modifying axioms when they clearly know that they should be rejecting the axiomatic method. That is to say, 100 years and zero improvement.

Or this. Brad DeLong's intro lecture features an apple cart selling apples. Supply equals demand and the market clears. But behind every grocery store in America, every single one, is a dumpster that reeks of rotting fruit. Is there some part of economics not repudiated by this simple contradiction?

Thornton Hall said...

To be fair, in 100 years there has been one victory. Game theory has helped the distribution of organs for transplant. People like to point to that.

Noni Mausa said...

reason talks about spending being equivalent to income. This is self evident, and therefore suspect. (In economics, everything that is self evident is suspect.)

Oh, most spending is equivalent to *someone's* income, but increasingly, collective spending in its myriad small increments is funnelled to a handful of capacious pockets. So, although gallons of water are being carried to the garden, they're not being sprinkled on the marigolds, they're being dumped in the pool.

You end up with two economies side by side, one an Olympic pool of liquidity, the other a desert.

Hey, no math in this metaphor! Therefore it must be true! /sarcastic whimsy


Thornton Hall said...

I should have started with this one. It's as conclusive as an inductive argument can get:

As a thought experiment, imagine an academic discipline where every practitioner shares the same basic error of reasoning. Now imagine that the critics of that discipline correctly point out all the bad results generated by the practioners, but fail to notice that they also endorse the faulty reasoning that drives all the errors. What would the discourse in that discipline look like?

Now open your eyes. Recognize anything?

gaddeswarup said...

The book mentioned on the average production function is freely downloadable from http://jesusfelipe.com/cat1