Saturday, September 5, 2009

A Dynamic Stochastic Cargo Cult

by the Sandwichman

"It was a hodgepodge of unsupported arguments, outright mistakes, and impenetrable jargon designed to 'test' its host journal's intellectual integrity."
In the end, I resorted to parody for a simple pragmatic reason. The targets of my critique have by now become a self-perpetuating academic subculture that typically ignores (or disdains) reasoned criticism from the outside. In such a situation, a more direct demonstration of the subculture's intellectual standards was required.
"But there is one feature I notice that is generally missing in cargo cult science."
It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty--a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid--not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked--to make sure the other fellow can tell they have been eliminated.

Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can--if you know anything at all wrong, or possibly wrong--to explain it.

...And it's this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science.


9 comments:

Max Jr (not the original one) said...

Very relevant quotes. But whom are you quoting?

The second one is from Feynman, and the first one? Alan Sokal?

On DSGE models: David Colander and Alan Kirman have been vocal critics. Maybe Barkley Rosser will have something to add.

Prima facie, any model that assumes an all knowing representative agent with rational expectations over infinite horizons seems like pure poppycock.

I guess that any model, no matter how absurd, that has been suitably calibrated with real world data will give 'acceptable' forecasts over a short time period during times of economic stability.

Barkley Rosser said...

I regularly bash DSGE models, here and abroad. I was unable to attend the Dahlem meeting where Colander and Kirman and some others issued a widely spread statement that has led to Colander testifying before the House Science and Technology Committee about the orientation of NSF research funding in economics.

It is amazing how many people have simply bought into Ohanian lock stock and barrel when there are so many holes in his argument, including some historically distorted facts.

Jack said...

There seems, to an outsider's eye with only a superficial view, that there are two areas of study that use the description of economics. One group without much attention from the lay public or the general media toil away at their business of collecting data, analyzing interactions, in general making an effort to make sense in a scientific manner out of economic phenomenon. The other group, with the ear of both the news media and our government officials, toil away at creating justifications for interpretations of the economic process. Many of the members of this group find practical business applications for their craft. Many come out of prestigious centers of study having been taught by predecessors who had set a certain form of research approach wherein models took the place of theory supported by empirical data. Many of this group participate in continuing to promote their unique economic perspectives through participation as leaders in those very centers of study which had fostered their conceptual development. These same leaders can often also be found participating in a uniquely
twentieth century phenomenon of the privately supported "research institute or foundation." The terms scholar and associate are often used to describe their relationship to those institutes. It seems not to be clear as to what financial arrangement exists between the parties.

When trying to understand the motives of professionals and men and women of science it is often worth while to look for the financial arrangement. Not only between the individual and the organization, but also between the organization, the individual and the group that provides financial support for the organization. One outstanding case in point is as follows, and I only use this as an example of how ideas can be made to flow with such a single minded intention. One might ask why has Chicago Booth School of Business harbored and produced such a persistent stream of economists with a single minded approach to their field of study? Here is a link to what is referred to by the school as the "Council on Chicago Booth," http://www.chicagobooth.edu/about/leadership/council.aspx

The Council is as good a representation of corporate America as one would hope to find. And the purpose of the Council? Again in the school's own words, "The Council on Chicago Booth advises Booth's leadership in their efforts to maintain faculty excellence and develop the highest quality management education programs in the world. We gratefully acknowledge the extraordinary contributions of Chicago Booth Council members."

What has this to do with the success and influence of Ohanian? It is only an example of how things proceed in some sectors of academia. Ohanian comes out of the University of Rochester and now Directs a privately funded program at a public university, UCLA. How does UCLA come to decide who will hold their professorial positions? Well they have a Board of Visitors to, "play(s) a key role in helping UCLA's Department of Economics retain and enhance its position as one of the premier Departments and leaders in the field of Economics. Presently one of the youngest and most vibrant economic departments in the country,.." How many academics sit on that Board of Visitors to assist in making those academic decisions? Take a look for yourself. http://www.econ.ucla.edu/community/bov/index.html

So you see it is always important to understand how organizational decisions are made and what influence they may have on the individuals who participate in the organizations activities. As the man once said, "Cause that's where the money is."

Sandwichman said...

As a professional courtesy (and occasional provocation), the Sandwichman routinely notifies the targets of his critiques by email. In this case, I sent an email to Meg Sullivan, author of the UCLA press release on the Ohanian paper. I received an Out of Office reply advising me, "If you're seeking information about the annual state of the unions report by the Institute of [sic] Research on Labor and Employment, please call (310) 923-5162."

It's the Institute for Research on Labor and Employment. So, I've emailed the Director, Chris Tilly, to see what he has to say about all this. Tilly's PhD dissertation, by the way, was on part-time and contingent work. So he might know a think or two or three about working time and productivity.

Anonymous said...

"When trying to understand the motives of professionals and men and women of science it is often worth while to look for the financial arrangement."

The starting point to understanding any economist's "financial arrangements" and, therefore, their position in the structure of things is muckety.com

There, you will see much of their connections within the Washington-Wall Street axis...

Anonymous said...

Chris Tilly was associated with Dollars and Sense magazine at one point.

Anonymous said...

The quote is from Richard Feynman, right?

-- Min

Sandwichman said...

Yes.

Jack said...

Walker,
I owe you gratitude for the introduction to Richard Feynman, whom I had never heard of previously.
How unfortunate for me that Feynman seems to have suffered the same fate as the psychologist Young that he referred to. So how is it that cargo junk science is able to proceed uninterrupted in our age of instant communication? Note the discussion above wherein we discuss Krugman's "failure" to adequately disclose the failures of his colleagues rather than the failures themselves. Another example of killing the messenger.