Economics is only now beginning to come to terms with the external consistency demands posed by research in psychology, sociology and other related disciplines.
I’ve been thinking a bit more about science, pure and applied. My two criteria for “science-ness” thus far have been explanation (providing the causal mechanisms that generate outcomes) and giving priority to the minimization of Type I error. These have to do much more with pure science than applied, however. When a geologist assesses whether a site is stable enough to build on, she is concerned quite as much, if not more, with false negatives than false positives.
Where the two overlap, however, is in the role given to what can be called external consistency. Internal consistency is the property of logical coherence. Economists and others use math, for example, to test for it. An argument is externally consistent, however, if it does not contradict claims thought to be true made by other researchers, typically in adjacent fields. For instance, to be externally consistent a geological theory must adhere to the recognized results obtained in physics and chemistry.
The case for external consistency can be seen as a corollary to Type 1 error minimization: if your hypothesis butts up against arguments or evidence with enough credibility elsewhere, you are at heightened risk of being wrong. But what counts as enough credibility? There is no getting away from the fact that this is an elastic criterion. On one end we have the more-or-less indisputable results of real, fully tested sciences: if you violate any of them, you can’t be right. (I once had a student who proposed a project design for urban rainwater collection that would have violated the second law of thermodynamics. This was extreme.) On the other we have plausible results that have not yet been subjected to critical tests. How much credence we give them, and how we trade off this consideration against others, is a matter of judgment—one characteristic of the quality of a practitioner.
Economics is only now beginning to come to terms with the external consistency demands posed by research in psychology, sociology and other related disciplines. A good applied economist is someone who knows this work, can evaluate how compelling it is and how much impact it should have on the validity of economic models and methods.