The Importance of Failed Models

Earlier, I discussed an article by Jonathan Newman discussing  what he sees as scientism, specifically:

Scientism has many forms, one of which is the use of empirical methods to do economic science, or the dismissal of claims not based on experiment results that question other claims that are based on experiment results.

I had addressed the first condition of his scientism claim in my last blog post.  Now, I’d like to address the other.

It is true that rejecting claims not based upon experiment results in favor of claims that are based on experiment results can lead to scientism.  Note I said “can lead to scientism.”  I outright reject Newman’s claim that the rejection is in and of itself scientism.  If one gets empirical results that conflict with one’s model, one should ask “why?”  Failure to ask this question is what leads to scientism, not the rejection thereof.  Furthermore, failure to ask this question can lead one to avoid important insights.

Let’s look at two major turning points in the history of economic thought: the Marginal Revolution and the development of Public Choice.

Prior to the Marginal Revolution, the idea of a Labor Theory of Value dominated the economic world.  Promoted by men like Adam Smith and David Ricardo, it was a key concept of economic theory.  However, empirical and philosophical issues were coming about calling into question the LTV.  Rather than simply dismissing these concerns as “scientism”, economists decided to look at the theory.  Carl Menger (whose writings on the matter would become the foundation of the Austrian School of thought) was one of the main questioners.  He developed the idea of diminishing marginal utility and people use rank-ordering of utility to make choices.  William Jevons and Leon Walras also made major contributions to the theory which would eventually become foundational to modern economics.*

About a century later, Gordon Tullock took up Mundell’s challenge for economists to explore why empirically there so little deadweight loss from trade compared to what theory would predict.  It’s a good thing Tullock did not simply dismiss the empirical evidence as “scientism,” but rather explored why the model was failing.  His work lead to the creation of Public Choice economics (you can read in more detail the story here).

One of my professors, Garret Jones, likes to say it is important to see how models fail, if they fail in interesting ways.  The models of the Labor Theory of Value and international trade failed in interesting ways.  One gave us a new understanding of economics (the Marginal Revolution).  They other gave us better insight in economic theory (Public Choice).  I feel confident to say that, if we had simply ignored the concerns raised by empirical results as “scientism,” then the economics profession would be worse off for it.

*I must apologize for not giving Jevons and Walras the time they deserve here for their contributions to the Marginal Revolution; space constraints force me to economize.

7 thoughts on “The Importance of Failed Models

  1. “If one gets empirical results that conflict with one’s model, one should ask “why?” Failure to ask this question is what leads to scientism.”

    The “why” is the healthy skepticism required of all scientific inquiry. Why do climate change models consistently fail to reproduce historical changes in climate? And, why have climate change predictions from years ago been proven so drastically wrong the farther away from those claims we get? Those who fail to ask those questions while demanding action now on climate change are practicing scientism.

    Liked by 1 person

  2. ( )

    Cargo Cult Science – 1974 by Richard P. Feynman
    Commencement speech at The California Institute of Technology

    The speech is important and readable. It explains the difference between mimicking the language and process of science compared to the real thing. It explains what “cargo cult” means applied to our present lives. [edited excerpts]

    === ===
    I often talked to the people in the psychology department at Cornell. One of the students wanted to do an experiment. Others had found that under certain circumstances X, rats did something A. If she changed the circumstances to Y, would they still do A?

    I explained that she should first repeat the experiment of the other person. Create condition X to see if she would also get result A. Then change to Y and see if A changed. She would know that the real difference was the thing she thought she had under control.

    She proposed this to her professor. He replied no, you cannot do that, because the experiment has already been done and you would be wasting time. This was in about 1947. It seems to have been the general policy then to not try to repeat psychological experiments, but only to change the conditions and see what happens.

    [Later, Feynman tells about the studies done by a Mr. Young which determined how perceptive rats really are when trying to test their behavior]

    Young did an A-number-one experiment from a scientific standpoint. His research made maze running experiments sensible, because it uncovered the clues that the rat is really using, not what you think it’s using. And that tells exactly what conditions you have to use in order to be careful and control everything in an experiment with rat-running.

    I looked into the subsequent history of this research. The next experiment, and the one after that, never referred to Mr. Young. They never used any of his criteria of putting the corridor on sand, or of being very careful. They just went right on running rats in the same old way, and paid no attention to the great discoveries of Mr. Young.

    His papers are not referred to, because he didn’t discover anything about rats. In fact, he discovered all the things you have to do to discover something about rats. But not paying attention to experiments like that is a characteristic of cargo cult science.
    === ===

    What is the similarity between our government and some superstitious Pacific islanders during World War II?
    ( ) Cargo Cult Economics

    Liked by 1 person

  3. You learn a lot from your own mistakes and failures as well as from the mistakes and failures of others. To blindly accept anything is not the scientific way.


Comments are closed.