The Case Against the Case Study

A single detailed case sits at the bottom of the usual hierarchy of evidence, and for most purposes the ranking is right. One school, one firm, one patient cannot tell you how often anything happens, and a discipline that generalises from single cases will generalise from unrepresentative ones, because the cases that get written up are the ones somebody found striking.

[A] But the ranking answers a question about frequency, and not every question is about frequency. If a theory says that a thing cannot happen, one well-documented instance of it happening settles the matter, and no number of cases in which it did not happen would have done the same work. [B] Cases also do something no survey can, which is to show a mechanism operating end to end in a setting nobody designed. A survey can establish that two variables move together across a thousand firms; it cannot show you the sequence of decisions by which they came to. [C] The honest position is not that cases are weak evidence but that they are evidence for different claims, and the error worth avoiding is not using them but forgetting which claim you were making. [D] A case offered as proof of how common something is deserves its low ranking; offered as proof that something is possible, it may be the strongest evidence there is.