Expert scientific witnesses willing to exchange dollars for their reputation have in recent years, and with considerable justification, been blamed for all manner of ill use of science in our courts. Yet entirely overlooked is an insidious alternative: Many superficially legal matters have a scientific core, causing lawyers every day to tread unknowingly on science—indeed, to trample it.
Good science almost invariably implies replication: that is, a decent sample size. Yet legal cases often concern only a single individual. In a dispute over a proposed clear-cut in the Good Hominy Unit, a timber stand containing potential spotted owl habitat, plaintiffs had to demonstrate imminent harm to “an identifiable animal” (United States v. West Coast Forest Resources Limited Partnership, Civ. No. 96-1575-HO [D. Ore. 1997]; emphasis added).
Of course, it would be much too costly and inefficient for government agencies to detail and then respond to the threats facing each individual owl, one owl at a time. Rather, Forest Plans and similar documents allow the government to manage and protect broad, geographically dispersed populations of trees, owls, and other organisms. Surely one might hope for some rough congruence between the spatial scale of these planning activities and the spatial scale of the litigation arising from them.
The news is not encouraging, however. When the Sierra Club challenged a Forest Plan, alleging that it allowed too much logging, the courts refused even to hear its complaint, concluding that the dispute was not “ripe”—a decision reached by considering both the likely harm to the parties from a delayed judicial decision and the court's desire to base any decision not on the abstract and contingent Forest Plan, but on a particular, concrete timber sale proposed under it (Ohio Forestry Association v. Sierra Club, 118 S. Ct. 1665 ). Yet because the territories of spotted owls and other indicator species are typically larger than any given timber sale, arguably N < 1 (that is, any given timber sale contains less than one territory).
There is a back door. One can challenge a particular timber sale proposed pursuant to a Forest Plan, and in this context courts generally allow a challenge to the underlying Forest Plan itself. Alas, when a plaintiff succeeds in opening this back door, they find that it leads not to the house itself, but to a broom closet. Even if a plaintiff succeeds in getting a court to declare a Forest Plan or comparable planning document illegal in some respect, the typical remedy will still pertain only to a particular timber sale or comparable specific site (see, e.g., Oregon Natural Resources Council Fund v. Forsgren, 252 F. Supp. 2d 1088 [D. Ore. 2003]). Having “succeeded,” the plaintiffs must still sue over each proposed timber sale, grazing permit, or other site-specific project.
This dramatically increases the plaintiff's litigation costs, particularly in comparison to the financial resources of the government, making it more likely that their comparative resources will determine the eventual outcome. What's worse, these cases tend to be decided with site-specific evidence that a scientist might rightly dismiss as anecdotal. Superficially, the legal dispute concerns whether the case is ripe, but in reality it is a dispute between those who want a good sample size and those who don't.
One might surmise that the lawyer's favorite sample size is N = 1. But this is overstated. Often in environmental cases, one side will focus narrowly on a particular site or individual, and the other will want to draw on a larger sample. For example, those opposed to the proposed logging of the Good Hominy Unit cited a vegetation map of the area in conjunction with data from other locations as to what constitutes good spotted owl habitat. The defendant timber company focused on the lack of evidence showing that the closest nesting pair of owls, the Chickahominy Creek owl pair, ever used the Good Hominy Unit. The court ordered that these two owls be radio-monitored to determine whether they used this particular timber stand (United States v. West Coast Forest Resources Limited Partnership).
Even if the court must decide whether a particular individual has been harmed, it will usually nevertheless entertain data derived from some larger population: In general, do spotted owls use non–old growth forests? Do PCBs (polychlorinated biphenyls) cause cancer in humans? This is a more generic back door. And no consistent rule tells us which side will argue for N > 1. If the population-level data help their case, the defendant's attorneys will want the court to consider them as evidence; if the data don't help their case, they will argue that these data are inadmissible or should be given little weight. So too will the plaintiffs.
Our legal system redresses grievances of particular individuals, not populations (with the exception of class actions). While courts focus on particular individuals, the scientist's statistical tools allow meaningful conclusions only about entire populations.
Thus the very structure of the law is sometimes biased against good science. In the present examples, it is the sample size that suffers collateral damage when the lawyers battle for their clients' interests. Indeed, even judges may unknowingly promote inferior science, as when the rules for making legal decisions implicitly give undue weight to absurdly small sample sizes.