Experimenting on People Who Did Not Volunteer

A researcher who wants to know whether a policy works has a strong methodological reason to assign it at random. If a city gives a benefit to whoever applies, the people who apply differ from those who do not in ways nobody can fully list, and any comparison between the two groups measures those differences as much as it measures the policy. Randomisation removes the problem at a stroke.

[A] It also creates one. Two households in identical circumstances receive different treatment because of a number generated by a computer, and neither has agreed to take part in anything. The usual defence is that the resource was scarce anyway, so somebody had to be turned away, and a lottery is at least impartial. [B] That defence is strong where the resource really is limited and much weaker where the scarcity was created in order to permit the study. [C] A second defence holds that the alternative is worse: continuing an unevaluated policy indefinitely also imposes costs, and those fall on people who did not consent either, with the difference that nobody is counting them. [D] Both defences are reasonable and neither settles the matter, which is roughly the state of the argument among the people who conduct such trials.