A Policy Pilot Needs Permission to End

A trial can become permanent before its question is answered

A public program is announced as a pilot. The limited scale suggests caution: officials will try an approach, assess it and decide what follows. Soon there are contracts, public promises, staff roles and people who depend on continuation. Ending the pilot now appears to threaten more than an intervention. It may threaten reputations and a story about progress. The experiment has acquired constituencies before the evidence has been examined. Unless its governance can tolerate an unfavorable answer, the word pilot may describe only the size of the launch rather than a genuine willingness to learn.

Trying something new is not automatically an experiment. A useful pilot identifies the question being tested, the conditions under which the result can be interpreted and what decisions the evidence could support. OECD discussion of policy experimentation emphasizes evaluation and a path toward scaling successful interventions or discontinuing unsuccessful ones. That general principle does not mean every public program should use the same study design. It means the arrangement should be capable of producing a consequential answer, including an answer that the people who launched it would rather not hear.

Continue this question: A Political Promise Is Incomplete Without Its Tradeoffs →

State what success would mean before the announcement

Imagine a hypothetical city testing an appointment system at two service centers. Officials want shorter waiting times. The first report shows that booked visitors are processed more quickly, but walk-in users wait longer and some residents cannot navigate the booking method. If success is defined only by the favored measure, the pilot can be declared effective while displacing the original problem onto a less visible group. A better evaluation would have identified relevant outcomes and affected users beforehand rather than discovering the excluded costs only after the headline result was chosen.

Ask which outcome the pilot seeks, how it will be measured, whose experience will count and what comparison is appropriate. Improvement after introduction does not by itself prove that the intervention caused it. Demand may have changed, staffing may have increased or another reform may have occurred. Evaluation design requires relevant expertise and proportionate resources. A general public question need not settle the method, but it can ask whether the comparison and assumptions are explained. Without that explanation, a before-and-after story may receive more authority than the evidence can carry.

Define review decisions that remain available

Before implementation, identify the review date, the authority responsible and the possible outcomes: continue, revise, expand, pause or end. Specify how serious harms or operational failures will be addressed before the scheduled review. A fixed duration should not authorize avoidable damage until the calendar expires. Equally, one inconvenient complaint should not automatically replace an established assessment process. The response needs to distinguish ordinary implementation issues, a flawed measurement and a substantial failure of the intervention. Those distinctions should be examinable rather than settled by whichever reaction is politically loudest.

Resources and commitments need an exit arrangement. Consider what happens to staff, participants, data and essential services if the pilot ends. Reversibility does not mean abandoning people who have come to depend on the program without preparation. It means avoiding unnecessary commitments that make a meaningful decision impossible. Relevant legal and administrative obligations require appropriate assessment in their own jurisdiction. Citizens can still ask a plain question: what practical conditions would allow the institution to stop if the evidence did not justify continuation? A confident answer about innovation is not the same as an answer to that question.

Continue this question: Being Consulted Is Not the Same as Having the Decision Explained →

Publish what the pilot learned, including its limits

A useful report separates implementation from impact. Was the intervention delivered as intended? What changed? What remains uncertain? A promising result at one small site may not establish that the same approach will work at a different scale or in a different population. A disappointing result may reveal a problem with the design rather than prove the underlying public need was imaginary. Learning should make the next decision more precise. It should not require every outcome to be translated into evidence that the original announcement was correct.

Public communication also matters after an unfavorable finding. Explain what will change and why, rather than silently extending a temporary program while waiting for a more convenient result. Recognize the work involved without requiring that work to validate its own continuation. Staff and participants may have contributed valuable information even when the intervention is not expanded. The capacity to end a pilot responsibly can therefore be evidence that the institution is taking its experimental claim seriously, not simply evidence of embarrassment or political defeat.

The CyGuru inquiry concerns whether authority creates structures that can withstand reality. A pilot is a concentrated test of that capacity because its stated purpose includes uncertainty. When stopping criteria, comparisons and responsibilities are absent, temporary implementation can become a route around scrutiny: the program is too provisional to evaluate and too established to end. A genuine trial avoids that trap. It gives the public a clear account of what is being learned and gives the decision-maker a legitimate route to act on the answer rather than merely preserve the appearance of momentum.