A Public AI Plan Is Not Yet Public Judgment

The real test of public AI is not capability or efficiency, but who retains agency when objectives conflict and systems fail.

The premise worth questioning is that introducing AI into public administration is primarily a capability upgrade: better analysis enters the system, so better government comes out. Evidence that AI support can improve measured performance does exist; one survey of 33 professionals found a moderate positive association between AI use and decision effectiveness, particularly in accuracy, speed, and operational efficiency . But that result does not establish that an institution has chosen the right objective, represented the right people, or preserved meaningful avenues for contesting a decision. Even technically successful decision support can optimize a criterion that residents would reject, conceal a conflict among legitimate values, or move practical authority away from the official who remains publicly accountable. Research on algorithmic search explicitly distinguishes structural balance from outcome quality, showing that a satisfactory result does not reveal whether the process explored alternatives symmetrically . That distinction matters more in government than in ordinary productivity software because residents experience not just an answer but the allocation of power that produced it. Before asking whether AI makes city decisions faster, the city should therefore answer a prior question: which decisions may be accelerated without making democratic disagreement harder to see?

The strongest evidence for human-AI collaboration actually weakens the case for treating deployment itself as the achievement. In one controlled framework, collaborative teams reached 91% decision accuracy and reported higher calibrated trust than human-only, AI-only, and sequential arrangements, while decision time fell across repeated cycles . Yet the same study found that AI-led ownership was much higher in analytical stages than in judgment and ethical stages, and its authors reserved case-dependent understanding, ethical values, and creativity for human reasoning . The useful unit was thus not “AI assistance” in the abstract but a division of labor that assigned different kinds of authority to different participants. A separate system for industrial task adjustment makes the point more sharply: only 18 of 28 workflow cases met expectations, autonomous strategy-workflow success was 3 of 10, and failures included semantic distortion, incomplete evidence, and missed invalid inputs . All four cases that passed preliminary checks, produced complete evidence, and reached final engineering review passed that review, suggesting that the surrounding gates—not generation alone—created the dependable result . Public agencies should consequently describe not merely what a model can propose, but who verifies its interpretation, what evidence must accompany it, who can stop it, and what happens when the workflow cannot produce a reviewable case.

Nor should the city assume that adding more consultation, communication, or interconnected data will automatically correct the weaknesses of an automated process. Research on distributed inference shows that communication can bring agents into agreement while making their decisions worse . In the paper’s linear examples, systems with identical eigenvalue and singular-value spectra produced gains of opposite signs solely because message orientation differed; one orientation raised accuracy from 72.6% to 91.2%, while another lowered it to 65.9% . The implication is uncomfortable for public engagement: consensus, connectivity, and average performance can all improve while the collective conclusion deteriorates for the actual task. The same work found that community-shared bias can allow higher mean individual accuracy to coexist with lower global-vote accuracy or harm to communities that did not share the bias . Calibration constraints reduced observed community harm while retaining much of the average benefit, but they did not guarantee protection . A credible public AI proposal therefore needs community-level failure tests, not merely an overall accuracy figure or a claim that broader information sharing will make the system fairer.

Fairness itself cannot safely be treated as one more adjustable objective whose weight officials can turn upward until the controversy disappears. A reinforcement-learning study of the ultimatum game found that increasing fairness pressure generally promoted fair outcomes, but moderate pressure unexpectedly made responders more willing to accept low offers . The reversal arose from competition between payoff-maximizing and fairness-oriented preferences rather than from the disappearance of either preference . That mechanism challenges the idea that naming fairness as an optimization goal tells us how a system will behave when fairness conflicts with material benefit, mercy, consistency, or institutional cost. It also shows why a model can produce behavior that appears forgiving while leaving unresolved whether vulnerable people are being protected or merely induced to tolerate worse treatment. City decisions routinely require judgments about which differences are relevant, whose losses deserve priority, and whether equal treatment is equitable under unequal conditions; those are normative choices, not technical properties supplied by an objective function. The public should be able to examine such choices through concrete dilemmas—much as people do when answering dilemmas on Decide—rather than encounter “fairness” only as a polished principle in a policy document.

The gap between principles and enforceable constraints is not hypothetical, although evidence from other political settings cannot by itself establish how New York will govern its systems. A study of AI governance across 22 Arab League states found that seven had both national strategies and dedicated bodies, yet none of those seven had enacted binding AI legislation . Across that corpus, ethics charters covered consumer-protection domains extensively while largely omitting regime-critical surveillance domains . International ethics alignment also co-occurred strongly with documented AI surveillance deployment, showing that formal ethical conformity and intrusive operational practice can exist together . Those findings should be used as a warning about institutional form, not as evidence that distinct jurisdictions are equivalent. A charter can make a program easier to endorse while leaving procurement rules, audit access, appeal rights, data retention, and prohibitions untouched. The premise to reject is that declaring principles meaningfully constrains a system; the relevant question is which actor can enforce each principle when compliance becomes costly.

Even retaining a human “in the loop” is inadequate if the interface determines what the human notices, understands, or feels entitled to challenge. A scenario-based experiment with 250 participants found that analytic structuring increased psychological empowerment, which in turn predicted higher perceived decision quality . The study measured perceived quality rather than independently verified correctness, so empowerment and confidence should not be mistaken for proof that a decision is right . It nevertheless demonstrates that presentation structure influences users’ sense of agency and reflective engagement, while self-reported trust regulation shaped the relationship between empowerment and perceived quality . Human review can therefore become ceremonial when an interface presents one recommendation as complete, compresses uncertainty, or makes alternatives expensive to retrieve, even if a person still performs the final click. Research on human-machine systems accordingly argues for balancing autonomous functionality with user control, as in assisted wheelchairs that combine user direction with automated safety features . For civic systems, genuine human authority should mean the practical ability to inspect assumptions, request alternatives, delay action, document disagreement, and reverse a recommendation—not simply the presence of a municipal employee at the end of the chain.

The better premise is that public AI must earn legitimacy at the level of decision architecture, one consequential judgment at a time. The evidence supports carefully partitioned collaboration, explicit verification, and structured review more strongly than it supports broad faith in either autonomous performance or human oversight by title alone . It also shows why aggregate success can conceal directional failures, community harms, and distorted internal search even when top-line accuracy improves . A city should therefore publish the objective of each system, the values it cannot reconcile automatically, the evidence required before action, the communities for whom performance is separately tested, and the official empowered to refuse the output. It should also identify decisions that remain non-delegable because they require contested judgments about dignity, desert, consent, or acceptable sacrifice, rather than pretending every disagreement can be converted into a stable score. None of the supplied studies proves that a particular municipal deployment will succeed or fail, and the agency framework that links non-anticipability to substantive choice explicitly does not prove that human decision-making satisfies its premises . That uncertainty is precisely why the burden should remain on the institution to demonstrate preserved agency and enforceable recourse, rather than on residents to discover after deployment where human judgment quietly ended.

Decisions · Articles · The Trolley Problem · Game Theory · Human–AI Alignment · Resonant