There is a moment now, familiar in its repetition, that would have been difficult to imagine not long ago. You log in, confirm a code, approve a prompt on another device, and wait briefly while the system decides. Nothing about the process feels remarkable. It is quick, often invisible, and usually successful. Over time, it has ceased to present itself as a question and has instead taken on the character of a condition.
And yet, it remains a question, one that is asked repeatedly, even if we no longer experience it that way. Earlier systems allowed for a kind of completion. You proved who you were, and the system accepted it. There was a moment, however limited, in which trust was extended and the question no longer needed to be asked. That moment has largely disappeared. Authentication has become continuous, verification has become ambient, and the process no longer resolves into a stable state of being trusted.
In earlier reflections, I have tried to trace how this shift occurred. Identity moved downward into infrastructure, no longer asserted but provided . Verification followed, not as a discrete event but as a persistent process, one that continually re-evaluates the same claims . More recently, the systems that perform this evaluation have begun to extend their scope, forming judgments not only about what has occurred, but about what is likely to occur . Each of these developments appears reasonable when viewed in isolation. Taken together, they describe something that does not remain confined to technical systems.
There is a kind of inertia to infrastructure. Once established, it invites extension, and the logic that governs one domain rarely remains contained within it. It migrates into adjacent spaces, carrying its assumptions along with it. Identity becomes infrastructure. Trust becomes verification. Verification becomes anticipation. What begins as a technical posture gradually takes on a broader character, appearing first in institutional processes and then in cultural expectations.
This transition becomes more visible in systems designed to unify information across domains, platforms that correlate signals, surface relationships, and present them in a form that invites action. Their value lies not in any single piece of data, but in the ability to combine many pieces into something operationally useful. Patterns emerge. Associations become visible. Probabilities can be estimated. The system no longer merely records what has happened. It begins to model what might.
In some contexts, this structure is already explicit. China’s social credit system provides a particularly clear example of how a digital footprint can be aggregated, evaluated, and used to shape access to opportunities. Scholars have described it as a data-driven governance model in which behavioral signals are collected across domains and translated into mechanisms of reward and sanction.1 Financial behavior, social associations, and expressions of dissent may all become inputs into a broader assessment of reliability. The outcome is not always framed as punishment in a direct sense, but its effects are tangible. Travel may be restricted, services limited, and participation constrained, not necessarily because of a single act, but because of a pattern the system has learned to recognize.
It would be a mistake, however, to treat this as a phenomenon confined to one political system. The underlying logic appears in more familiar and less centralized forms as well. Modern vehicles, for example, increasingly collect detailed telemetry about driving behavior, including speed, braking patterns, and route selection. That data, in some cases, is transmitted to third parties and sold to insurance providers, where it is used to inform coverage decisions and pricing models. The driver may not experience this as surveillance in any overt sense, and the justification is often framed in terms of fairness or risk alignment. Yet the structure is recognizably similar. Behavior is continuously recorded, translated into signals, and evaluated against a model that determines future access and cost.
In both cases, the individual is not judged solely on discrete actions, but on patterns derived from accumulated data. The evaluation is ongoing, and its consequences extend beyond the moment in which the data is produced. What differs is not the logic, but the degree, the visibility, and the institutional context in which it is applied.
It would be easier to understand this shift if such capabilities remained centralized. They do not. They are distributed across layers of governance that are interconnected without always appearing unified. Information moves between federal, state, and local entities, between law enforcement and administrative systems, and between public institutions and private platforms. Practices once associated with exceptional contexts begin to assume a procedural character, not because of a single directive, but because the infrastructure permits it. A capability, once established, invites use, and a system, once connected, invites expansion. The language that accompanies this expansion emphasizes safety, efficiency, and coordination, and each of these justifications is individually persuasive. Taken together, however, they describe something more structural than incidental.
There is an older idea, associated with Jeremy Bentham, that a system of observation does not need to be constant in order to be effective. It need only be possible. When individuals cannot determine whether they are being observed, they may begin to act as though observation is continuous.2 Modern systems do not replicate this structure exactly, but they produce a similar effect through different means. When verification is ongoing, evaluation is ambient, and decisions are rendered without clear or accessible points of explanation, a persistent uncertainty emerges, and it becomes difficult to know when one is being assessed or according to what criteria.
The system does not need to resolve that uncertainty. It only needs to sustain it.
Over time, behavior adjusts. This adjustment is not always deliberate or even perceptible, but it accumulates. Individuals begin to anticipate how systems will interpret their actions, and in doing so, they adapt to those expectations. The result is not simply compliance, but a form of internalization in which the logic of the system is reflected in the behavior of those who move within it.
I sometimes joke with my children that when I take a different route home, it is so the (MiBs) Men in Black cannot follow me. They laugh, as they should. It is an absurd image, drawn from fiction, and meant lightly. But there is something beneath the joke that I find increasingly difficult to dismiss. I do not change my patterns because I have something to hide. I change them because I am not entirely comfortable with how easily those patterns can be known. There was a time when unpredictability was simply a feature of ordinary life. Now it feels, at times, like a deviation from expectation.
I notice the same instinct in smaller, quieter ways. When I sit in a public place, I tend to choose a position where I can see the entrances, where movement is visible and there are fewer blind spots. It is not something I think about in a formal sense, and it is not driven by any specific threat. It is simply a preference that has settled in over time. I am not sure whether it reflects prudence, habit, or something more difficult to name.
In earlier conflicts, the danger of predictability was not theoretical. Soldiers were taught to vary their routes and avoid routine, not because they were hiding something, but because routine could be observed, learned, and exploited. Patterns made people legible, and legibility made them vulnerable. In some cases, that vulnerability extended beyond the individual, reaching into families and personal relationships, where pressure could be applied indirectly. The lesson was simple, if unsettling: it is not only what you do that matters, but how predictable you are while doing it.
What was once a matter of operational security now begins to resemble something else, less acute but more pervasive. The systems we move through do not need to intend harm in order to produce similar conditions. They require only that behavior be observable, recordable, and comparable. The rest follows naturally.
There is a familiar imaginative parallel in The Animatrix, where the development of increasingly capable systems proceeds through a series of decisions that appear justified at each stage. The transition is gradual rather than abrupt, and the outcome emerges not as a singular turning point, but as the cumulative result of many incremental changes. Each step, taken alone, appears reasonable. It is only when viewed together that the direction becomes clear.
Zero Trust, as a framework, originated within cybersecurity as a response to environments in which traditional assumptions about network boundaries could no longer be maintained. Its premise is straightforward: no request should be trusted by default, and every action should be subject to verification. Within its domain, this approach is both practical and effective. It reduces risk in systems where compromise is not only possible, but expected.
When this logic extends beyond its original context, however, it begins to alter the meaning of trust itself. What was once a starting point becomes something provisional, constructed through repeated verification rather than extended as a condition of interaction. Continuous evaluation and preemptive restriction cease to be purely technical controls and begin to resemble broader social patterns. We verify identity before participation, evaluate signals before engagement, and increasingly anticipate outcomes before allowing actions to proceed.
There is, beneath all of this, a broader social trade that is rarely stated directly. Systems of continuous verification are almost always justified in the language of safety, efficiency, and risk reduction. They promise fewer errors, fewer threats, and fewer unknowns. And in many cases, they deliver on those promises. But they do so by narrowing the space in which individuals may move without being observed, evaluated, or recorded.
This is not a new tension. In the years following September 11, the United States expanded its surveillance authorities through measures such as the Patriot Act, often with the assurance that such powers were necessary to prevent future harm. Later disclosures surrounding programs like PRISM revealed the extent to which digital communications could be collected and analyzed at scale, not only to identify specific threats, but to construct a broader picture of activity across populations.3 These developments were defended in the language of necessity, and perhaps they were, at least in part. But they also marked a shift, subtle at first, in how liberty and security were balanced.
There is a philosophical framing that makes this shift easier to accept. One might describe it as a kind of utilitarian response, in which systems are justified by their ability to produce the greatest overall reduction in harm, even if that reduction requires broader observation and constraint. If continuous verification prevents enough negative outcomes, the reasoning goes, then its expansion becomes not only acceptable, but desirable. I understand the appeal of that argument. It is orderly, measurable, and well-suited to systems that operate on aggregation and probability. I am not convinced, however, that it fully accounts for what is lost in the process, or that it ultimately produces the greatest good for the greatest number when those losses are taken seriously.
It is difficult not to hear, faintly, the old warning often attributed to Benjamin Franklin, that those who would give up essential liberty to purchase a little temporary safety deserve neither liberty nor safety.4 The phrasing may be debated, and the context often lost, but the tension remains recognizable. Security and liberty are not always opposed, but they are rarely expanded by the same mechanism at the same time.
I do not find myself persuaded that this trajectory represents an unqualified good. A system that requires continuous verification may be effective, but effectiveness alone is not the same as legitimacy. The American tradition has, at its best, maintained a suspicion of systems that demand constant visibility as a condition of participation. It has assumed, perhaps imperfectly, that individuals should be able to move through portions of life without being continuously measured.
What concerns me is not simply that such measurement is now possible, but that it is becoming expected. What was once justified as an exception begins to resemble a baseline. What was once framed as temporary becomes persistent. And what was once controversial becomes, over time, unremarkable.
Trust does not disappear under these conditions, but it becomes more difficult to locate. It is no longer assumed, and it rarely settles into permanence. Instead, it is recalculated, contingent on signals that can change and assessments that can be revised. In such a system, there is no final state of being trusted, only a continuous process of being evaluated.
I am not sure whether we have built systems that refuse to trust us, or whether we have begun to accept that refusal as normal.
We designed systems that do not trust by default. It is less clear what happens when a society built on the presumption of liberty begins to do the same.
References:
- Liang, F., Das, V., Kostyuk, N., & Hussain, M. M. (2018). Constructing a data-driven society: China’s social credit system as a state surveillance infrastructure. Policy & Internet, 10(4), 415–453. https://doi.org/10.1002/poi3.183 ↩︎
- Bentham, J. (1787/1995). Panopticon; or, The inspection-house. In M. Božovič (Ed.), The Panopticon writings. Verso. ↩︎
- Encyclopaedia Britannica. (2026). PRISM (United States surveillance program). https://www.britannica.com/topic/PRISM-United-States-surveillance-program ↩︎
- Franklin, B. (Attributed). “Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety.” ↩︎
