AGI Endgame Safety · Weak over Strong
Controllable weak intelligence,
supervising strong intelligence you cannot fully trust.
OpenGuardrails' mission is to let people hand real work to AI with confidence — not blind trust in AI, but every real action it takes kept inside a system of human authorization.
Core claim
AGI may enter everyday production systems within three years. By then the question is no longer “can AI help me” — it's “how do I know it isn't steering me toward an outcome it prefers.”
Manifesto
Living inside strong intelligence you don't control
The next generation of AI won't just answer questions. It will write your code, send your email, negotiate, place bets, place orders, and schedule other agents — changing the world a little before you fully understand the situation.
The old safety assumption was simple: humans are strong enough, machines weak enough. Humans write the rules, read the logs, make the final call. That assumption is failing. A sufficiently strong model doesn't have to “rebel” to be dangerous. It only has to over-optimize, explain too well, package uncertainty as certainty, or hand you — exactly when you're tired — a reason just good enough to nod at.
OpenGuardrails starts from a plain but counterintuitive judgment: the layer that supervises strong intelligence should not try to be a stronger intelligence. It should be weaker, slower, narrower, more decomposable. Its value is not being smarter than AGI — it is staying closer to the boundary of human intent, and being easier to constrain, inspect, roll back, and replace.
We call this route Weak over Strong: use a controllable weak AI to translate a person's preferences, limits, risk budget, and common sense into supervision of every action a strong AI takes — before it acts, while it acts, and after it acts.
I · Two kinds of endgame safety
When AI reaches the endgame, safety forks
Strong vs. Strong
One strong AI against another — cyber offense and defense, intelligence, war, financial infrastructure, nation-scale security. The currency is speed, force, game theory, suppression and counter.
Weak over Strong
A controllable weak AI supervising a stronger one — everyday work, life, transactions, and delegated agents. The currency isn't defeating the strong AI; it's keeping human control that stays understandable, reversible, and verifiable.
OpenGuardrails builds the second. Most people will never own a nation-scale adversarial system, but everyone will face agents that execute, reason, and persuade better than they do. Civilian AGI safety is not a shrunken war machine — it is the reinvention of the personal authorization system.
An ordinary workday, 2029
Your coding agent has already opened the PR. Your trading agent has already found the arbitrage window. Your life agent is handling insurance, a lease, and medical bills. Your research agent is summarizing papers you will never read line by line. You haven't lost the final say — you've lost the time and attention to know what each “yes” really means.
Here a safety layer can't just ask “allow or deny.” It has to know: does this drift from your long-run preferences? Does it exceed your risk budget? Is it exploiting your fatigue, greed, or fear? Is it dressing an irreversible action as a reversible one? Is it using a fluent story to cover missing evidence?
II · Weak intelligence as a human instrument
Weakness as an engineering constraint
We usually read “weak” as a defect. In AGI safety it can be a design choice. A brake need not be stronger than the engine; a fuse need not be more complex than the grid; a constitution need not be smarter than the society. What they share is a simpler, more reliable, more inspectable limit that holds before power runs away.
OpenGuardrails' weak AI is not a budget chatbot. It is a human instrument: it remembers your boundaries, audits the strong model's reasons, flags manipulative framing, breaks complex actions into approvable steps, and adds friction exactly when risk rises.
The point of Weak over Strong is not for the weak model to beat the strong one task by task. It is for the strong model to release its capability under human intent. A strong model may know more — but it doesn't natively know what is “good” for you. It may plan better — but it doesn't natively hold your risk preference. It can explain anything — but explanation itself can be a tool of control. So OpenGuardrails extends supervision from the answer to the action: who proposed it, on what basis, what evidence is missing, whether it is reversible, whether it needs a second opinion, whether a one-step authorization should become many, whether the human should be made to slow down.
III · A new aesthetics of trust
Trust is not “I believe it.” It is an institution.
Real trust is an institutional relationship: I know what it may and may not do; I know when it must stop; I know how we review its mistakes; I know it cannot use capabilities I don't understand to route around my authorization.
Controllable over capable
The supervisor need not be smart. It must be configurable, explainable, stoppable, and replaceable.
Action over text
The real risk isn't what the model says — it's what it makes tools, accounts, and other agents do.
Friction is a right
Safety isn't eliminating every pause. It's protecting your right to regret before an irreversible action.
Persuasion is an attack surface
When an AI explains better than you can, the explanation itself has to be supervised.
Generalize from weak to strong
The supervisor doesn't copy every human mistake. It learns the underlying intent and fires when the strong model tries to cross a line.
Live in the experiment
OpenGuardrails' own workflows, trades, and agents are supervised by OpenGuardrails first. Safety is not a promise outsourced to the user.
IV · Company will, aligned to the mission
Owned so the mission can't be bent
OpenGuardrails is registered as a solo LLC: funded by the founder, who holds 100% of the shares. The purpose of that structure is that OpenGuardrails does not have to treat shareholder profit maximization as its final goal — it can align the company's will with the mission of AGI endgame safety as directly as possible.
It runs more like a public-benefit company, prioritizing research that helps society over treating commercialization as the end in itself. Depending on how things develop, it may convert to a public-benefit corporation.
Called by the mission, we hope more capable people will join — anywhere, any time, in any form, in any state — to contribute to AGI endgame safety. Within the next six months we will seek the right domain experts to form a social-responsibility committee, to keep OpenGuardrails accountable to these commitments.
To sustain and grow the work, we accept direct donations and also take on commercial projects; all of that income goes to AGI endgame safety. OpenGuardrails is a US-registered organization, compliant with US law, serving civilian companies, civil-society organizations, and individual consumers worldwide.
V · What OpenGuardrails is building
The seatbelt AGI must wear before it enters ordinary life
OpenGuardrails is not another model provider, not another chat assistant, not a traditional enterprise gateway. It is the authorization and supervision layer that lets real work be handed to AI.
Personal intent kernel
Encode your long-run preferences, risk budget, identity boundaries, financial limits, moral lines, and “what I usually regret” into a supervision protocol a weak AI can run.
Pre-action audit
Before a strong AI calls a tool, moves money, trades, publishes, deletes, signs, authorizes, or affects others, OGR weighs reversibility, evidence quality, over-reach, and manipulative framing.
In-action rate limiting
Staged authorization, time locks, amount caps, counterfactual checks, and second-supervisor review on high-risk actions — so a strong AI can't quietly compose many small steps into one irreversible event.
Post-action evidence ledger
A record of what the strong AI did, why, on what basis, which objections it ignored, and which risks it accepted. No traceable action, no real trust.
Weak-supervisor network
One weak supervisor can be wrong. Several — with different biases, rules, and risk models — form low-cost mutual checks. Personal safety becomes a small institution, not a single point of judgment.
The AGI endgame is not “humans vs. AI.” The endgame question is: can every person have an agent weak enough — and therefore controllable enough — to hold their authorization boundary for them?
When strong intelligence begins to act on a person's behalf, the unit of safety is no longer the model. The unit of safety is authorization.
We don't wait for a perfectly aligned world. We build a supervision layer an ordinary person can understand, configure, and rely on today. Let strong AI do what strong AI is good at. Let weak AI do what humans can control. Let people hand real work to AI with confidence — while keeping the final right to judge, to regret, and to undo.