How to govern AI agents in a small business

The week your agents start handling intake, drafting estimates, and routing exceptions, you inherit a new job: deciding which rules that system runs under, who is allowed to change those rules, and how you would know if the system stopped matching your actual business. That job is governance, and most versions of it in small companies fail at the same moment. An artifact gets produced, a policy, a quality score, an approved workflow, and because the artifact exists, it starts getting treated as authority. Emad Mostaque’s The Last Economy has a chapter (his fifth) I keep coming back to on this: objections are part of the build, not commentary after it. A policy, metric, or model should not get authority until it has been forced through the hardest hostile case already visible. This essay is my attempt to turn that rule, and several adjacent ones, into a sequence an owner can actually run.

No authority without a stress test

The useful objection is almost never hypothetical. It is already on the floor. Maybe your claims bot performs well overall but fails on the billing disputes where proof duties matter. Maybe a staffing metric rewards speed while the hardest jobs keep landing on the same senior tech. Maybe a compliance policy assumes a reviewer will catch escalation failures while the queue already shows nobody owns that review window. Each is a visible contradiction between the control surface and the operating reality under it, and “we should red-team this later” is not a governance rule, it is a delay.

So the promotion rule I use: test the metric against the edge case it currently hides. Test the policy against the queue or handoff it assumes away. Test the model against the business exception that would make a confident answer dangerous. Test the review path against current staffing, not ideal staffing. If the contradiction holds, no authority.

The mistake is expensive because authority compounds. Once a metric drives bonuses, people optimize to it. Once a policy becomes the approval gate, operators route around its blind spots. Once a model embeds in the workflow, its failure pattern becomes the training environment for everyone downstream. And what should come out of a passed stress test is not a stamp that says “AI approved.” It is something narrower: a policy with an explicit failure lane and escalation rule, a metric that shows where it stops being trustworthy, a model gate that names the decision classes it may not make alone.

Keep two clocks

Mostaque names a second structural mistake in Chapter 13: the fast loop that competes inside the current game should not be trusted to rewrite the game itself. In a small business, the game is your operating contract. What counts as acceptable proof. When a customer gets a refund. Which discount is allowed. Which exception can skip review. Your agent loop should absolutely be fast inside those rules. It must not quietly rewrite them.

The drift is rarely dramatic. An agent sees repeated urgency around late jobs and starts auto-approving overtime. A service lead notices close rates improve when estimates skip a verification step and quietly leaves the step out. A billing team finds cash collects faster under a looser documentation rule and starts treating the shortcut as normal. The dashboard goes greener at roughly the rate governance gets weaker, and no single moment looks like a decision.

The design answer is two clocks. The fast clock runs live work: triage, drafting, routing, follow-up, exception recovery. The slow clock changes the contract: proof requirements, customer promises, payout rules, QA thresholds. When the live system keeps hitting the same exception, it should flag the pressure, attach examples, propose a rule change, and route the decision to a review lane owned by someone with authority over the business contract. In a small company that is usually you, with one weekly review block. The boundary is short enough to memorize: the execution system can propose; it must not silently normalize.

Make cooperation the winning move

Before writing any policy, look at what the workflow actually pays for. Mostaque’s game-theory lesson is that cooperation is stable when the work remembers what happened, keeps shared commitments visible, shows whether the outcome was real, and makes future interaction matter. Betrayal wins when the loop is one-shot, opaque, and forgetful. A delegated workflow that drops context each run rewards confident improvisation. Private approvals reward local expedience. Hard-to-retrieve evidence rewards narrated success over proof. If your workflow has that shape, cutting corners is the rational move for people and agents alike, and no policy page will out-argue the payoff structure.

The fix is structural, not motivational. Put durable memory in the loop. Make the commitment explicit before execution. Attach proof to the result instead of to a separate reporting ritual. Keep the next operator, reviewer, or customer close enough to the record that bad behavior creates visible cost.

Governance is layout work

Chapter 17 of the same book gives the test I trust most: healthy governance reshapes the route. It changes defaults, opens trusted channels, adds friction where capture is forming, and prunes stale routes that keep power concentrated after the original reason is gone. A rule that leaves the old shortcut as the fastest path has not governed anything. When the default draft can still ship without source review, and the only real escalation path is “ask the owner,” the rule exists and the path ignores it.

The moves are concrete. An intake form now requires the source artifact before an agent can open a customer case. A routing board exposes a named review lane instead of letting every ambiguity land in chat. A pricing workflow blocks unsourced changes by default. An old override path gets deleted once the team trusts the new standard. The review question that follows: stop asking only what rule you wrote. Ask which path became easier, which path became harder, and which stale path you removed.

Stay in the alignment layer

Once agents make output cheap, the tempting way to stay in control is to review everything. That keeps the supervision cost of the old business while giving up the ownership work. Mostaque’s Chapter 15 argument is that when intelligence gets cheap, the scarce thing is the objective function: someone still has to decide what the system is for, which values it protects, which tradeoffs it refuses, and which exceptions deserve real judgment. If you spend the day clicking approve on agent output, that scarce input is being spent on output sanitation.

Keep yourself responsible for four things that compound: setting the operating target, defining protected commitments, deciding the real exception classes, and changing the playbook when the pattern is stable. Then make the judgment stick. When a discount exception is approved, the reason should become policy or stay visibly rare. When a callback is reclassified, the distinction should become teachable. A system that consumes your judgment without preserving it will need you forever, which is founder dependence with better tooling.

Demand governed work, not activity

The platform world is rebuilding around this same requirement, which is worth knowing before anyone tells you it is theory. GitLab announced a restructuring alongside its agentic engineering thesis: fewer countries with small teams, fewer management layers, R&D reorganized into roughly 60 smaller teams, and internal reviews, approvals, and handoffs rewired with agents. In GitLab Act 2, CEO Bill Staples puts the requirement in one line: “Enterprises don’t need agent activity. They need running software that moves the business forward.” The company’s five architectural bets, machine-rate infrastructure, lifecycle orchestration, connected context, governance in the core, and one platform across human-owned, agent-assisted, and agent-autonomous work, describe a work system around agents, not a smarter chat window.

Take the requirement and skip the restructuring. A small business usually cannot compress its way into competence; it is already thin. The missing asset is not another person to remove. It is the governing surface around the work: named workflows, visible rules, evidence trails, exception queues, and the authority to improve the method. Your plumbing company does not need an agent that chats about dispatch. It needs the right job assigned, the exception surfaced, the photo evidence attached, the invoice checked, and the callback risk reduced, with a record that proves each of those happened.

Preserve value, let chatter expire

Agents produce coordination text all day: recaps, status pings, handoff drafts, suggested plans. Chapter 18 of The Last Economy argues a healthy system needs one surface for preserved value and another for circulation, and I think that split becomes urgent the moment coordination gets cheap. Approved rules, customer promises, signed decisions, playbooks, and contracts belong on the value surface, each with an owner, a location, and a review path. Status pings and generated recaps belong on the coordination surface: cheap to create, easy to discard, and clearly non-governing unless promoted across an explicit review boundary.

Preserve everything and you get memory inflation, where the next operator searches for the customer rule and finds six summaries, three recaps, and one actual approved answer. Preserve what the next owner must trust; let the rest circulate and disappear.

Keep an outside witness in the loop

There is now a research result that names the failure mode underneath all of this. A Physical Review Letters paper, Lost in Retraining: Closed-Loop Learning and Model Collapse in Exponential Families (public preprint on arXiv), shows that in exponential-family statistical models, closed-loop training on model-generated data collapses. Adding one data point from outside the loop, or a prior from previous knowledge, prevents the collapse, and even an infinite volume of machine-generated data does not erase that outside point. King’s College London reported it as a way to overcome AI “data cannibalism”; the team found similar evidence in Restricted Boltzmann Machines and plans to test larger networks. The result comes from simple models rather than production LLMs, so treat it as a principle, and the principle is enough: a closed loop needs an outside witness.

Your agent stack runs near the same failure. A summary drops a strange exception because it looks like noise. The next summary reads the omission as absence. A proposed category reads both and decides the normal case is the whole case. Picture an agent that sees repeated warranty notes and proposes one clean bucket, WarrantyCallback. That bucket may hide five different business facts: a workmanship failure, a manufacturer defect, a customer access problem, goodwill work done to save an account, and a sales promise that never should have reached dispatch. The distinctions live in real records, the job where you refused to bill because the sales promise was bad, the photo packet showing the part failed rather than the tech.

So the rule I run: no generated summary, dashboard category, or playbook change becomes governing memory unless the source anchor stays inspectable. This invoice, this dispatch note, this payer manual, this customer complaint. A summary can propose. A source-backed case can constrain. A review gate decides. Provenance is not a citation ornament; it is the control that keeps the agent layer attached to your actual business.

Separate finding answers from shipping them

One more boundary worth keeping explicit. Mostaque’s cathedral-and-bazaar framing separates exploration from delivery: the bazaar keeps alternatives alive long enough to find something better, the cathedral makes a chosen method disciplined enough to carry real work. A single agent loop asked to research the question and ship the answer will optimize for speed, and contradictions start looking like delay. It ends up executing the last plausible answer with better formatting. Keep a discovery record (sources checked, competing explanations, open contradictions, rejected paths and what would reopen them) separate from the execution commitment (chosen path, owner, review gate, acceptance evidence, and the condition that sends the question back to discovery).

Start with one protected seed

Do not roll this out as a company-wide program. Our judgment, carried from the same reading: a governance idea is unfinished until one bounded workflow carries it under real pressure. Pick a boundary small enough to govern: one workflow, one customer promise, one decision class, one operator role, one approval path. Protect it with rules that survive contact without mutating: source requirements, budget limits, customer-claim review, rollback paths, an exception queue, and a visible owner veto. Judge it on judgment transfer rather than activity: can the next person see why, find the source, and inherit the stop rules? When it works, scale it by imitation. Write the export packet, the governed problem, the boundary, the rules, the health signals, the failure modes, and what a copy requires, and let the next workflow copy the seed instead of receiving a manifesto.

One honesty note to close. The Last Economy argues at macro scale, the collapse result is proven in simple statistical models, and GitLab’s evidence comes from enterprise software; nobody has validated this assembled playbook by controlled comparison, including me. The pilot is the honest test. Run the sequence inside one bounded workflow, keep the parts whose absence shows up in the pilot’s own records, and drop the parts that only produce paperwork.

---

Source evidence used in this note: Emad Mostaque, The Last Economy, for the arguments that objections belong in the build before authority (Chapter 5), that the fast loop should not rewrite the game it competes in (Chapter 13), that governance works by reshaping defaults, channels, friction, and pruning (Chapter 17), that cheap intelligence makes the objective function scarce (Chapter 15), that systems need separate surfaces for preserved value and circulation (Chapter 18), plus the intelligent-game-theory and cathedral-and-bazaar lessons. GitLab Act 2 by Bill Staples for the restructuring facts, the governed-work quote, and the five architectural bets. King’s College London, Scientists come up with way to overcome AI ‘Data Cannibalism’, and the Physical Review Letters paper Lost in Retraining (arXiv preprint) for the model-collapse result and its scope. Hadto interpretation: the promotion rule, the two-clock design, the layout test, the owner functions, the source-anchor rule, and the protected-seed pilot are operating judgments translated from these sources and not validated by controlled comparison.