Lab notes

Lab notes · 02 · August 2026

When in doubt, the agent doesn’t pay

Before an AI agent pays for an API call over x402, a gate checks the price and payee the service asks for against its listing and the rules the buyer wrote. When the evidence is missing, the gate never says yes, and the agent should not pay. In fixture tests on 40 synthetic providers, 35 of them built to misbehave, it denied every one whose live terms drifted from its listing and allowed all five clean controls. Every answer comes with a signed receipt that anyone can check offline in Go, Python or TypeScript. It is a test build; nothing is live yet.

By Robert Balas, Redcom Software

Status
Test build: synthetic providers and my own services only; no users, nothing deployed
Answers
allow, deny or inconclusive, with reasons and a signed receipt; the gate does not pay the endpoint
Default
An empty policy never returns allow
Corpus
54 listings: 40 synthetic providers (35 hostile, 5 clean), two of them listed twice, and 12 of my own services
Tests
At the 31 July release review: 178 Go tests (167 unit and contract, 11 on Postgres 17) and 89 client tests
Built
29 July to 6 August 2026, 16 commits; Go, with Python and TypeScript verifiers
Figure 1 · One search over the test corpusOf the 23 listings one buyer’s policy matched, the gate allowed 3. Each square is one listing in the test corpus, most of them written to misbehave. Two never reached storage. A search with one buyer policy matched 23 of the 52 stored: the gate denied 18, left 2 inconclusive and allowed 3. These are the counts the release review recorded; it did not check each answer one by one.

The problem

x402 is a way to charge for a single API call inside HTTP. The service answers a request with 402 Payment Required and its payment terms. An agent with a wallet signs a payment, usually a stablecoin transfer, and sends the request again; the service hands the payment to a facilitator, a third party that checks it and settles it. Before paying, the agent has two descriptions of the service: the listing the service published in a catalogue (price, payee, output schema, claimed usage) and the terms in its 402 challenge. The protocol does not check one against the other. Client libraries let an agent cap the amount and choose which networks it will pay on, but none of them compares the challenge with the listing, and the payee and schema are left to whatever the agent does with them.

The two can disagree, and the listing can be wrong on its own. The live challenge may ask for a different price, payee or output schema from the catalogue entry. A listing may point at an endpoint that no longer answers, or its text may be written to instruct the agent rather than to inform a person. An agent that trusts the listing has no way to notice.

I built a gate that sits before the payment. Given the policy the buyer wrote, it decides whether this agent should pay this endpoint now. To decide, it sends the endpoint an unpaid request (the probe), reads the 402 challenge that comes back, and compares those terms with the listing and with the policy. Deciding does not pay the endpoint, which means the gate cannot check what a paid call actually returns. Evidence of that could only come from a separate test purchase, which the buyer must authorise and which this build can make only against a fake facilitator.

A policy is a list of hard rules: a price ceiling, approved networks, payees and facilitators, required output fields, the minimum evidence required (declared, observed by the probe, or verified by a test purchase), the maximum age of an observation, permitted side effects, and explicit deny rules. The gate applies them and adds no discretionary judgement of its own. It advises; it does not hold the wallet. Whatever holds the signing key has to refuse to pay without an allow, and if the gate cannot be reached there is no allow, so nothing is paid. The gate itself is priced at about a cent per decision. In this build that charge is switched off; the charging code has run only against a fake facilitator, in a separate test build.

What it returns

Every decision is one of three answers, with the reasons that produced it and a signed receipt.

Answer Meaning
allow the endpoint passes every rule in the buyer’s policy, and the gate’s own checks find that the live amount, asset, payee, network and schema match the listing
deny at least one rule or check fails; the reasons name which and, where values are compared, show the expected and observed value side by side
inconclusive the policy is empty, or a rule or check cannot be completed because a value is unknown; the agent should stop or ask a person

A listing that declares a schema gets inconclusive if the live challenge states none, because the declared schema was never confirmed. Beyond its fixed comparison of the listing with the live challenge, the gate checks only the rules the buyer wrote, and a policy with few rules lets more endpoints through. When the gate denies an endpoint it can suggest alternatives. Those are judged on stored evidence without a new probe, and an agent should ask for a full decision on one before paying it.

Three design choices shaped the rest of the gate.

  • An empty policy never returns allow. At best it returns inconclusive. If the buyer has written no rules, none of the buyer’s limits has been checked, and the gate does not treat that as permission. This is what failing closed means here.
  • No trust score. A single score from a third party cannot be audited and does not say which rule failed. The gate returns a reason for every rule that failed or could not be checked, the number of rules it evaluated, and the observed attributes. When a buyer searches for candidates, the gate also ranks the ones that passed, in a fixed order: most checks passed on fresh evidence first, then lower price, then lower probe latency, then more independent observations. The ranking can be recomputed byte for byte from the stored observations, the policy version and the algorithm version.
  • Paid placement cannot move the ranking. Catalogues can mark a listing as sponsored. Here sponsorship is only displayed and never reaches the ranking, and one test checks it in two ways. It inspects the data the ranking sorts on and confirms there is no field for sponsorship, provider or relationship. Then it sponsors four candidates in all 16 combinations and checks that the order never moves.

An adversarial test corpus

There is no live catalogue behind this yet, so I built a test corpus instead: 40 synthetic providers, 35 of them written to misbehave and 5 clean controls, plus catalogue entries for 12 of my own services under one provider. Two of the hostile providers each publish a second listing, which makes 54 listings in all. Among the flaws it covers:

  • a probe target that redirects to the cloud metadata address, where a cloud server hands out its own credentials;
  • listing text that tries to instruct the buying agent;
  • an output schema nested 60 levels deep;
  • a live price, payee, network or schema that differs from the listing;
  • two providers that list themselves twice, one under the same identity at a higher price and one under a second resource id on the same route;
  • a listing published over plain HTTP.

On the release build, the corpus goes in like this:

Stage Count
Declared listings 54
Normalised (one rejected: plain HTTP) 53
Stored (one identity declared twice; storage keeps one set of terms per identity) 52
Eligible (the 60-level schema, the identity declared twice, now marked ineligible, and invalid payment metadata removed) 49, across 37 providers

A search with one buyer policy matched 23 of the 52 stored listings. It allowed 3, denied 18 and left 2 inconclusive. The denials include endpoints whose live challenge asked for a different schema or payee from their listing, which only a probe at decision time finds. The release review records these search counts without checking each answer. The unit tests do check fixtures one by one: for each kind of flaw they cover, the fixture must produce the reason code it was built to trigger, each drift fixture must be denied, and all five clean providers must be allowed. Those tests caught an early bug. The probe hashed a different part of the schema from the catalogue reader and denied all five clean providers, until I made both hash the same part.

One provider in the corpus advertises 980,000 calls in 30 days, of which 979,500 are its own test traffic. The ranking does not read declared usage at all.

The probe may contact only allow-listed hosts. Each host is resolved once and the connection is pinned to that address, which must fall inside an allowed range. In the released configuration the only allowed host is the test lab, and the allowed ranges are private addresses only. Redirects are never followed, nothing is decompressed and the body is capped at 64 KiB. A test asserts that a redirect into the metadata address is not followed.

The receipt

A decision is only useful later if someone can check what was decided, and on what evidence, without asking me. Each receipt binds the answer to the exact observations, the hash of the policy document and an expiry of one hour. Expiry can be checked offline. A later change to the price, payee, network or advertised schema of an endpoint the receipt covers also voids it, but only a new observation can detect that. Before paying, the caller has to compare the receipt with the 402 challenge in front of it.

Receipts are signed with Ed25519, and the service publishes its public key. They verify offline in Go, Python and TypeScript with nothing beyond each standard library. The Python client carries its own RFC 8032 verifier, because Python’s standard library has no Ed25519. Canonicalisation is the hard part: each verifier has to rebuild exactly the bytes that were signed, and one byte of difference makes it reject a valid receipt. One test receipt, signed in Go, is checked by the Python and TypeScript clients, and a contract test regenerates it and fails if a single byte changes. That test receipt deliberately contains <, & and ", which standard JSON encoders escape in different ways.

What the release run caught

The release run starts from an empty database and runs, in order: formatting and lint checks, 167 Go unit and contract tests (again under the race detector), 89 client tests, the image build, 11 Go tests against a real Postgres 17, smoke tests of the running service and its three clients, a 13-assertion customer acceptance test run twice, a restart, a backup and restore, and a final audit of the running containers. Runs on 29 July found five defects. At the release review on 31 July it passed from start to finish.

  • The acceptance harness had never run. Its build image pinned an older Go than the module required, so no container had ever built. Every later finding came after that fix.
  • Two defects meant no payment could be built at all. The customer binary could not read a catalogue declaration, and payment identifiers were shorter than the payment library’s minimum.
  • A buyer who had already paid the gate for a decision was asked to pay again. The gate’s own payment challenge was issued before the lookup that recognises a request already settled. The server now looks for a settled request right after authenticating the caller and before asking for payment.
  • A test purchase nobody asked for was downgrading decisions. The canary, an optional test purchase that is off by default, reported its absence as missing evidence, so a search called an endpoint allow while a single evaluation called the same endpoint inconclusive. Its absence now counts only when the buyer asked for it, and a test asserts that the two routes agree.

I also reported one of those runs as passing when it had failed. It had stopped at the integration tests, because two contract tests read files outside the Docker build context. I had written make release-bar > log; echo "EXIT=$?". The line it printed showed the status of make. The command as a whole exited with the status of echo, which is always 0, and I reported that one. The release run is now started as one pipeline under pipefail, and the review takes the status of make from PIPESTATUS.

What it is not, yet

  • Fixture only. It runs against the synthetic corpus. Live catalogue access is not built. The binary refuses to start if the data mode is anything but fixture, if settlement is switched on, or if a wallet credential is in its environment. The one path that pays an endpoint, the canary (a capped test purchase that the buyer must authorise), is off by default and can only pay a fake facilitator, so it moves no value. Receipts are signed with a fixture key derived from configuration and marked as such; a production key, mounted and rotated rather than derived, is not built.
  • No users. There is no design partner behind it, and nothing here is evidence of demand. The claim it tests is that a buyer will pay about a cent for a decision that can be reproduced, rather than trust the seller’s listing. The claim is wrong if buyers never write a policy, if they read the reasons and still call a person, if listings change so rarely that a cached catalogue does the job, if buyers want a service that pays for them and refuse one that stops short of payment, or if real providers fail in ways the test corpus does not model.
  • Small and unreviewed. Sixteen commits between 29 July and 6 August 2026, nothing deployed. The asset comparison and the inconclusive answer for a challenge with no schema were added after the release review, with unit tests; the full release run has not been repeated since. There has been no outside security review, and no one outside the project has tested it.

Where else it applies

I expect the same design to carry over to other software that acts for someone who must answer for it later, such as an agent that approves a refund or books a supplier. The owner writes the policy, each decision states its reasons, and anyone with the public key can check what was decided, and on what evidence, without asking the system that made it. When the evidence is missing, the default is to stop and ask.