BLOG

🌱 digital garden

Basé à Nice, France - CET (UTC+1)

BLOG

🌱 digital garden

Basé à Nice, France - CET (UTC+1)

BLOG

🌱 digital garden

Basé à Nice, France - CET (UTC+1)

Your sceptic agent can read the answer

Running a second model to challenge the first is weak evidence if both share a filesystem. Notes on making agent isolation a property of the kernel, not of the prompt.

A pattern keeps recurring in useful agent work. A script gathers data. Several agents analyse it in parallel. Then a second agent, the sceptic, tries to break each conclusion, and a rule fixed in advance decides what survives.

The third step is the one everybody likes to draw on the whiteboard, and it is usually fake. I drew it that way myself before I checked what the second agent could actually read.

Two agents agreeing is not evidence

If the sceptic runs in the same working directory as the analyst, it can read the analyst's notes, scratch files and reasoning. Models are agreeable readers. Shown a confident argument, they tend to check it, not attack it, and "I verified the analysis and it holds" is the cheapest possible output.

You can write "do not read the other agent's files" in the prompt. That is a request. An agent with a Read tool and a directory listing will find analysis_notes.md, and the more capable the model, the more thoroughly it explores. The property you want (the sceptic has never seen the reasoning) cannot be established by asking nicely. It has to be true of the machine.

Make it a filesystem fact

cadence is a workflow engine I wrote around this one requirement. Every step item gets a fresh directory tree:

<run>/<step>/<item>/
    work/      the process working directory
    work/out/  where the step writes results
    meta/      stdout, stderr and metadata, owned by the engine
<run>/<step>/<item>/
    work/      the process working directory
    work/out/  where the step writes results
    meta/      stdout, stderr and metadata, owned by the engine

A downstream step never reads an upstream directory. It declares inputs:, and the engine copies exactly those values into its work/ before the process starts:

- id: challenge
  needs: [analyse]
  for_each: '{{ .steps.analyse.items }}'
  isolate: true
  inputs:
    report.md: '{{ .item.result }}'
  agent:
    tools: [Read]
- id: challenge
  needs: [analyse]
  for_each: '{{ .steps.analyse.items }}'
  isolate: true
  inputs:
    report.md: '{{ .item.result }}'
  agent:
    tools: [Read]

The sceptic's world contains report.md and nothing else. There is no analyst reasoning to find because it was never put there. Information flow is the data flow graph, and the graph is in a YAML file a human can review.

meta/ sits beside work/, never inside it, so a step cannot read the engine's record of itself, which includes the full stdout of its own earlier attempts.

Then make the kernel enforce it

A tidy directory layout stops accidents. It does not stop a process from running cat ../../analyse/0/work/notes.md. So each step runs under an OS sandbox: a generated sandbox-exec profile on macOS, bubblewrap with a fresh mount namespace on Linux. The profile denies by default. Writes are allowed in the step's own work directory only, and reads there plus the system paths a process needs in order to start. A neighbouring jail is not hidden. It is unreachable.

The tests for this do not inspect the generated profile and trust it. They run a real process in a real jail and assert that reading a sibling's file fails:

func TestEnforcedBackendBlocksReadingNeighbourJail(t *testing.T)
func TestEnforcedBackendBlocksWriteOutsideJail(t *testing.T)
func TestEnforcedBackendBlocksReadingNeighbourJail(t *testing.T)
func TestEnforcedBackendBlocksWriteOutsideJail(t *testing.T)

A sandbox you have not tried to escape is a configuration file, not a sandbox.

Build the environment, do not inherit it

Files are one channel. The environment is another, and it leaks more quietly. A child process normally inherits everything in the parent's environment, including every API key in your shell.

In cadence a step gets an allowlist (PATH, HOME, TERM, LANG, LC_ALL, TZ, SHELL, TMPDIR) plus whatever its own env: block names. TMPDIR points inside the jail, so scratch files are private too. A shell step that formats JSON has no reason to hold a cloud credential, and now it cannot.

Say so when you cannot enforce it

Sometimes neither mechanism is available. The engine then falls back to a backend that sets the working directory and environment and nothing more. That is a real loss, and the worst thing a security feature can do is degrade silently, so the type that represents a sandbox has to answer one question:

// Enforced reports whether the kernel actually prevents escape. A false
// here is a real degradation and is surfaced to the user, never hidden.
Enforced() bool
// Enforced reports whether the kernel actually prevents escape. A false
// here is a real degradation and is surfaced to the user, never hidden.
Enforced() bool

cadence doctor prints the active backend and what it does not do. A workflow can set require_sandbox: true and will refuse to run instead of degrading. I would rather a scheduled job fail loudly at 7 am than produce a "challenged" result that was never challenged.

The honest limitation

On macOS the agent CLI keeps its credentials in the keychain, which is not reachable from inside a seatbelt jail. So an agent step, under the strictest profile, cannot log in. There is an explicit override that forces the unenforced backend, and the tool reports when it is in effect. I do not love this. It is written down in the README because a sandbox with an undocumented hole is worse than one with a documented one.

What this is really about

Multi-agent systems borrow the vocabulary of organisations: analyst, reviewer, sceptic. Organisations learned long ago that independent review requires actual independence. Auditors do not share an office with the people they audit.

With models the equivalent is cheap. It is a directory, a mount namespace and forty lines of profile. I do not know yet how much better the conclusions get once the sceptic is really isolated. I have not measured it, and I would like to. What the isolation requires is deciding that "the second agent did not see the first agent's reasoning" is a claim your system makes, and then asking the question you would ask of any other claim: what enforces it?

This article was co-authored with AI.