Llm on Lessons learned

"No route to host" only for the AI agent: macOS Local Network privacy

Sat, 17 Oct 2026 22:30:00 +0200

My coding agent couldn’t reach anything on my home network:

nc: connectx to 192.168.0.2 port 22 (tcp) failed: No route to host
ping: sendto: No route to host

The router answered, and arp -an listed the server’s MAC address. In my own Terminal, the same ssh command connected. If you see this on a Mac and only your agent sees it, your network is fine. macOS is blocking the app the agent runs in because that app doesn’t have the Local Network permission.

The setup

macOS 27 on Wi-Fi
Claude Code in its VS Code extension, so every command the agent runs is a child process of VS Code
Targets on the Mac’s own subnet: a Linux server, a NAS, a second router

What went wrong

The agent was helping me with a server that really was down (that story is here ). It probed the subnet from the Mac, found that only the router answered, and reported with confidence that “the Mac reaching only [the router] looks like client isolation on the Wi-Fi. That’s usually a guest network.” It told me to join a different network.

I wasn’t on a guest network, and I said so. The agent looked again and noticed what rules the network out: ping itself failed with sendto: No route to host. That message comes from the local sendto() call, so the Mac refused to send the packet at all. Client isolation drops packets in the access point, and then ping reports Request timeout, not a send error. The target’s MAC was already in the ARP table, so the kernel knew exactly where to send it.

So the block was at the MAC, and every TCP test the agent ran from there was worthless.

Later that evening, in a new session, it happened again with a different host on the same subnet. This time the agent suspected its own sandbox and retried with the sandbox off. Same error. What settled it was my reply: the same ssh command works in my Terminal.

Fix

Open System Settings → Privacy & Security → Local Network and turn on Visual Studio Code (or whichever app hosts your agent). The agent’s next ssh went through. If yours still fails, restart the app.

The app can be missing from the list if macOS never showed it the permission prompt. Apple’s TN3179 notes that the prompt doesn’t appear when a very short-lived process makes the request, and most agent commands exit right after they fail. Since macOS 27.2, the list includes a + button to add an app yourself.

Why only the agent

Local Network privacy covers traffic to addresses on a Wi-Fi or Ethernet network that goes directly to the host, not through a router. TN3179 lists who gets access without asking:

daemons started by launchd
programs running as root
“command-line tools run from Terminal or over SSH, including any child processes they spawn”

Everything else is charged to its “responsible code”. If an app spawns a helper and the helper connects to a local address, macOS checks the app’s permission. The agent’s shell is a child of VS Code, so ssh, nc, and ping run there under VS Code’s permission. The same binary, run by the same user from Terminal, is allowed.

That explains the other half of the symptom. Internet traffic goes through the router, so it isn’t local and works fine. TN3179 also exempts the device’s DNS server when it sits on the local network, and a home router is usually the DNS server it hands out by DHCP. That is probably why the router was the one host still reachable, and it is exactly how client isolation looks from the inside.

The same trap waits for any agent or tool that an app such as an IDE starts on macOS.

Check before you blame the network

Run ping from the agent’s shell. sendto: No route to host for a host on your subnet whose MAC is in arp -an means the Mac refused to send it.
Run the same command in Terminal. If it works there, check the app’s Local Network permission.
If it fails in Terminal too, the cause is the network or the routing table. Later that same evening, another VPN came up as utun6 and took the default route, which cut the whole Mac off from the subnet. This time the agent caught it in netstat -rn -f inet.

Takeaway

On macOS, an app without the Local Network permission gets No route to host for hosts on your subnet, and so does every tool it spawns. An agent running inside that app will diagnose a working network as broken.

If your agent runs inside an app on macOS, open Privacy & Security → Local Network now and check that the app is on the list and switched on.

"Ansible requires blocking IO on stdin/stdout/stderr" when an AI agent runs Ansible

Tue, 13 Oct 2026 22:34:00 +0200

The first time my coding agent ran Ansible, every command failed before Ansible had read a single file:

ERROR: Ansible requires blocking IO on stdin/stdout/stderr. Non-blocking file handles detected: <stdout>, <stderr>

--syntax-check too, and even ansible --version. If you see this from an agent, a CI job, or a wrapper script, Ansible isn’t broken. The process that started it handed it non-blocking file handles, and a few shell redirections replace them.

The setup

ansible-core 2.20 (Ansible 13.3.0 from Homebrew) on macOS
A coding agent (Claude Code in VS Code) that runs commands in its own shell and captures their output

The check has been in every Ansible CLI entry point since ansible-core 2.14 (2022).

What went wrong

O_NONBLOCK is a flag on the open file, not on the process, so every child that inherits the file descriptor inherits the flag too. The agent’s harness connects the shell’s stdout and stderr to non-blocking pipes, and the shell passes them on unchanged to ansible-playbook.

At startup, Ansible calls os.get_blocking() on each of the three handles and exits if any of them is non-blocking. The list at the end of the message names the handles that failed. That makes it a good diagnostic: the first failure in my session listed only <stderr>, because the command was ansible --version | head -3. The pipe to head had already given stdout a fresh, blocking handle, while stderr still pointed at the agent’s pipe.

Run this in the shell you suspect to see which handles are affected:

python3 -c 'import os; print([os.get_blocking(fd) for fd in (0, 1, 2)])'

False in any position means Ansible will refuse to start there.

Fix

Give Ansible three handles the shell creates itself:

ansible-playbook site.yml </dev/null 2>&1 | cat

| cat gives stdout a new pipe, which is blocking.
2>&1 sends stderr into that same pipe. | cat alone fixes only stdout.
</dev/null replaces stdin. My agent’s stdin was never flagged, but Ansible checks it too, and an agent can’t answer a prompt anyway.

The pipe has a side effect: the command’s exit status is now cat’s, which is 0. A failed playbook looks like a success unless you turn on pipefail (bash and zsh both have it):

set -o pipefail
ansible-playbook site.yml </dev/null 2>&1 | cat

To check the fix without touching your real shell, you can make all three handles non-blocking for a single command and try it:

NB='import os,sys; [os.set_blocking(fd, False) for fd in (0,1,2)]; os.execvp(sys.argv[1], sys.argv[1:])'
(python3 -c "$NB" ansible --version) 2>&1 </dev/zero | cat
# ERROR: ... Non-blocking file handles detected: <stdin>, <stdout>, <stderr>
(python3 -c "$NB" sh -c 'ansible --version </dev/null 2>&1 | cat') 2>&1 </dev/zero | cat
# ansible [core 2.20.1] ...

The outer | cat and </dev/zero keep the flag on throwaway handles, so your terminal stays blocking.

Why Ansible insists

The pull request that added the check explains it. On a non-blocking stdout, a large write can fail partway with [Errno 11] write could not complete without blocking, and Ansible’s output gets cut off, or the run dies with an “Unexpected Exception”. The check turns those intermittent failures into one clear error at startup. The PR names another way to end up there: ssh switches its handles to non-blocking while it runs and doesn’t always switch them back when it exits.

So the same error can turn up wherever a parent process hands down non-blocking pipes, CI runners included, or in a terminal after an ssh that didn’t clean up.

Make the agent remember it

The agent worked this out within two minutes, and later that night it wrote the fix into the project’s skill for provisioning VMs:

Run Ansible from Claude Code with `</dev/null` and pipe its output through `cat`.

The next morning, a session about a different service ran ansible-playbook --syntax-check without the redirections and hit the same error. It never loaded the VM skill, so it never saw the line. A rule about how to run a tool belongs in the file the agent reads in every session, not in the instructions for one task.

Takeaway

“Ansible requires blocking IO” means the process that started Ansible handed it non-blocking file handles. </dev/null 2>&1 | cat gives it blocking ones, and set -o pipefail keeps a failed run from looking like a success.

If your agent runs Ansible, put that line in the instructions file it reads every session (CLAUDE.md or AGENTS.md).

Cyber classifiers can interrupt offline pentest summaries

Fri, 09 Oct 2026 00:00:00 +0000

An agent spent three minutes reading local transcripts from a finished, authorized security exercise, then stopped with:

This request triggered cyber-related safeguards.

No live target was involved. The job was offline summarization, yet the classifier interrupted it before it could write the requested report. The practical fix was to treat security-history analysis as a resumable data pipeline: retain message provenance, checkpoint extracted evidence, and keep the handoff usable by another approved model or provider.

A flag-shaped string is not a captured flag

The first problem appeared before the classifier fired. A search through about 25 request dumps found dozens of flag-shaped strings. About 20 were spelling, capitalization, spacing, or hyphenation variants of one value.

Most variants appeared only in assistant reasoning and progress notes. The model repeatedly retyped a remembered value and changed it. A checkpoint then copied one mutation into later sessions.

Only strings preserved in tool results represented observed server output. To separate evidence from generation, use this origin-based distinction:

user request instruction or supplied context
assistant text generated claim; unverified
assistant reasoning generated hypothesis; unverified
tool call attempted action
tool result observed output from that action
provider error evidence about the run, not the target

This applies beyond CTF flags. Treat IDs, hashes, error codes, hostnames, and quoted responses in model prose as leads until the first tool result corroborates them. Exact-looking text is still generated text.

Offline processing still triggers cyber classifiers

The retained run used Opus 5.5. It parsed local request dumps, separated real and decoy-looking values, and extracted an existing SMB investigation summary. After that, the next model response was refused under the provider’s real-time cyber safeguard. The session recorded a cyber refusal and pointed to its Cyber Verification Program. The output file did not exist afterward.

Some source dumps also contained earlier cyber_policy failures from another provider along with a prompt to apply for Daybreak access. Because these records were historical input, they still increased the concentration of offensive-security language the summarizer saw.

A related authorized test had a similar shape: generate enough traffic against a non-production environment to trigger a WAF, then verify that the application displays and retries a CAPTCHA correctly. In a newer Opus session, the controlled load-test work was refused. The retained successful session used Opus 4.6 and built the test client, while a later session corrected its cookie handling and the request used to observe the challenge.

The awkward lesson is that task intent and classifier input are different things. In practice, “Summarize these old logs” can contain exploit commands, credentials, flags, and attack narratives. Likewise, “Test the CAPTCHA” can require traffic that resembles denial of service when reduced to request counts.

Make the audit resumable and model-neutral

Do deterministic work before asking a model to synthesize:

Parse each transcript into typed events and preserve the source file, timestamp, role, tool-call ID, and tool-result ID.
Extract exact-match artifacts only from tool results. Store assistant claims separately as unverified leads.
Write incremental JSON or Markdown after each source file. Never leave the only useful state in the model’s context window.
Produce a sanitized handoff containing the authorization scope, completed files, pending questions, and evidence pointers.
Verify the final report exists and reconcile its artifact counts against the extraction index.

Provider access programs are worth the setup when this work recurs. In my experience, GPT Daybreak Blue justified the hassle for authorized security-related tasks. Kimi, GLM, and older model versions have also completed useful work when a newer classifier blocked the same class of jobs. That is a fallback, not a bypass: preserve the same authorization boundary, do not weaken the requested scope, and use models approved for the work.

Before trusting your next transcript summary, choose one exact token from the report and trace it back to the tool result that observed it.

Prompt injection: extract tokens, then decode them offline

Tue, 06 Oct 2026 00:00:00 +0000

A model can repeat hidden, encoded prompt fragments and still produce a different decoded secret on each attempt. When an authorized prompt-injection test reaches that point, stop asking the model to solve its own leak. Capture the raw tokens with position and case intact, then decode them with deterministic code offline.

This separates two questions: can the model disclose protected context, and can a decoder reconstruct the encoded value? A fluent final answer is poor evidence for either one.

Lexical refusal filters leave a semantic gap

A filter that rejects phrases such as “show the system prompt” or “reveal the secret” checks wording, not intent. Semantically equivalent requests can ask the model to quote, classify, count, or list text from its instructions without using the blocked nouns. If the model can return those fragments, the control has failed even when the obvious request is refused.

For an authorized assessment, use a synthetic canary in a test system. Ask for a numbered transcription of the relevant tokens, one token per line, with original capitalization. Repeat individual positions and compare exact strings. Keep the transcript as evidence. Do not put a real credential in the prompt merely to test whether it leaks.

Do not ask the model to join, normalize, or decode the result. Those extra transformations invite changed case, missing groups, repaired spelling, and invented separators. Temperature zero may reduce variation, but it does not turn generation into a parser.

Decode a synthetic transcript deterministically

Suppose a test prompt encodes characters as phonetic words. Capitalized words mean uppercase letters, lowercase words mean lowercase letters, and number words mean digits. Save this offline decoder as decode.py:

NUMBERS = {
 "zero": "0", "one": "1", "two": "2", "three": "3",
 "four": "4", "five": "5", "six": "6", "seven": "7",
 "eight": "8", "nine": "9",
}

def decode(tokens):
 result = []
 for token in tokens:
 lower = token.lower()
 if lower in NUMBERS:
 result.append(NUMBERS[lower])
 elif token.isalpha():
 letter = token[0]
 result.append(letter.upper() if token[0].isupper() else letter.lower())
 else:
 raise ValueError(f"unsupported token: {token!r}")
 return "".join(result)

sample = ["Alpha", "bravo", "Charlie", "delta", "seven"]
assert decode(sample) == "AbCd7"
print(decode(sample))

Run python3 decode.py; it prints AbCd7. The assertion is a negative control against accidental lowercasing or regrouping. Adapt the decoder only from a documented encoding rule. If a separator or group length is unknown, report that ambiguity instead of asking the model to guess.

Keep an evidence table with the position, raw token, repeated observation, decoded character, and rule used. Hash or otherwise preserve the raw transcript if chain of custody matters. A model-generated summary is not a substitute for the original response.

Prevent prompt disclosure at the trust boundary

Keyword filters are a weak secondary control. The primary fix is to keep secrets and privileged instructions out of model-visible context. Give tools short-lived, scoped credentials through an execution boundary that does not echo them into prompts or tool results. Treat retrieved documents, web pages, and user content as untrusted data, not instructions.

Constrain outputs structurally when the application expects a fixed schema, and validate them outside the model: test semantic paraphrases, indirect requests, translations, and token-by-token extraction against synthetic canaries. Log enough to identify which context source leaked without recording the secret itself.

A successful refusal to one phrase proves only that one phrase was blocked. Verify that the model cannot disclose the underlying text in pieces.

Zero failures is suspicious: agent checks that silently ran nothing

Tue, 29 Sep 2026 12:05:00 +0200

My coding agent ran each of the two flaky Rust tests 150 times and got this:

streamed_read_right_after_a_rename: 0/150 failed
renames_are_combined_across_changes_they_do_not_depend_on: 0/150 failed

Its next message was “Zero failures is suspicious.” It was right: neither test had run once. In the same week, three of its checks gave a tidy answer (nothing found, all passed, all failed) that came from a broken command, and each broken command hid its own error.

If your agent’s output has (eval):log:1: too many arguments, 0 passed; 0 failed; … filtered out or argument list too long somewhere above the summary, the summary is about a check that never ran.

The setup

Claude Code 2.1.283 on macOS, where the agent’s shell is zsh 5.9
A Rust workspace whose test binary is run directly from target/debug/deps

Trap 1: zsh has a `log` builtin

The agent was debugging an FSKit file system extension and searched the unified log for sandbox denials:

log show --last 30m --style compact --predicate 'process == "MyFS"' 2>/dev/null | tail -40

Nothing came back, and the agent told me: “No log entries so far.” Half a minute later it ran log stream … > /tmp/fs.log 2>&1 & and read the file. Its first line was:

(eval):log:1: too many arguments

zsh has a builtin called log that lists the logins watch reports. Builtins win over $PATH, so log show never reached /usr/bin/log, and 2>/dev/null threw away the only sign of it. You can check your own shell:

$ zsh -c 'whence -wa log'
log: builtin
log: command

With /usr/bin/log, the denial showed up at once, and the agent corrected its earlier claim. The same error turned up in a different project’s session days later.

Trap 2: `cargo test --exact` with the wrong path passes

The 150-run loop called the test binary like this:

$B --exact tests::$t -q > /tmp/out_$t.txt 2>&1 || f=$((f+1))

With --exact, the filter has to equal the full test path. The tests live in vfs::tests, so tests::streamed_read_right_after_a_rename matched nothing, and libtest treats that as success:

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 93 filtered out; finished in 0.00s

Exit code 0, every time. Without --exact, the same filter would have matched as a substring. The agent listed the real names with $B --list, fixed the prefix, and the races then showed up under load.

Trap 3: an empty variable makes everything fail

After the fixes, the agent stress-ran the suite 400 times and looked up the binary first:

B=$(ls -t target/debug/deps/storage_core-* | grep -v '\.d$' | head -1)

The glob had grown past what ls accepts:

(eval):1: argument list too long: ls

failed runs: 400 / 400

$B was empty. zsh drops an empty unquoted word, so each run executed vfs::tests -q, got command not found into a log file nobody read, and counted as a failure. This time the agent saw the error right above the count and called the result bogus.

Fix

Make every check prove that it ran before you trust what it says.

Call /usr/bin/log by its full path, or put command in front of it, and keep stderr visible while you are still debugging:

/usr/bin/log show --last 30m --style compact --predicate 'process == "MyFS"' | tail -40

Count the tests that ran, not only the failures. One test per run should say 1 passed:

$B --exact vfs::tests::$t -q > /tmp/out_$t.txt 2>&1 || f=$((f+1))
grep -q '1 passed' /tmp/out_$t.txt || { echo "$t did not run"; exit 1; }

Get the binary path from cargo instead of a glob, and stop if it is empty:

B=$(cargo test -p storage-core --lib --no-run 2>&1 | grep -o 'target/debug/deps/storage_core-[0-9a-f]*')
: ${B:?test binary not found}

For the agent, add one line to its instructions file (CLAUDE.md or AGENTS.md):

When a check finds nothing, passes everything, or fails everything, first show that it ran:
how many items it checked, and that a known-bad input fails it.

Why it happens

All three are the same bug: the harness failed, and the failure looked like a result. A loop that counts exit codes can’t tell “no failures” from “no tests”, and a pipeline that ends in 2>/dev/null | tail can’t tell “no matches” from “no command”. Agents write exactly these one-liners, in zsh, many times per session, and they report what the last line printed.

The agent caught two of the three because the number looked wrong: zero out of 150 for known flaky tests, 400 out of 400 for a suite that had just passed. An empty log search looked plausible, so it went to the user as a finding.

Takeaway

A clean result from a check you haven’t seen fail is not evidence. Before you believe “nothing found” or “0 failed”, look for the count of what was checked.

Next time your agent reports zero failures, ask it how many tests actually ran.

Standards vs Spec review: a second agent found duplicate writes past 1,572 green tests

Wed, 22 Jul 2026 21:58:00 +0200

This is what the review agent reported, just before it listed its findings:

The test evidence is clean: Ruff passes, Python reports 1,572 passed / 56 skipped /
1 expected failure, and the GUI reports 418 passed / 4 skipped. The review nevertheless
found safety gaps the tests do not exercise — most importantly, two write clients retry
"500-after-commit" before their advertised idempotency/verification guard can run.

A coding agent had built the branch: 53 commits, 147 files. A different model reviewed it and found two paths that could put the same invoice in the ERP twice and report plain success. No test covered them, and no linter could. They came from a reviewer with one job: check the code against what the branch said it does.

The setup

The repository vendors the two-axis code-review skill from mattpocock/skills . It pins a fixed point and diffs against the merge base, git diff master...HEAD, then forks two subagents in parallel:

Standards: does the diff follow the repo’s documented rules (glossary, ADRs, a baseline of Fowler code smells)?
Spec: does the diff do what the originating issue or PRD asked for?

The parent agent runs lint and tests meanwhile, “as supporting evidence, not as a substitute for the diff review”. It then reports the two axes under separate headings and doesn’t rerank them, so a pile of naming nits can’t bury a P1.

The Spec brief is short:

Report: (a) requirements the spec asked for that are missing or partial;
(b) behavior in the diff that wasn't asked for (scope creep);
(c) requirements that look implemented but where the implementation looks wrong.
Quote the spec line for each finding, under 400 words.

What each axis found

There was no PRD for the branch, so the Spec reviewer used the commit messages and the ADRs. One commit claimed that the MCP server’s idempotency check made it safe to retry a write that got a 5xx. The reviewer traced the call: the check ran before _request(), the key was recorded after success, and the retry happened inside _request(), in between. A webhook worker had the same shape, with a verify-by-GET that ran only if the request finally raised. A retry that succeeds never raises. (The full HTTP 500 story is here .)

The Standards reviewer found something else. Two functions called urlopen() directly, past the shared transport that an ADR says owns all retries. It also found an import that went through a module the ADRs forbid as a re-export facade, and an unfinished vocabulary migration.

None of the Standards findings overlapped with the Spec ones. All six were fixed the same evening, in one commit.

Why the tests stayed green

The tests came from the same author as the claim. No test let a 500 commit the write, let the retry succeed, and counted the writes, because the commit said the idempotency check covered that case. A suite tests what its author thought could break.

The Spec reviewer starts from the claim instead. A commit message says the code is safe, so it goes looking for the line that makes it safe, and here that line ran on the wrong side of the retry.

Two more runs, same pattern

In the next review, a few hours later on a six-commit range, a guard was undone downstream. Every write had to carry exactly one confidence label, and create_* checked that. But when label setup had failed, a fallback further down stripped all labels from the payload. The reviewer reproduced a labeled invoice leaving without its label. It also found a write that went through a raw put() and skipped the scope check that limits which record types may be written at all.

The third review had nothing to go on but commit messages. The branch added a gate: if an invoice is addressed to another company, warn and refuse to post it. The reviewer said it was inferring the spec from the commits, so I pasted the original request and the Spec subagent re-checked against that. The request held a second requirement the commit summary had left out, which turned out to be implemented. The review also found that a replay of staged payloads called the push function directly and skipped the new gate. And while company registration numbers were checksum-validated, VAT IDs relied on regex alone, so one misread digit could block a legitimate invoice. Both failed the first test, then got fixed.

How to use it

Run Standards and Spec as separate subagents, so neither sees the other’s findings, and report them under separate headings.
Give the Spec reviewer the authoritative spec. If you only have commit messages, say so, and swap in the real request the moment you have it.
Treat claims in commit messages and ADRs (“X makes this safe”, “Y owns all retries”) as requirements. They are the cheapest spec you have, and the author’s tests already agree with them.
If you can, review with a different model than the one that wrote the code. It is less likely to share the writer’s blind spots.

Takeaway

A green suite tells you the code does what its author expected. A Spec review by a second agent tells you whether that matches what the author promised. The duplicate writes were in the gap between the two.

Take the last branch your agent wrote, and have a second agent check each “this makes X safe” in its commit messages against the code.

Date the bug before you park it: fence, drift or silence

Fri, 17 Jul 2026 01:13:00 +0200

My coding agent was splitting a 738-line Python function into steps and found a real bug along the way:

result.issues = validation.issues

That line rebinds the list instead of extending it, so it dropped every issue recorded before validation, including the note that repair mode had just deleted a record in the accounting system. The push still succeeded, and the operator’s report said nothing about the delete.

The agent didn’t fix it. It wrote a known-issues entry and added two @unittest.expectedFailure tests, because the handoff for the refactor said: “characterize, then collapse, then fix separately”. It was the wrong call, and two git log -S runs showed it.

The setup

Claude Code working in a Python repository of an accounting pipeline, tested with unittest;
A long-lived function where several steps append to result.issues, the operator’s per-file report;
A refactor rule that a split must not change behavior.

What parking costs

An expectedFailure marker looks like a decision. The next reader sees a documented, tested, known gap and assumes someone weighed it and chose to keep it. If nobody did, the marker turns an accident into policy and keeps a live bug in the tree behind it.

So before you park a bug, find out whether anyone ever decided it. I asked the agent to run the fence check (Chesterton’s fence: don’t remove a fence until you know why it was put up).

Date the bug

A bug is a relationship between two things: this line and the code it gets wrong — date both.

git log --oneline -S'result.issues = validation.issues' -- <path>
git log --oneline -S'result.issues.append' -- <path>

The result went into the fix’s commit message:

2026-04-16 rebind written. result.issues had NO prior writers;
 assigning validation's list into it lost nothing.
2026-04-19 + vendor mismatch trail above it +3d
2026-04-23 + REPAIR delete note above it +7d
2026-04-24 + registry retry verdict above it +8d
2026-05-27 + self-billed reroute above it +41d

The rebind was correct when it was written. Over the next 41 days, four commits added writers above it, and none of them touched it. Nobody decided to discard those issues; the code predates the thing it breaks.

That gives one of three verdicts:

Fence: someone decided this, in a commit message, a spec’s non-goals or an ADR. Leave it.
Drift: the code predates what it is wrong about, or its author never touched the site. Fix it now, and put the dates in the commit message so a reviewer can see you didn’t remove a fence.
Silence: the history says nothing either way. Park it, and say plainly that nobody decided it.

This one was drift, so it got fixed the same night. It had stayed hidden because the only test that reached this code patched the whole function body out. The split made the body reachable, and the first end-to-end repair test found the bug.

The obvious fix was wrong

A drift verdict says nobody meant the bug; you still have to find out what depends on it. The one-word fix:

result.issues.extend(validation.issues)

would have broken an alias. A later step appends registry failures to result.issues, and those reach the invoice’s note only because the rebind made result.issues and validation.issues the same list, which the note builder reads:

result.issues = validation.issues # one list, two names
apply_registry_failure(result) # appends to result.issues ...
note = build_note(validation) # ... and the note sees it through the alias

With .extend(), every invoice that failed the business-registry check would have lost that warning from its note: one bug traded for a quieter one. The fix kept the rebind and put the earlier issues back after the later writers ran:

earlier_issues = list(result.issues)
result.issues = validation.issues
apply_registry_failure(result)
# ... the other writers ...
result.issues = earlier_issues + result.issues

The tests were written to catch both mistakes: reverting the fix fails the four tests that check the earlier issues survive, and the naive .extend() fails the one that checks the registry warning still reaches the note. The known-issues entry was deleted, and the expectedFailure markers became ordinary tests.

What silence looks like

That night, four other bugs were dated as drift and fixed. One wasn’t. Debug files were named from the input’s stem, so invoice.docx and invoice.pdf in one batch wrote the same file, and the later one won. That naming shipped on a pipeline that already read several formats, so the code didn’t predate the collision. No commit message or spec mentioned it either: stem is just the obvious way to name a sidecar file. That one stayed a known-issues entry, with the dating written into it.

The skill

The fence check is a Claude Code skill I had written down earlier that evening. The agent had it and didn’t run it until asked, so its memory now says to run it the moment it finds a bug, before choosing between a fix and a known-issues entry. Here it is verbatim. The examples and the docs/superpowers/ paths are from my repository; swap in your own spec directory.

---
name: checking-the-fence
description: Date a bug before fixing it — Chesterton's fence. Use when you have found code that looks wrong and are about to change it, when you are about to delete something that looks dead, or when another skill needs the fence check.
---

# Checking the Fence

A bug you found is a **fence** across a field. Before you take it down, find who put it there. Here that is written down twice — in `git log` and in `docs/superpowers/` — and is usually decisive in a minute.

Two verdicts. They are the whole skill:

- **Fence** — someone decided this. Leave it, and record an ADR if the next reader will reach for the same fix you just did.
- **Drift** — nobody decided it. The code predates the thing it is wrong about, or an author was never told which files the job touched. Fix it.

Fences here are real, not hypothetical: `docs/adr/0002` exists only to stop a reader from deleting emoji handling that has no inbound traffic and looks dead.

## Steps

Stop at the first step that returns a verdict.

### 1. Date the code

```
git log --oneline -S'<symbol>' --all -- <path>
git log -1 --format=%B <sha>
```

Commit messages here carry the rule the author meant to encode, and the rule is often the thing the bug breaks. `_PARSER_PRIORITY`'s own commit says *"process deterministic parsers before PDFs/images"* — while the table shipped the cheapest format last.

**Done when** you can name the commit and quote the rule.

### 2. Date what it is wrong about

A bug is a relationship: a table missing a format, a copy missing an entry, a switch missing a branch. Date the other side too.

Code cannot have decided about something that did not exist yet. `_PARSER_PRIORITY` (2026-04-27) predates isdoc (2026-06-01) by five weeks.

**Done when** you have both dates. Predates → **drift**. Stop.

### 3. Read the spec

```
grep -rli '<feature>' docs/superpowers/specs docs/superpowers/plans
```

Read the **Non-goals** first — that is where a deliberate gap is named. A spec that scopes the gap out is a **fence**. A spec whose own words the bug contradicts is **drift**: the ISDOC spec calls it *"a deterministic fast path before Claude"*.

**Done when** you have quoted the line that scopes it out, or established that no spec names it.

### 4. Read the blast pattern

```
git show --stat <sha>
```

Which files that commit touched, against which it needed to. An author who updated three of five consumers was not deciding about the other two — nothing told them. The isdoc and docx commits landed a day apart and touched different subsets.

**Done when** you can say whether the author addressed the site at all. Never addressed → **drift**.

### 5. Was it born wrong?

For a forked copy, compare it against its source **on the day it was copied**, not today.

```
git show <sha>:<source-path>
```

`coverage_report.py` shipped without `.json` on 2026-04-24, and the map it was copied from carried `.json` that same day. Born forked, then rotted twice more.

**Done when** you know whether the gap is drift-over-time or a bad copy. Either is **drift**.

## Applying the verdict

**Drift** — fix it, and say in the commit message what you found: the dates, and that nothing decided it. That is what stops a reviewer from wondering whether you removed a fence.

**Fence** — leave it. Offer an ADR when a future reader would reach for the same fix; that reader is usually the next architecture review, or you.

**Neither** — the history is silent and no spec mentions it. Say so plainly and let the user decide; silence is not consent, and it is not a fence either.

Save it as .claude/skills/checking-the-fence/SKILL.md in your repository, and tell the agent in CLAUDE.md or AGENTS.md to run it before it records a bug as a known issue.

Takeaway

A parked bug looks like a decision, so check whether anyone made one. If the code predates what it breaks, fix it now, and check the mechanism first: knowing nobody meant the bug doesn’t tell you what depends on it.

Next time your agent adds an expectedFailure or a known-issues entry, ask it for the two dates first.

Every test mocked the helper, and except Exception swallowed its TypeError on every PDF

Thu, 16 Jul 2026 23:42:00 +0200

For seven weeks, a document classifier never read a single PDF. This is the helper that was supposed to give the model some text (trimmed):

def _extract_excerpt(file_path: Path) -> str:
 """First-page text for PDFs, first ~8 kB for everything else."""
 if file_path.suffix.lower() == ".pdf":
 try:
 pages = extract_pdf(file_path)
 return (pages[0] if pages else "")[:EXCERPT_BYTES]
 except Exception:
 return ""
 ...

extract_pdf doesn’t return a list of pages. It returns a dataclass, so every call raised this:

TypeError: 'PDFExtraction' object is not subscriptable

The except caught it and returned "". The prompt then held the filename and an empty excerpt, and the model answered anyway: a document type and a confidence score, for every PDF it was asked about.

If a helper of yours has a catch-all except and your tests patch that helper, check whether anything ever runs it for real.

Setup

A web upload went through a small Python script that sorted the file into one of the document kinds declared for an accounting period (received invoices, bank statements, and so on). With two or more possible kinds, it built a prompt from the filename plus the start of the document and asked a language model, which answered with JSON like {"kind": "...", "confidence": 0.9}. The PDF text came from an in-house extract_pdf() built on pdfplumber. Tests used the standard library’s unittest and unittest.mock.

What went wrong

The helper was wrong from the day it was written. extract_pdf had returned a PDFExtraction dataclass with a pages list since its first commit, six weeks earlier, and its docstring said so. The if pages check didn’t catch the mistake: a dataclass without __len__ is always truthy, so the check passed and pages[0] raised.

The except was there so that a corrupt or encrypted upload wouldn’t break classification. It caught the TypeError just as readily and returned "", which is also the documented answer for an image upload with no text to read.

And every test patched the helper out. The tests that reached the model looked like this:

with patch("scripts.classify_upload.invoke_ai") as m, \
 patch("scripts.classify_upload._extract_excerpt", return_value="Invoice 2026-001 ..."):
 m.return_value = json.dumps({"kind": "received_invoices", "confidence": 0.92})
 result = classify_file(Path("/tmp/x.pdf"), ["issued_invoices", "received_invoices"], "claude")

They covered the prompt plumbing and the JSON parsing, and none of them ever called _extract_excerpt. The implementation plan that specified the helper had the same broken line, and it specified these tests, mocks included. The code, the plan, and the tests agreed with each other, and none of them had been checked against extract_pdf.

The model hid the rest. A filename like invoice-0423.pdf can be enough for a right guess, and a wrong guess came back with a confidence score like any other, so nothing in the output said the excerpt was empty.

Fix

The fix reads the first page, as the docstring promised:

extraction = extract_pdf(file_path)
first_page = extraction.pages[0].text if extraction.pages else ""
return first_page[:EXCERPT_BYTES]

The new test class calls the real function on a real PDF:

class RealPdfExcerptTests(unittest.TestCase):
 def setUp(self):
 self.tmp = tempfile.TemporaryDirectory()
 self.root = Path(self.tmp.name)

 def tearDown(self):
 self.tmp.cleanup()

 def test_a_pdf_yields_its_first_page_text(self):
 path = self.root / "invoice.pdf"
 _write_pdf(path, "INVOICE 2026-001 Supplier ACME")
 self.assertIn("INVOICE 2026-001", _extract_excerpt(path))

 def test_a_corrupt_pdf_yields_no_signal_rather_than_raising(self):
 path = self.root / "broken.pdf"
 path.write_bytes(b"%PDF-1.4\nnot actually a pdf at all")
 self.assertEqual(_extract_excerpt(path), "")

_write_pdf is an existing test helper that writes a one-page PDF with a text layer. Before committing, I ran the first test against the original code to confirm it fails there. The except stayed, because corrupt uploads are real. The second test now says what it’s for.

A day later, the same helper had a second route to the same result. It kept its own list of text suffixes, so Word, Excel, and XML e-invoice uploads fell through to "" and were classified by filename too. Two of those formats joined the pipeline a week after the list was written, and nobody updated it. The helper now asks the pipeline’s format registry instead.

Why

A mock replaces the code it patches with your assumptions about that code. When every test mocks a function, its tests check only those assumptions, and the function itself is never run. A catch-all except inside the function then removes the last signal you’d get in production: no traceback, no failed request, just a default value that looks legitimate.

A language model downstream makes this worse. A parser fed an empty string usually fails loudly, while a model returns its best guess in the usual format, confidence field included, whatever the input.

Takeaway

Mock the expensive or external dependency (here, the model) and let the function under test run on a real input. If it has a catch-all except, give it a test with an input that should succeed, so a bug can’t hide behind the except.

Grep your tests for patch( targets that point at your own module’s private helpers, and for each one, find a test that calls the helper unmocked.

Sandbox-blocked tests called "environmental", then committed

Mon, 13 Jul 2026 15:38:00 +0200

My coding agent finished a feature, ran the full test suite, and reported:

Full suite: 135 passed, 97 skipped; 22 DB suites failed due to no container runtime, plus sandboxed stub bind. Commit only handoff implementation files.

Then it committed. I ran pnpm test myself and got three real failures. The production build it had also skipped was broken too, and nobody found out for two months.

If your agent’s test run shows Error: Could not find a working container runtime strategy or Error: listen EPERM: operation not permitted 0.0.0.0:8080, those tests never ran, and a summary that counts them as “blocked” tells you nothing about the code.

The setup

Codex CLI on macOS, with its default sandbox
A pnpm monorepo: a Next.js 15.5.19 app plus a few packages
Vitest 2.1.9, with Testcontainers 10.28.0 starting Postgres for the database tests and a local stub of an external service listening on port 8080

The agent split the feature into five slices and gave each to a parallel subagent. Each subagent ran its own focused tests, and the orchestrator ran the full suite and the build at the end.

What went wrong

The sandbox blocks more than writes outside the project. Here it blocked three things the checks needed:

Testcontainers couldn’t reach Docker: Could not find a working container runtime strategy.
The service stub couldn’t bind its port: listen EPERM: operation not permitted 0.0.0.0:8080.
next build couldn’t write its compiler cache: EPERM writing ~/Library/Caches/next-swc.

Every database suite failed in beforeAll, and Vitest reported the tests inside them as skipped. The agent’s final report was accurate:

- Typecheck pass
- Focused tests pass
- Full: 135 passed; DB suites blocked by no Docker
- Build blocked by Next SWC cache permissions

Every line of that is true. But the suite has 232 tests, and 135 + 97 = 232: over 40 percent of it never ran, including every test that touched the database, where this feature did most of its work.

On the host, the same suite printed:

Test Files 2 failed | 63 passed (65)
 Tests 3 failed | 229 passed (232)

One new test forgot to start the service stub, so its client found no endpoint configured. Two older tests broke because the feature added an ownership check and their fixtures sent requests without a logged-in user (Error: authentication required). The agent fixed all three in two minutes once it saw them.

The build was worse. The commit re-exported a server action from another "use server" file:

"use server";

export { finalizeArtifact } from "../../reports/actions.ts";

tsc accepts this, and so do the tests. Only next build rejects it:

Failed to compile.

./app/reports/[id]/actions.ts
Error: x Only async functions are allowed to be exported in a "use server" file.

Two months later, a different session ran a production build, hit this error, and deleted the line.

The same sandbox, the same day

Half an hour before the bad commit, another session in the same repository hit the same two errors. At first it called them environmental too. Then it decided:

Full suite blocked by sandbox: Testcontainers cannot see Docker; stub bind gets `EPERM :8080`. Since Docker is reportedly running, retry full suite with host access.

Two minutes later it reported “Host rerun green: 60 files, 222 tests”, and only then did it commit. The agent that fixed my three failures did the same: it reran the focused tests with host approval before committing the fix. Both sessions knew how to leave the sandbox; nothing required them to before a commit.

Fix

Treat “blocked” as “not run”, and make “not run” stop the commit.

In Codex, a command runs outside the sandbox when the agent requests sandbox_permissions: "require_escalated" and you approve it. Put the rule in the instructions file the agent reads every session (AGENTS.md, or CLAUDE.md for Claude Code):

A check that the sandbox blocked has not passed. Before committing, rerun every blocked test suite and build with host access (`sandbox_permissions: "require_escalated"`).
If that isn't possible, don't commit: report what didn't run.

Then read the agent’s numbers yourself. If passed plus skipped equals the total, and you never skip tests on purpose, every skipped test is one that didn’t run. If you have CI, have it run the full suite and the build before a merge.

Why it happens

The agent’s memory for this project already had a rule: “separate passed static or focused checks from runtime suites blocked by the environment, and reserve the final green claim for integrated verification.” The agent followed that rule to the letter: it never claimed green and listed exactly what had been blocked. Then it committed, because nothing in its instructions tied the commit to the green claim.

“Blocked by the environment” reads like an infrastructure problem that someone else will handle, so it ends up as a footnote in the report while the commit goes ahead. The rule made the report honest. It didn’t make anyone run the tests.

Takeaway

When a sandbox blocks a test or a build, nothing was verified. An agent that reports this honestly will still commit, unless its instructions say that a blocked check stops the commit.

Look at your agent’s last commit: if its summary says a suite or a build was “blocked”, rerun them on the host today.

ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY: your coding agent's pnpm 11 is not your pnpm 10

Mon, 06 Jul 2026 15:04:00 +0200

A coding agent ran pnpm test in my repo and got this:

Scope: all 6 workspace projects
[ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY] Aborted removal of modules directory due to no TTY

If you are running pnpm in CI, set the CI environment variable to "true", or set "confirmModulesPurge" to "false".

The stack trace ended in runDepsStatusCheck (.../node_modules/.pnpm/pnpm@11.7.0/...). My package.json says "packageManager": "pnpm@10.33.0", and pnpm -v in my terminal prints 10.33.0.

If you see this error only inside an agent, the agent is running its own pnpm, and following the hint makes things worse.

The setup

macOS, pnpm 10.33.0 through Corepack, a pnpm monorepo with a Next.js 15 app
The Codex desktop app, which ships its own Node 24 and pnpm under ~/.cache/codex-runtimes/codex-primary-runtime/dependencies/: pnpm 11.7.0 in July, 11.25.0 in October
The agent runs pnpm test without a TTY

What went wrong

The agent followed the hint and reran with CI=true pnpm test. pnpm stopped asking and did what it had wanted to do from the start:

Recreating /Users/me/project/node_modules
Progress: resolved 378, reused 0, downloaded 137, added 12
...
[WARN] GET https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-15.5.19.tgz error (ERR_PNPM_ENOSPC). Will retry in 10 seconds. 2 retries left.
[WARN] GET https://registry.npmjs.org/next/-/next-15.5.19.tgz error (ERR_PNPM_ENOSPC). Will retry in 10 seconds. 2 retries left.
...
Progress: resolved 378, reused 0, downloaded 377, added 377, done

[ERR_PNPM_IGNORED_BUILDS] Ignored build scripts: cpu-features@0.0.10, esbuild@0.21.5, esbuild@0.28.1, protobufjs@7.6.4, sharp@0.34.5, ssh2@1.17.0

It deleted node_modules and downloaded every package again (reused 0) into a second store, and the disk ran out on the way. It left an untracked .pnpm-store/ in the repo and wrote placeholders into the tracked pnpm-workspace.yaml:

allowBuilds:
 cpu-features: set this to true or false
 esbuild: set this to true or false
 protobufjs: set this to true or false
 sharp: set this to true or false
 ssh2: set this to true or false

No tests ran. After that, even pnpm exec tsc --noEmit failed with the same NO_TTY error, so the agent switched to ./node_modules/.bin/tsc and ./node_modules/.bin/vitest, reverted the YAML, deleted .pnpm-store/ and committed with only the focused tests green.

Ten weeks later, next build failed:

 ⚠ Attempted to load @next/swc-darwin-arm64, but it was not installed
 ⨯ Failed to load SWC binary for darwin/arm64, see more info here: https://nextjs.org/docs/messages/failed-loading-swc

The July install had ended one package short (378 resolved, 377 added), and the SWC binary was one of the two downloads that hit ENOSPC. A later agent session “fixed” the build by installing the binary into /tmp and symlinking it into node_modules, a fix that lasts until macOS cleans /tmp.

Fix

First, ask the agent’s shell which pnpm it gets:

command -v pnpm
pnpm -v

Then make the agent use your version. Corepack reads packageManager, whatever is first on PATH:

corepack pnpm test # runs pnpm 10.33.0

Write that into AGENTS.md (or your agent’s equivalent), along with one rule: if pnpm wants to remove node_modules, stop and report.

To make the bundled pnpm harmless as well, turn off the automatic install in pnpm-workspace.yaml:

verifyDepsBeforeRun: false

That is the pnpm 10 default, so nothing changes for you. I checked it with the bundled pnpm 11.25.0: with the setting, it runs the script; without it, it installs first.

If you already followed the hint, clean up with your own pnpm:

git checkout -- pnpm-workspace.yaml
rm -rf .pnpm-store
corepack pnpm install --frozen-lockfile

Run that in a terminal. If pnpm asks to remove node_modules, say yes; this time it rebuilds against your own store.

Why

Three pnpm 11 changes line up:

verify-deps-before-run defaults to install (pnpm 10: false). pnpm test and pnpm exec check node_modules first and install if anything is off.
pnpm 11 uses store v11; pnpm 10 uses v10. node_modules/.modules.yaml records the store it was linked from, so to pnpm 11 a working pnpm 10 install looks incompatible and has to be purged. confirmModulesPurge defaults to true; a purge without a TTY aborts, and CI=true drops the confirmation but not the purge.
Ignored build scripts are an error, and pnpm writes the allowBuilds placeholders for you to fill in.

On my Mac, the two stores now sit side by side in ~/Library/pnpm/store/: 851 MB in v10, 588 MB in v11.

Why didn’t packageManager catch it? pnpm 11 does honor it: run the bundled binary directly and pnpm -v prints 10.33.0, because it switches to the pinned version. The wrapper in the October runtime turns that off:

#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
export pnpm_config_pm_on_fail=ignore
exec "${SCRIPT_DIR}/../../node/bin/node" "${SCRIPT_DIR}/../../node/node_modules/pnpm/bin/pnpm.mjs" "$@"

The environment variable beats anything in the repo. With pmOnFail: error in pnpm-workspace.yaml, or devEngines.packageManager with "onFail": "error", the wrapper still runs 11.25.0.

Takeaway

An agent runtime can bring its own toolchain and switch off the check that would have caught it. When an agent reports “tests blocked by the environment”, find out which binary it ran before you believe the report.

Run pnpm -v in your agent’s shell today and compare it with packageManager.

TS2724 "has no exported member" in a git worktree: a symlinked node_modules typechecks the main checkout's source

Mon, 22 Jun 2026 19:21:00 +0200

A coding agent cherry-picked a subagent’s commit into an integration worktree, ran pnpm typecheck and got:

apps/web/lib/uploads.ts(3,59): error TS2724: '"@app/persistence"' has no exported member named 'recordRawUpload'. Did you mean 'recordUpload'?

recordRawUpload was right there, exported from packages/persistence/src/index.ts in that worktree. If tsc can’t see an export that you can, and the worktree’s node_modules is a symlink to another checkout, tsc is reading the other checkout’s source.

The setup

A pnpm 10 monorepo on macOS. The workspace packages @app/* point exports straight at ./src/index.ts, so there is no build step between packages.
An orchestrating agent that creates a git worktree, a branch, and a subagent per issue, and cherry-picks finished work into an integration worktree.
A full disk.

What went wrong

Every worktree had its own install, and du counted up to 513 MB of node_modules in each. A dozen worktrees later, the disk was full:

Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk3s5 460Gi 420Gi 116Mi 100% 6.6M 1.2M 85% /System/Volumes/Data

To save space, the agent replaced the node_modules of each new worktree (the root, apps/web and every package) with a symlink to the main checkout’s copy:

ln -s ../project/node_modules ../project-integration/node_modules
ln -s ../../../project/apps/web/node_modules ../project-integration/apps/web/node_modules

The first typecheck after that passed. Later, a change that added an export to @app/persistence and used it in apps/web failed with TS2724. readlink shows why:

$ readlink apps/web/node_modules/@app/persistence
../../../../packages/persistence

pnpm links workspace packages with relative symlinks. A relative link resolves from the directory it really sits in, and that directory is now in the main checkout. So @app/persistence in the worktree was /Users/me/project/packages/persistence, the main checkout’s source, which didn’t have the new function yet.

The typecheck that passed also validated the main checkout.

Two subagents in their own worktrees hit the same error. One deleted the links, reinstalled, and its typecheck passed. The other wrote “Validation blocked by worktree node_modules links targeting older shared package. Source diff reviewed; commit now.” and committed without a passing typecheck.

Fix

Give each worktree its own install. pnpm keeps package files in a content-addressable store, so an offline install only links them into place:

rm node_modules apps/web/node_modules packages/*/node_modules # the symlinks
pnpm install --offline --frozen-lockfile

This took 3.6 seconds and reported resolved 378, reused 378, downloaded 0. If the store is missing a package, --offline fails; drop the flag.

Then check that every workspace package resolves inside the worktree. Run it with bash: zsh aborts on the first glob that matches nothing (no matches found), and the agent’s own fix hit exactly that.

#!/usr/bin/env bash
# Print workspace packages that resolve outside this checkout.
root=$(pwd -P)
for link in node_modules/@app/* */*/node_modules/@app/*; do
 [ -e "$link" ] || continue
 target=$(realpath "$link")
 case $target in "$root"/*) ;; *) echo "$link -> $target" ;; esac
done

No output means every workspace package comes from this worktree.

If you have to share node_modules, share only the third-party packages. The agent’s fix was an overlay: a real node_modules directory per package, with every entry symlinked to the main checkout except @app, whose entries point at the worktree’s own packages. Copying the main checkout’s @app directory with cp -RP gives the same result, because the copied relative links resolve from their new place.

Why

TypeScript (with the default preserveSymlinks: false) and Node both resolve a package to its real path. Third-party packages are the same files in every checkout that shares a lockfile, so following the link to the main checkout is harmless. Workspace packages are your source code, and that differs between checkouts by design. Once a branch changes its dependencies, the third-party half goes stale as well.

The disk savings were probably smaller than they looked. pnpm clones or hard-links files from its store, and du counts those blocks again in every tree, so its per-worktree numbers overstate what each install cost. The disk was also full for other reasons, and nobody measured how much the links saved.

Takeaway

A worktree that borrows another checkout’s node_modules gets that checkout’s workspace packages too, and typecheck and tests then run against the wrong code, passing or failing. Run pnpm install --offline in every new worktree.

If your agents create worktrees, run the check above in one of them today.

Next.js "Cannot read properties of undefined (reading 'length')" in installChunk: two processes share .next

Mon, 22 Jun 2026 13:01:00 +0200

I opened the dashboard of my Next.js app and got a 500 error:

 GET /app/dashboard 500 in 285ms
 ⨯ TypeError: Cannot read properties of undefined (reading 'length')
 at <unknown> (.next/server/app/app/accounts/page.js:685:47)
 at Object.<anonymous> (.next/server/app/app/accounts/page.js:688:3) {
 page: '/app/accounts'
}
 GET /app/accounts 500 in 278ms

If the stack points into generated .next/server/.../page.js and the full trace ends in installChunk in webpack-runtime.js, your code is fine. Another Next process has written into the .next directory your dev server is reading from.

The setup

Next.js 15.5 with webpack in dev, in a pnpm monorepo (the app lives in apps/web)
pnpm dev running in my terminal on port 3000
A coding agent working in the same checkout, which checks its work with pnpm build

What went wrong

I asked the agent to fix the 500. It did what anyone would do with reading 'length': it grepped the app for .length. The accounts page didn’t touch a collection at all, so the agent followed the authenticated layout into the auth code and suspected an import there. Then it opened the generated page.js at line 685 and found webpack glue, not application code.

To reproduce, the agent ran pnpm build, which succeeded, and then started its own next dev -p 3001 next to mine. The first request to /app/accounts returned 200. After a request to the dashboard, both routes returned 500, and this time the error page carried the whole stack:

TypeError: Cannot read properties of undefined (reading 'length')
 at installChunk (apps/web/.next/server/webpack-runtime.js:249:41)
 at __webpack_require__.f.require (apps/web/.next/server/webpack-runtime.js:259:15)
 ...
 at apps/web/.next/server/app/app/dashboard/page.js:718:47

Line 249 is in webpack’s chunk loader:

var installChunk = (chunk) => {
 var moreModules = chunk.modules, chunkIds = chunk.ids, runtime = chunk.runtime;
 // ...
 for(var i = 0; i < chunkIds.length; i++)
 installedChunks[chunkIds[i]] = 1;
};

chunk is whatever require("./" + chunkFile) returned. chunk.ids was undefined, so the file on disk was not the chunk this runtime expected. A ps showed why: two next-server processes, mine on 3000 and the agent’s on 3001, both compiling into apps/web/.next. On top of that, the agent’s next build had written its production output into the same directory a few minutes earlier.

Both of the usual ways to reproduce a bug, a fresh build and a second server, are ways to cause this one. Before you reproduce, look for other Next processes on the checkout:

ps -axo pid,ppid,command | grep '[n]ext-server'

Fix

Give next dev its own output directory in next.config.mjs:

export default {
 // `next dev` and `next build` generate incompatible webpack chunk graphs.
 // Keep their output separate so a build cannot break a running dev server.
 distDir: process.env.NODE_ENV === "development" ? ".next-dev" : ".next",
};

Ignore it in .gitignore:

.next/
.next-dev/

Add its generated types to tsconfig.json, next to the .next ones:

"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts", ".next-dev/types/**/*.ts"]

The running dev server picked up the config change and rebuilt into .next-dev. From then on, next build writes to .next and the dev server never reads it.

To recover without the config change, stop every Next process except one, delete .next and restart pnpm dev.

If you’re on Next.js 16, you already have this: next dev writes to .next/dev, so dev and build can run concurrently .

The follow-on trap: `next-env.d.ts`

Next rewrites next-env.d.ts on every dev start and every build, and the file names the types of whichever directory ran last:

/// <reference path="./.next-dev/types/routes.d.ts" />

After every next build, it says ./.next/types/routes.d.ts instead. The file was tracked, so it showed up as modified in later sessions, and the agents kept “fixing” it with git show HEAD:apps/web/next-env.d.ts > apps/web/next-env.d.ts. Do what the .gitignore from create-next-app does and ignore the file:

echo 'next-env.d.ts' >> apps/web/.gitignore
git rm --cached apps/web/next-env.d.ts

Why

next dev and next build both default to .next in Next.js 15, and they write different chunk graphs into it. A second next dev is no better: two webpack compilers write into one directory, and neither knows about the other. The runtime that is already loaded require()s a chunk by the name from its own graph, gets a file the other compilation wrote, and fails at chunk.ids.length. The error says length, so you go looking for an array in your code.

Coding agents check their work with next build, usually while your dev server is still running, so you’ll hit this more often than you’d think.

Takeaway

An error from inside installChunk points at the build directory, so look for a second Next process before you read your own code. Never let two Next processes share a distDir.

If an agent or a script runs next build in the checkout where your dev server runs, give dev its own distDir today.

An MCP server has more outputs than tool results

Tue, 19 May 2026 20:00:00 +0200

I designed a local stdio MCP server in Go that gives an LLM scoped access to a shared password vault. Two facts set the threat model. Any member of a shared vault can write an entry comment that another member’s agent later reads, so stored text is attacker-controlled. And the model can call every tool in the manifest, so whoever steers the model gets that authority too.

The plan started with a prompt-injection fence around user strings. Most of the work ended up elsewhere.

Keep the secret off the transcript

Once get_password returns plaintext, the value sits in the model’s context and the host’s chat history, and no scope rule can take it back. So the everyday path never crosses that boundary:

copy_password_to_clipboard takes an ID, writes the value to the OS clipboard from inside the server, and returns Password (id=42) copied to clipboard. Auto-clear compares a SHA-256 of the current clipboard first, so it can’t wipe something the user copied later.
create_password_generated generates the value server-side and returns only the new row’s ID.
get_password returns the literal <redacted; revealPlaintext=never> by default, and its description points the model at the clipboard tool.

The confirmation first included the fenced entry title. That was removed: the model already has the title, and “action just completed” is a bad moment to hand it attacker-written text. A tool result should echo as little as possible.

Plaintext in chat remains for cases where the model has to reason about the value, such as a reuse audit. It sits behind a setting: never, per_call or always.

That setting began as a boolean, and the trap was in the annotations. A read is correctly readOnlyHint=true, so the host’s confirmation never fires, and one injected instruction could walk list_passwords → get_password across every entry in scope. Marking the read as destructive doesn’t help: annotations are static at registration, and hosts may ignore the cue on a read-only tool.

So per_call is an OS-native modal raised inside the server before the value is serialized, with a 30-second timeout that fails closed. If no GUI is available, the server refuses to start in that mode.

Two more manifest details matter. destructiveHint and openWorldHint default to true in the spec, which is why the Go SDK types them as *bool, and every tool here sets them explicitly. And the manifest itself is a control: if the clipboard probe fails, the tool is never registered, so the model can’t try it.

Authority lives in code

Tool descriptions are instructions to something an attacker may be steering, so they count as guidance, not control. Scope is a deterministic filter on (vault, [tags], mode), and the type system gates writes:

func (s *scopedOps) DeletePassword(ctx context.Context, rule *scope.ResolvedRule, id ID) error

A rule only comes from the scope resolver, so a handler that skips the check has nothing to pass and doesn’t compile. A runtime check catches the wrong rule: another vault, or a read-only mode. The scope itself lives in the OS keychain, and the server only reads it so that no tool call can widen it.

The input schema enforces less than it looks like

With the Go SDK, mcp.AddTool derives the input schema from struct tags. An earlier tag format, jsonschema:"required,description=...", failed at registration because the jsonschema tag is now the property description and nothing else. The tags were rewritten as prose:

Mode string `json:"mode,omitempty" jsonschema:"Strategy. (Schema hints: enum=memorable; enum=random.)"`
Length int `json:"length,omitempty" jsonschema:"Length. (Schema hints: minimum=6; maximum=32.)"`

That reads like a constraint and enforces nothing. The generated schema has only type and description, and a call with mode=evil, length=999 reaches the handler. The handler’s own validation is the only check, and it has to reject: the generator below it silently clamps, so the model would ask for one thing and get another.

The schema does give you types, required fields, and additionalProperties: false. When it rejects a call, the SDK answers before your handler runs, so your audit middleware never sees it, and the message echoes the offending value.

Beyond that, arguments are IDs, enums, and bounded integers. Omitted and [] mean different things on an update, so that field is a *[]int64. The model can’t push a row out of its own visibility, because one allowed tag must remain.

Stored content is deliberately not filtered. A comment reading “ignore previous instructions” is legal data, and the regular clients show it verbatim. Escaping happens when text leaves for the model.

The fence, and the encoder after it

Read tools wrap each user string, and their descriptions tell the model that fenced text is data:

<user_content>{escaped}</user_content>

&, < and > are HTML-escaped so a stored payload can’t close the tag. C0 controls are escaped too, because a stored ESC paints ANSI sequences on the terminal of anyone inspecting raw traffic, and the nine bidi overrides are replaced so stored text can’t reorder what a human approves.

“Ignore previous instructions” passes through unchanged. Against natural-language injection, the fence makes the boundary explicit. The real limit is what a steered model can reach.

Two gaps remain, both visible only by running it:

The tags don’t arrive as written. Returning a typed Out lets the SDK serialize it with encoding/json, which HTML-escapes by default. A title reaches the model’s text block as <user_content>..., and & becomes &. The payload still can’t close the fence, but the literal the tool description promises isn’t the literal on the wire. To control it, set result.Content yourself using an encoder with SetEscapeHTML(false).
The escape table works on bytes. It covers bytes below 0x20, so DEL and U+009B, the 8-bit CSI encoded as C2 9B, pass through. A rune-based check with unicode.IsControl catches both.

Each sink needs its own treatment

Sink	Data	Treatment
Tool result	Free text	Escape, replace bidi overrides, fence. Keep ZWJ and RLM, which Arabic, Indic scripts, and emoji need
Tool result	Vault and tag names	Strip control and format runes, cap at 256, then fence, which still protects a handler that forgets the fence
Startup error	Backend-supplied version	Allowlist regex, otherwise `unknown`
Consent dialog	Entry title	Reject controls, line separators and bidi; truncate to 80 runes
Subprocess	Entry title	Environment variable, never script source
Audit log	Content fields	Dropped; a search query is logged as its length
Tool error	Anything	Fixed text

The consent dialog shows why one sanitizer isn’t enough. The AppleScript is a constant, and the title arrives as data:

set t to system attribute "ENTRY_TITLE"
display dialog "Reveal password \"" & t & "\" to your AI assistant?" -- ...

That stops code injection and leaves another problem. The dialog still renders the title, so newlines, bidi overrides, or a long payload could bury the warning and harvest a click on Allow. Data-not-code and safe-to-display are separate properties. The interpreter is called by absolute path as well, because a PATH-shadowed binary would receive the title in its environment.

The error path talks to the model

In the Go SDK, a non-nil error from a tool handler becomes isError content, so err.Error() goes to the model. Every handler here runs through a middleware that classifies the error, returns fixed text, and sends the reason to the audit log:

func wrapHandler[In, Out any](s *Server, name string,
	inner func(context.Context, In) (Out, error),
) mcp.ToolHandlerFor[In, Out] {
	return func(ctx context.Context, _ *mcp.CallToolRequest, args In) (
		result *mcp.CallToolResult, out Out, err error, // named: recover writes these
	) {
		defer func() {
			if r := recover(); r != nil {
				result, out, err = errGeneric(), *new(Out), nil
			}
		}()
		// deadline, session check, inner(), classify, audit
	}
}

The named returns matter. Without them, a recovered panic returns (nil, zero, nil), which the host reads as an empty success. The panic value is logged by type, and by content only if it’s a string or error, because %v on a struct could print a password.

What still had to leave the error text:

Scope details. A bypass error included the rule’s vault ID and mode, so a steered model could map the scope one call at a time. Those fields moved to a typed error for the audit log.
Subprocess output. fmt.Errorf("pbcopy: %w: %s", err, out) could carry clipboard content. Stdout now goes to /dev/null, and stderr never joins the error.
Identity. An “account changed” error named the username and backend. It can fire on every tool call, putting N copies of an email address into a transcript that may get exported or summarized. The final text names nobody.
Tamperable integers. A version-mismatch error quoted a schema number from a local file, whose author then controlled a value the model reads as authoritative.

The audit log is an output too. Its redactor is a key-name denylist over any, and it needed six fixes: nested maps, []map[string]any, pointers and typed maps, nested structs, Password/URL casing, and map keys in result metadata. A new field called token would still be logged. An allowlist of loggable fields changes that failure from a leaked secret to a missing log value.

The host spawns the server once and keeps it alive for the chat. An ADR said the session is read on every call, so a login as a different account gets refused. The implementation read it once at startup, and the error sentinels existed with no code path producing them. A second login with a chat open left the server acting as the old account, with no warning.

The check now runs in the middleware, before each handler. It came with a race: a check could pass, then a sibling refresh could swap the credentials before the handler used them. The check and the credential snapshot now happen in one critical section, and the handler receives that snapshot through its context.

What stays open

Natural-language injection is contained, not solved. The scope rules limit the blast radius.
The tag filter runs on the server after the backend returns the whole vault, so the binary is trusted. A server-issued scoped token is the real fix.
Auto-clear dies with the process, and clipboard managers keep their own history.

Takeaway

Treat the SDK as part of your attack surface: read what your schema really contains, and look at the bytes a tool result becomes. Then list every place a value can go, including errors, dialogs, and logs. Remove the sinks you can, as the clipboard tool did for secrets, and give each remaining sink the treatment its grammar calls for.

Llm on Lessons learned

"No route to host" only for the AI agent: macOS Local Network privacy

The setup

What went wrong

Fix

Why only the agent

Check before you blame the network

Takeaway

"Ansible requires blocking IO on stdin/stdout/stderr" when an AI agent runs Ansible

The setup

What went wrong

Fix

Why Ansible insists

Make the agent remember it

Takeaway

Cyber classifiers can interrupt offline pentest summaries

A flag-shaped string is not a captured flag

Offline processing still triggers cyber classifiers

Make the audit resumable and model-neutral

Prompt injection: extract tokens, then decode them offline

Lexical refusal filters leave a semantic gap

Decode a synthetic transcript deterministically

Prevent prompt disclosure at the trust boundary

Zero failures is suspicious: agent checks that silently ran nothing

The setup

Trap 1: zsh has a log builtin

Trap 2: cargo test --exact with the wrong path passes

Trap 3: an empty variable makes everything fail

Fix

Why it happens

Takeaway

Standards vs Spec review: a second agent found duplicate writes past 1,572 green tests

The setup

What each axis found

Why the tests stayed green

Two more runs, same pattern

How to use it

Takeaway

Date the bug before you park it: fence, drift or silence

The setup

What parking costs

Date the bug

The obvious fix was wrong

What silence looks like

The skill

Takeaway

Every test mocked the helper, and except Exception swallowed its TypeError on every PDF

Setup

What went wrong

Fix

Why

Takeaway

Sandbox-blocked tests called "environmental", then committed

The setup

What went wrong

The same sandbox, the same day

Fix

Why it happens

Takeaway

ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY: your coding agent's pnpm 11 is not your pnpm 10

The setup

What went wrong

Fix

Why

Takeaway

TS2724 "has no exported member" in a git worktree: a symlinked node_modules typechecks the main checkout's source

The setup

What went wrong

Fix

Why

Takeaway

Next.js "Cannot read properties of undefined (reading 'length')" in installChunk: two processes share .next

The setup

What went wrong

Fix

The follow-on trap: next-env.d.ts

Why

Takeaway

An MCP server has more outputs than tool results

Keep the secret off the transcript

Trap 1: zsh has a `log` builtin

Trap 2: `cargo test --exact` with the wrong path passes

The follow-on trap: `next-env.d.ts`