Git on Lessons learned

'Downgrading is unsupported' after a config restore: the update guard was right

Fri, 16 Oct 2026 00:00:00 +0000

After a configuration restore, an updater refused with:

that release predates the gateway runtime these session stores now use. Downgrading is unsupported.

The operator requested an update. The restored configuration selected an older code line than the databases required. Restoring the intended per-install channel corrected target selection; bypassing the guard would have put incompatible code over newer state.

A newer release number can select older capabilities

The checkout followed a development branch. Its session databases already contained runtime-ledger history. The restored configuration backup lacked the install’s channel record, so release selection fell back to the stable channel.

The stable release had a higher patch version, but it came from a release branch without those runtime-ledger tables. Version ordering did not describe the relevant capability ordering.

The historical guard checked two things: whether the proposed target descended from the running commit, and whether a non-descendant target contained the runtime schema marker. It also read the session stores for ledger rows. Its refusal named the affected stores and stopped before moving the checkout.

The implementation explained the consequence: older code could not delete or prune sessions with that runtime history. This was a specific compatibility check, not proof that every forward commit or every future schema change would be safe.

Repair target selection before changing code

The diagnosis followed the configuration lookup:

Identify the actual install path and the channel record associated with it.
Compare that record with the backup that was restored. Check defaults when the record is absent.
Resolve the updater’s proposed target to a commit, rather than relying on its release label.
Compare the target’s ancestry and required state capabilities with the running checkout.
Reassert the intended channel through the updater’s supported configuration command, then run its check-only mode.

The corrected check selected the development branch and reported available forward commits—that verified channel selection. The update itself was deliberately not run, so the record does not establish a completed runtime upgrade.

For a Git-based installation, this read-only check is useful once TARGET is set to the exact resolved commit. Run it inside that installation’s checkout:

git merge-base --is-ancestor HEAD "${TARGET:?Set TARGET to the resolved update commit}"

Exit status 0 means the target descends from the current checkout. Status 1 means it does not; other failures require investigation. A non-descendant can be a valid release-branch target, so this command alone is not a compatibility verdict. Check the particular schema capability named by the refusal.

A config-only restore can create a mixed generation

The database kept its newer state while the restored configuration lost an additive setting. In this implementation, channel records were keyed by install path identity. Restoring an older backup, or moving an install without recreating its record, could therefore change how later updates were selected.

Deployment automation should assert the intended channel after install or restore, then read it back through the supported interface. Do not hard-code a path-derived identifier from another machine. Following a development branch is appropriate only when that is the deliberate operating policy; it is not a universal fix for update errors.

For an intentional rollback, follow the application’s documented procedure for compatible code and pre-upgrade data together. Preserve the current state first, and account for work absent from the older backup. Editing a schema-version value or removing ledger rows merely to silence the guard would discard the evidence it uses to protect the store.

Add an update-target check to your restore drill before allowing the restored installation to update itself.

Date the bug before you park it: fence, drift or silence

Fri, 17 Jul 2026 01:13:00 +0200

My coding agent was splitting a 738-line Python function into steps and found a real bug along the way:

result.issues = validation.issues

That line rebinds the list instead of extending it, so it dropped every issue recorded before validation, including the note that repair mode had just deleted a record in the accounting system. The push still succeeded, and the operator’s report said nothing about the delete.

The agent didn’t fix it. It wrote a known-issues entry and added two @unittest.expectedFailure tests, because the handoff for the refactor said: “characterize, then collapse, then fix separately”. It was the wrong call, and two git log -S runs showed it.

The setup

Claude Code working in a Python repository of an accounting pipeline, tested with unittest;
A long-lived function where several steps append to result.issues, the operator’s per-file report;
A refactor rule that a split must not change behavior.

What parking costs

An expectedFailure marker looks like a decision. The next reader sees a documented, tested, known gap and assumes someone weighed it and chose to keep it. If nobody did, the marker turns an accident into policy and keeps a live bug in the tree behind it.

So before you park a bug, find out whether anyone ever decided it. I asked the agent to run the fence check (Chesterton’s fence: don’t remove a fence until you know why it was put up).

Date the bug

A bug is a relationship between two things: this line and the code it gets wrong — date both.

git log --oneline -S'result.issues = validation.issues' -- <path>
git log --oneline -S'result.issues.append' -- <path>

The result went into the fix’s commit message:

2026-04-16 rebind written. result.issues had NO prior writers;
 assigning validation's list into it lost nothing.
2026-04-19 + vendor mismatch trail above it +3d
2026-04-23 + REPAIR delete note above it +7d
2026-04-24 + registry retry verdict above it +8d
2026-05-27 + self-billed reroute above it +41d

The rebind was correct when it was written. Over the next 41 days, four commits added writers above it, and none of them touched it. Nobody decided to discard those issues; the code predates the thing it breaks.

That gives one of three verdicts:

Fence: someone decided this, in a commit message, a spec’s non-goals or an ADR. Leave it.
Drift: the code predates what it is wrong about, or its author never touched the site. Fix it now, and put the dates in the commit message so a reviewer can see you didn’t remove a fence.
Silence: the history says nothing either way. Park it, and say plainly that nobody decided it.

This one was drift, so it got fixed the same night. It had stayed hidden because the only test that reached this code patched the whole function body out. The split made the body reachable, and the first end-to-end repair test found the bug.

The obvious fix was wrong

A drift verdict says nobody meant the bug; you still have to find out what depends on it. The one-word fix:

result.issues.extend(validation.issues)

would have broken an alias. A later step appends registry failures to result.issues, and those reach the invoice’s note only because the rebind made result.issues and validation.issues the same list, which the note builder reads:

result.issues = validation.issues # one list, two names
apply_registry_failure(result) # appends to result.issues ...
note = build_note(validation) # ... and the note sees it through the alias

With .extend(), every invoice that failed the business-registry check would have lost that warning from its note: one bug traded for a quieter one. The fix kept the rebind and put the earlier issues back after the later writers ran:

earlier_issues = list(result.issues)
result.issues = validation.issues
apply_registry_failure(result)
# ... the other writers ...
result.issues = earlier_issues + result.issues

The tests were written to catch both mistakes: reverting the fix fails the four tests that check the earlier issues survive, and the naive .extend() fails the one that checks the registry warning still reaches the note. The known-issues entry was deleted, and the expectedFailure markers became ordinary tests.

What silence looks like

That night, four other bugs were dated as drift and fixed. One wasn’t. Debug files were named from the input’s stem, so invoice.docx and invoice.pdf in one batch wrote the same file, and the later one won. That naming shipped on a pipeline that already read several formats, so the code didn’t predate the collision. No commit message or spec mentioned it either: stem is just the obvious way to name a sidecar file. That one stayed a known-issues entry, with the dating written into it.

The skill

The fence check is a Claude Code skill I had written down earlier that evening. The agent had it and didn’t run it until asked, so its memory now says to run it the moment it finds a bug, before choosing between a fix and a known-issues entry. Here it is verbatim. The examples and the docs/superpowers/ paths are from my repository; swap in your own spec directory.

---
name: checking-the-fence
description: Date a bug before fixing it — Chesterton's fence. Use when you have found code that looks wrong and are about to change it, when you are about to delete something that looks dead, or when another skill needs the fence check.
---

# Checking the Fence

A bug you found is a **fence** across a field. Before you take it down, find who put it there. Here that is written down twice — in `git log` and in `docs/superpowers/` — and is usually decisive in a minute.

Two verdicts. They are the whole skill:

- **Fence** — someone decided this. Leave it, and record an ADR if the next reader will reach for the same fix you just did.
- **Drift** — nobody decided it. The code predates the thing it is wrong about, or an author was never told which files the job touched. Fix it.

Fences here are real, not hypothetical: `docs/adr/0002` exists only to stop a reader from deleting emoji handling that has no inbound traffic and looks dead.

## Steps

Stop at the first step that returns a verdict.

### 1. Date the code

```
git log --oneline -S'<symbol>' --all -- <path>
git log -1 --format=%B <sha>
```

Commit messages here carry the rule the author meant to encode, and the rule is often the thing the bug breaks. `_PARSER_PRIORITY`'s own commit says *"process deterministic parsers before PDFs/images"* — while the table shipped the cheapest format last.

**Done when** you can name the commit and quote the rule.

### 2. Date what it is wrong about

A bug is a relationship: a table missing a format, a copy missing an entry, a switch missing a branch. Date the other side too.

Code cannot have decided about something that did not exist yet. `_PARSER_PRIORITY` (2026-04-27) predates isdoc (2026-06-01) by five weeks.

**Done when** you have both dates. Predates → **drift**. Stop.

### 3. Read the spec

```
grep -rli '<feature>' docs/superpowers/specs docs/superpowers/plans
```

Read the **Non-goals** first — that is where a deliberate gap is named. A spec that scopes the gap out is a **fence**. A spec whose own words the bug contradicts is **drift**: the ISDOC spec calls it *"a deterministic fast path before Claude"*.

**Done when** you have quoted the line that scopes it out, or established that no spec names it.

### 4. Read the blast pattern

```
git show --stat <sha>
```

Which files that commit touched, against which it needed to. An author who updated three of five consumers was not deciding about the other two — nothing told them. The isdoc and docx commits landed a day apart and touched different subsets.

**Done when** you can say whether the author addressed the site at all. Never addressed → **drift**.

### 5. Was it born wrong?

For a forked copy, compare it against its source **on the day it was copied**, not today.

```
git show <sha>:<source-path>
```

`coverage_report.py` shipped without `.json` on 2026-04-24, and the map it was copied from carried `.json` that same day. Born forked, then rotted twice more.

**Done when** you know whether the gap is drift-over-time or a bad copy. Either is **drift**.

## Applying the verdict

**Drift** — fix it, and say in the commit message what you found: the dates, and that nothing decided it. That is what stops a reviewer from wondering whether you removed a fence.

**Fence** — leave it. Offer an ADR when a future reader would reach for the same fix; that reader is usually the next architecture review, or you.

**Neither** — the history is silent and no spec mentions it. Say so plainly and let the user decide; silence is not consent, and it is not a fence either.

Save it as .claude/skills/checking-the-fence/SKILL.md in your repository, and tell the agent in CLAUDE.md or AGENTS.md to run it before it records a bug as a known issue.

Takeaway

A parked bug looks like a decision, so check whether anyone made one. If the code predates what it breaks, fix it now, and check the mechanism first: knowing nobody meant the bug doesn’t tell you what depends on it.

Next time your agent adds an expectedFailure or a known-issues entry, ask it for the two dates first.

TS2724 "has no exported member" in a git worktree: a symlinked node_modules typechecks the main checkout's source

Mon, 22 Jun 2026 19:21:00 +0200

A coding agent cherry-picked a subagent’s commit into an integration worktree, ran pnpm typecheck and got:

apps/web/lib/uploads.ts(3,59): error TS2724: '"@app/persistence"' has no exported member named 'recordRawUpload'. Did you mean 'recordUpload'?

recordRawUpload was right there, exported from packages/persistence/src/index.ts in that worktree. If tsc can’t see an export that you can, and the worktree’s node_modules is a symlink to another checkout, tsc is reading the other checkout’s source.

The setup

A pnpm 10 monorepo on macOS. The workspace packages @app/* point exports straight at ./src/index.ts, so there is no build step between packages.
An orchestrating agent that creates a git worktree, a branch, and a subagent per issue, and cherry-picks finished work into an integration worktree.
A full disk.

What went wrong

Every worktree had its own install, and du counted up to 513 MB of node_modules in each. A dozen worktrees later, the disk was full:

Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk3s5 460Gi 420Gi 116Mi 100% 6.6M 1.2M 85% /System/Volumes/Data

To save space, the agent replaced the node_modules of each new worktree (the root, apps/web and every package) with a symlink to the main checkout’s copy:

ln -s ../project/node_modules ../project-integration/node_modules
ln -s ../../../project/apps/web/node_modules ../project-integration/apps/web/node_modules

The first typecheck after that passed. Later, a change that added an export to @app/persistence and used it in apps/web failed with TS2724. readlink shows why:

$ readlink apps/web/node_modules/@app/persistence
../../../../packages/persistence

pnpm links workspace packages with relative symlinks. A relative link resolves from the directory it really sits in, and that directory is now in the main checkout. So @app/persistence in the worktree was /Users/me/project/packages/persistence, the main checkout’s source, which didn’t have the new function yet.

The typecheck that passed also validated the main checkout.

Two subagents in their own worktrees hit the same error. One deleted the links, reinstalled, and its typecheck passed. The other wrote “Validation blocked by worktree node_modules links targeting older shared package. Source diff reviewed; commit now.” and committed without a passing typecheck.

Fix

Give each worktree its own install. pnpm keeps package files in a content-addressable store, so an offline install only links them into place:

rm node_modules apps/web/node_modules packages/*/node_modules # the symlinks
pnpm install --offline --frozen-lockfile

This took 3.6 seconds and reported resolved 378, reused 378, downloaded 0. If the store is missing a package, --offline fails; drop the flag.

Then check that every workspace package resolves inside the worktree. Run it with bash: zsh aborts on the first glob that matches nothing (no matches found), and the agent’s own fix hit exactly that.

#!/usr/bin/env bash
# Print workspace packages that resolve outside this checkout.
root=$(pwd -P)
for link in node_modules/@app/* */*/node_modules/@app/*; do
 [ -e "$link" ] || continue
 target=$(realpath "$link")
 case $target in "$root"/*) ;; *) echo "$link -> $target" ;; esac
done

No output means every workspace package comes from this worktree.

If you have to share node_modules, share only the third-party packages. The agent’s fix was an overlay: a real node_modules directory per package, with every entry symlinked to the main checkout except @app, whose entries point at the worktree’s own packages. Copying the main checkout’s @app directory with cp -RP gives the same result, because the copied relative links resolve from their new place.

Why

TypeScript (with the default preserveSymlinks: false) and Node both resolve a package to its real path. Third-party packages are the same files in every checkout that shares a lockfile, so following the link to the main checkout is harmless. Workspace packages are your source code, and that differs between checkouts by design. Once a branch changes its dependencies, the third-party half goes stale as well.

The disk savings were probably smaller than they looked. pnpm clones or hard-links files from its store, and du counts those blocks again in every tree, so its per-worktree numbers overstate what each install cost. The disk was also full for other reasons, and nobody measured how much the links saved.

Takeaway

A worktree that borrows another checkout’s node_modules gets that checkout’s workspace packages too, and typecheck and tests then run against the wrong code, passing or failing. Run pnpm install --offline in every new worktree.

If your agents create worktrees, run the check above in one of them today.