Authentication on Lessons learned

Waiting for logout before login still left a race inside the handshake

Mon, 21 Sep 2026 14:49:00 +0200

Correction 2026-10-31: The earlier version attributed the reported failure to an idle timer. Later source review ruled out that timer for this app. The injected-teardown reproduction and exclusion-gate fix remain valid; the production trigger is unknown.

Users authenticated with Face ID, and the login failed anyway with a generic “login failed” error. The password was right, and the biometric enrollment was fine. The server’s log for the affected user showed the last handshake step arriving on a session in its initial state, with no user bound to it.

The app waited for any teardown already running before starting login. A dedicated test injected a new teardown after that wait and reproduced the same server-state signature. Holding an exclusion gate for the whole handshake fixed that reproduction, but did not establish the cause of the reported incident.

Setup

A stateful, multi-step login (SRP-style): start a session, send the identity, send the proof, finish. The server tracks progress per session cookie.
A session teardown: sign out on the server, then clear the local cookie jar. begin() returns immediately, and concurrent begin() calls join one teardown.
An earlier fix for a related bug: before its handshake, login called waitForCompletion() so a sign-out still in flight couldn’t destroy the new session.
The reproduction explicitly calls begin() during the handshake. It does not rely on an idle timer.

login: { credential in
 await sessionTeardown.waitForCompletion() // once, before the handshake
 return await client.login(credential) // four round trips
}

Reproduce with the real stack

A dedicated live test drove the production login code against a development server and fired begin() at a fixed delay into the handshake. At 500 ms:

start -> {"state":"new"}
send identity -> {"state":"identified", "salt":...}
send proof (M1) -> {"state":"proof-verified", "M2":...}
logout() <- the teardown fires here
POST /logout
finish (keys sent)
Logout success
Session teardown: cookie jar cleared
Error: state "new"
-> failed(.other)

This matched the server-state signature logged for the user. In the reproduction, the one-time wait had nothing to wait for: no teardown was running when login started. The test started one between two round trips.

The trap inside the trap: which effect does the damage

The code comments blamed the cookie-jar clear: the concluding step would go out without a session cookie and land on an anonymous session. That would suggest a small fix: defer only the local cleanup.

In the reproduction’s trace, the cookie jar was cleared after the finish step had already gone out with valid cookies. The server sign-out reset the half-authenticated session, and the concluding step then arrived at a session the server had put back to new. Deferring only the jar clear would still fail this exact trace.

Before you choose what to defer, read the ordering on the wire. The local effect you can see easily may not be the one that broke things.

Fix: hold exclusion for the whole protocol

The teardown gets a gate. Login holds it from the first round trip to the last. A begin() that arrives while the gate is held is honored but deferred—neither its sign-out nor its cookie clear runs until the handshake releases the gate.

private struct TeardownState {
 var inFlight: Task<Void, Never>?
 var handshakes = 0 // a count, so overlapping logins can't release each other's gate
 var isHeldBack = false
}

begin: {
 state.withValue { s in
 guard s.inFlight == nil else { return } // join the running one
 guard s.handshakes == 0 else { s.isHeldBack = true; return }
 s.inFlight = Task { await runTeardown() } // clears inFlight when done
 }
},
beginHandshake: {
 // Wait out a running teardown, then claim the gate. Loop, because the slot
 // is only free on a later pass.
 while let running = state.withValue({ s -> Task<Void, Never>? in
 guard let t = s.inFlight else { s.handshakes += 1; return nil }
 return t
 }) {
 await running.value
 }
},
endHandshake: {
 state.withValue { s in
 s.handshakes -= 1
 guard s.handshakes == 0, s.isHeldBack else { return }
 s.isHeldBack = false
 s.inFlight = Task { await runTeardown() }
 }
}

func excludingTeardown<T>(_ body: () async -> T) async -> T {
 await beginHandshake()
 defer { endHandshake() }
 return await body()
}

login: { credential in
 await sessionTeardown.excludingTeardown {
 await client.login(credential)
 }
}

The other properties stayed the same. begin() still returns immediately. A second begin() still means one teardown, whether that teardown is running or held back. A teardown whose sign-out throws still releases its waiters. Logout still happens: it runs right after login, which then ends in a signed-out state on purpose.

Later source review found that this app created its idle timer only after the full login completed and dropped it when the login screen appeared. The shipped revision had the same wiring. The earlier claim that this timer disproved a signed-in-only assumption was withdrawn. The gate provides exclusion without depending on that reachability argument.

A sibling app did retain its timer after logout and needed a signed-in guard. That was a separate timer-lifetime defect, not evidence that the timer caused this app’s reported failure.

Verification

Unit test for the new direction. A teardown begun inside excludingTeardown runs neither effect until the body returns. The expected order is [handshake, signOut, clearCookieJar]. Inside the body, the test awaits waitForCompletion() rather than Task.yield(), so a buggy teardown is guaranteed a chance to run. A yield would leave that to the scheduler.
Unit test for the old direction. A teardown already running still finishes before the handshake starts. That is the guarantee the earlier fix provided.
Injected-teardown sweep. The reproduction fires the teardown at 100, 200, 300, 400, 500 and 700 ms. It had been marked XCTExpectFailure. After the fix, every delay logged in:

teardown at 100ms -> loggedIn
teardown at 200ms -> loggedIn
...
teardown at 700ms -> loggedIn

The expected-failure marker was removed in the same commit.

What this doesn’t prove

The reproduction establishes a failure mechanism and verifies the exclusion fix. It does not prove that this mechanism caused the reported incident or identify its triggering call. We ruled out the idle timer for this app; the field trigger remained unknown.
The dedicated reproduction and the unit tests verify the fix. The live test class still exited with failures. They came from a separate host-app dependency problem in the test process, and another test in that class still tracks an unrelated second-login bug. Don’t read “the sweep passes” as “the live suite is green”.
The ordinary test plan passed, but live tests are opt-in there, so it says nothing about the server interaction.
Which delays hit the window depends on latency to the server. A sweep is better than a single delay, but it is still a sample.

Takeaways

Waiting once for cleanup before a multi-step protocol protects only that instant. If cleanup can start during the protocol, exclude it for the whole protocol.
Defer cleanup, don’t drop it. Queue the request and run it when the gate is released.
Count holders instead of using a flag, so two overlapping logins can’t release the gate from under each other.
Read the wire ordering before you choose what to defer. Here, the server sign-out did the damage, not the local cookie clear that everyone suspected.
A matching reproduction does not identify the production trigger. Trace the initiating call in the affected build before naming it as the cause of an incident.

When the server must compute on confidential data, split the data by who may read it

Sun, 21 Jun 2026 20:00:00 +0200

I designed a web app that computes a person’s tax on their investment trades. Professional advisory firms run it for their own clients. Clients connect broker APIs or upload statements, and the server runs lot matching, currency conversion, and the holding-period tests. A client with a messy case can hand it to an advisor.

The confidentiality requirement may sound contradictory. The advisor owes the client professional secrecy from the first contact. The platform operator should not be able to read client data at rest, and the same goes for a stolen backup, a tenant boundary bug, or a subpoena served on the platform. The server still has to compute on all of it, including in background jobs while the user is offline.

Neither usual answer works:

Disk or database encryption protects against a stolen disk. The operator, its DBAs, and a subpoena still get plaintext.
Client-side end-to-end encryption fails because the server is the calculator. Background sync, recomputation after new data, and the advisor’s view all need plaintext on the server. In a web app it is weaker than it sounds anyway: the server ships the JavaScript on every load, so an operator that serves malicious code gets the password.

What worked was a question I had skipped: which fields does the computation actually read?

List what the engine needs

For every realized event, the tax engine needs a date, a quantity, amounts in the reporting currency, fees, an event type, and an asset category (security, crypto, derivative). To match lots, it has to know that two trades are the same instrument. It doesn’t need the instrument’s name, ticker, or ISIN, and it doesn’t need the broker, account number, or person.

One step needs real identity: classifying an instrument into a category. A broker may report a CFD under “stocks”, and a share can be fractional or whole. That happens once, at ingestion, inside a per-broker adapter. The adapter sees identity in memory, emits a de-identified, categorized record, and the identity goes elsewhere.

So the data splits into two zones, each with its own readers:

	Zone 1: tax skeleton	Zone 2: identity vault
Contents	surrogate instrument ID, category, dates, quantity, amounts, fees, event type	surrogate → name/ticker/ISIN, broker, account, personal data
Encrypted under	per-tenant key encryption key in a KMS (envelope encryption)	key chain rooted in the user’s password
Readers	engine (in memory, during compute), the firm after handoff	the user in session, the firm after handoff
Platform operator	ciphertext at rest; if it decrypts, it gets anonymous arithmetic	ciphertext only
Background jobs	read and write	write only, through a public-key layer
Lost password	survives; the case can still be computed and filed	lost unless the user has the recovery code

Zone 1 is intentionally not under the user’s key. Under the user’s key, the background sync, the in-year recompute, and the advisor’s computation would all stop working as soon as the user logged out. Background jobs write to Zone 2 by encrypting to the user’s public key. Only a logged-in user can decrypt.

Surrogate IDs: a keyed hash, scoped per user

Lot matching groups trades by instrument, so the skeleton needs a stable ID that is not the instrument. The design is an HMAC with a per-user key held in the KMS:

// Stable across one user's accounts and years, unlinkable across users.
export async function deriveSurrogateInstrumentId(userId: string, instrument: string, broker: string) {
 if (!instrument || !broker) throw new Error("surrogate requires instrument identity and broker");
 return kmsHmac(`surrogate:user:${userId}`, JSON.stringify([instrument, broker]));
}

Each alternative breaks one requirement:

A plain hash of the ISIN falls to a dictionary attack. The set of listed instruments is small, so hashing them all takes seconds.
One global HMAC key makes the same instrument correlate across every user in a dump.
A random ID per trade cannot be recomputed. Next year’s sync, or a second account at another broker, has to land on the same ID without reading the vault, or carried-in lots stop matching.

The keyed hash protects a database dump. An operator who can call the KMS live can still test “does this user hold this instrument”. That is past the threat model’s at-rest boundary, so write it down instead of claiming more.

Access and liability begin at the same instant

Before handoff, the firm’s advisors have no access to a prospect’s case. For identity, the user’s key enforces that cryptographically. The anonymous skeleton sits under the firm’s own key, so for it, access control is the barrier. It matches the legal position: no engagement contract, no professional liability, only precontractual confidentiality.

Handoff is one atomic event. The engagement contract forms, the firm’s anti-money-laundering (AML) and know-your-customer (KYC) duties start, and the vault’s relevant portion is rewrapped to the firm’s key. Data access and liability start together, so neither exists without the other.

The coding agent’s first draft at this step was wrong. It proposed default-minimal disclosure: share only the vault fields the flagged items needed. I rejected it because an advisor who takes on liability and an AML duty cannot sign off on a case half-blind. The boundary that held was purpose, not field sensitivity: everything the past outcome was built from transfers in full, and anything forward-looking (plans, rebalancing ideas, notes) never transfers. Data minimization still applies, but the line runs between past and future, not through the case.

One password, auto-unlock, never transmitted: pick an aPAKE

The first plan was standard session login plus a client-side KDF for the vault key, with OPAQUE as a later upgrade. That leaves two bad choices:

Two passwords: one for login, sent to the server, and one for the vault, used only in the browser. Users hate it.
One password, still POSTed for login, also used to derive the vault key. It gives auto-unlock, and it quietly hands the operator the vault password on every login. Zone 2 stops protecting anything.

With a standard login, one password cannot also stay off the server. An asymmetric PAKE fixes the login side. OPAQUE (RFC 9807) authenticates without the server learning the password, and gives the client an export_key that the server never sees. Root the vault in it:

// K_vault = HKDF(export_key); unwraps user key → RSA private key → vault entries
function deriveVaultKeyBytes(exportKey: Uint8Array): Uint8Array {
 if (exportKey.length === 0) throw new Error("export_key is required");
 return hkdf(sha256, exportKey, new Uint8Array(0), K_VAULT_INFO, 32);
}

A password reset is a new OPAQUE registration with a new export_key. The mandatory recovery code rewraps the vault chain to the new root, and Zone 1 is untouched. Use a vetted library: the registration and login flows and the export_key handling are now part of the vault’s security.

The tenant boundary belongs in the database

Each firm is a tenant, and one firm’s prospects must never show up in another firm’s queries. A forgotten WHERE firm_id = ? should return nothing, not another tenant’s rows. With Postgres row-level security, three details decide whether it fails closed:

alter table client owner to app_migrator; -- the app role is not the owner
alter table client force row level security; -- the owner obeys policies too
create policy firm_isolation on client
 using (firm_id = current_setting('app.firm_id')::uuid)
 with check (firm_id = current_setting('app.firm_id')::uuid);

await db.transaction().execute(async (tx) => {
 await sql`select set_config('app.firm_id', ${firmId}, true)`.execute(tx); // true = transaction-local
 return work(tx);
});

Table owners bypass RLS unless it is forced. Superusers and BYPASSRLS roles bypass it always, so the app must connect as neither.
current_setting without missing_ok, plus the ::uuid cast, raises when the tenant was never set or is empty. A missing tenant is an error, not an empty match.
set_config(..., true) resets at commit or rollback. With false, a pooled connection carries one firm’s ID into the next request, and a transaction-mode pooler can hand the setting to a different client.

Write down the floors

The design makes specific claims and lists what it does not cover. That list is part of the design:

Plaintext exists in memory during ingestion and compute. The claim is “zero to the operator at rest”, never “the platform never sees it”.
amount ÷ quantity approximates a unit price that can be matched against market data. It is exploitable only by someone who already decrypted Zone 1.
The web JavaScript trust gap stays open. OPAQUE stops offline cracking of a stolen verifier table, not a malicious bundle. A native client is the eventual fix.
An API-connected account reveals which broker it is, because the token names it. A manual upload keeps the broker in Zone 2. The user gets that choice in the UI.
A forgotten password without the recovery code loses the labels. The skeleton survives, and the user is told to keep their own copies of the source statements.

Takeaway

When the server has to be the calculator, don’t choose between “encrypt the disk” and “encrypt everything client-side”. List the readers (user, professional, engine, operator) and what each one needs. Then check whether the computation can run without identity. When it can, identity moves into a zone the computation never opens, and the remaining decisions follow from that: keyed surrogates, a handoff that rewraps keys, an aPAKE for the single password.

In practice, I ran the design as an interview with a coding agent. It asked one question at a time, proposed options with a recommendation, and recorded each decision as an ADR with a “Why this is non-obvious” section: the design a reader would expect, and why it was rejected here. Several recommendations were reasonable but wrong. Two of my corrections removed whole planned pieces: the default-minimal handoff, and an instrument catalog that event-driven classification made unnecessary.

Codex 401 in an agent shell: check credentials before blaming API key precedence

Tue, 05 May 2026 00:00:00 +0000

A regular terminal reported a ChatGPT login, but Codex requests inside an agent’s shell failed with:

HTTP error: 401 Unauthorized

An inherited OPENAI_API_KEY looked suspicious. The useful next step was a controlled credential-context comparison. The recorded investigation never proved that the key overrode the ChatGPT login, and it never showed a successful retry after removing it.

What the May 5 investigation established

The installed CLI was codex-cli 0.128.0. The user reported that codex login status returned Logged in using ChatGPT before launching the agent. Inside the agent, the same status command returned Not logged in, and this request repeatedly failed:

codex exec --skip-git-repo-check "echo hello"

An early environment check found OPENAI_API_KEY present. A separate request to the OpenAI models endpoint, explicitly using that variable, returned HTTP 401. That established that the credential failed that direct probe. It did not establish which credential Codex selected.

Later checks found the variable absent, yet Codex still failed. One retry explicitly supplied an empty OPENAI_API_KEY and also received 401 responses. The chain contains no controlled env -u comparison that restored subscription access. The document-cleanup task continued with Python; Codex authentication was not shown to recover.

Calling this a confirmed case of an environment key shadowing OAuth would overstate the evidence. The login-status mismatch deserved investigation before declaring the subscription token expired or replacing it with an API key.

Compare the launch contexts without exposing secrets

Run these checks locally in both the working terminal and the failing agent shell:

command -v codex
codex --version
codex login status
python3 - <<'PY'
import os
for name in ("HOME", "CODEX_HOME", "OPENAI_API_KEY", "CODEX_API_KEY"):
 print(f"{name}: {'set' if name in os.environ else 'unset'}")
PY

Keep login-status output private if it includes credential details. The Python check prints names and presence only. Do not dump the environment, search credential files, or paste tokens into a troubleshooting conversation.

Compare the actual HOME and CODEX_HOME paths privately, too. Presence alone cannot tell you whether two processes use the same directory. Check the executable and launch configuration for differences before changing authentication.

OpenAI’s authentication documentation distinguishes ChatGPT subscription sign-in from API-key access. It also describes credential storage in a file under CODEX_HOME or an OS credential store. A profile HOME mismatch or unavailable credential store is a separate explanation for a missing login.

To isolate the suspected variable on a Unix-like system, keep the same executable, working directory, HOME, and CODEX_HOME, then compare:

codex login status
env -u OPENAI_API_KEY codex login status

Removing one variable affects only the second process. It does not delete the saved login or change the parent shell. If you intend to make a model request, repeat the same minimal prompt with and without that variable and compare completion and exit status. A status check alone does not prove a request will succeed. This paired request test was not completed in the May investigation.

Verification on October 31: avoid a universal precedence rule

A separate check on 2026-10-31 used installed Codex 0.166.0 with isolated, empty HOME and CODEX_HOME directories. codex login status returned Not logged in both without a key and with a dummy OPENAI_API_KEY. No model request or real credential was used. This checks status behavior without a saved login; it does not test precedence over OAuth.

The upstream authentication loader , inspected that day, has an environment-key precedence branch for CODEX_API_KEY, conditional on that mechanism being enabled and API-key authentication being allowed. That is not a universal rule about OPENAI_API_KEY, nor proof of what version 0.128.0 selected in May.

The earlier trusted-directory error was separate: --skip-git-repo-check bypassed that check and exposed the authentication failure. Initializing unrelated directories as Git repositories would not repair credentials.

Before logging out or deleting a working login, compare the credential context of your two launch paths and change one variable at a time.