<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Backup on Lessons learned</title><link>https://eghuro.cz/tags/backup/</link><description>Recent content in Backup on Lessons learned</description><generator>Hugo</generator><language>en-us</language><copyright>© Alexandr Mansurov · Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)</copyright><lastBuildDate>Sat, 31 Oct 2026 22:29:00 +0100</lastBuildDate><atom:link href="https://eghuro.cz/tags/backup/index.xml" rel="self" type="application/rss+xml"/><item><title>Backing up ~/.claude and ~/.codex without archiving your AI logins</title><link>https://eghuro.cz/posts/ai-client-local-state-backup/</link><pubDate>Sat, 31 Oct 2026 22:29:00 +0100</pubDate><guid>https://eghuro.cz/posts/ai-client-local-state-backup/</guid><description>&lt;p&gt;If you back up the whole &lt;code&gt;~/.claude&lt;/code&gt; or &lt;code&gt;~/.codex&lt;/code&gt; folder, the restore works, and the backup now holds everything an infostealer wants. The same folder holds your settings, every session transcript in plaintext, and, on Linux, the login itself:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;~/.claude/.credentials.json # Claude Code login, mode 0600&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;~/.codex/auth.json # Codex login when file storage is in use&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;~/.claude/projects/*/*.jsonl # transcripts: prompts, tool output, file contents&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Split the folder into two lists. Back up what you wrote: settings, instructions, skills, hooks, and memory. Keep transcripts as optional encrypted data. Leave login files out of routine backups and log in again after a restore.&lt;/p&gt;&#10;&#10;&lt;h2 id="where-ai-client-state-lives-on-macos-and-linux"&gt;Where AI client state lives on macOS and Linux &lt;/h2&gt;&#10;&lt;p&gt;I checked these paths against Claude Code 2.1.316, Claude Desktop for Linux 4.4212.0, ChatGPT for macOS 1.2026.183, the Codex desktop app for macOS 26.1027.30824, and Codex source at commit &lt;code&gt;67f0994&lt;/code&gt;. &amp;ldquo;Documented&amp;rdquo; means the vendor docs name the path. &amp;ldquo;Observed&amp;rdquo; means I found it by inspecting the official build so that it can change in any release.&lt;/p&gt;&#10;&lt;table&gt;&#10;&#9;&lt;thead&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Client&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;State root&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Login material&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Evidence&lt;/th&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/thead&gt;&#10;&#9;&lt;tbody&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Claude Code&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;~/.claude/&lt;/code&gt; and &lt;code&gt;~/.claude.json&lt;/code&gt;, both under &lt;code&gt;CLAUDE_CONFIG_DIR&lt;/code&gt; if set&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Keychain on macOS, or &lt;code&gt;~/.claude/.credentials.json&lt;/code&gt; when the Keychain rejects the write (locked in an SSH session, for example). On Linux, always that file&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Documented&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Claude Desktop&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;macOS: &lt;code&gt;~/Library/Application Support/Claude/&lt;/code&gt;. Linux (beta): &lt;code&gt;${XDG_CONFIG_HOME:-~/.config}/Claude/&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Electron cookies and Local Storage, extension secrets encrypted with &lt;code&gt;safeStorage&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;MCP config documented, rest observed&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;ChatGPT desktop, macOS only&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;~/Library/Application Support/com.openai.chat/&lt;/code&gt;, &lt;code&gt;~/Library/Group Containers/2DC432GLL2.group.com.openai.chat/&lt;/code&gt;, &lt;code&gt;~/Library/Preferences/com.openai.chat.plist&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Keychain items in OpenAI&amp;rsquo;s access group&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Observed&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Codex desktop, macOS only&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;~/Library/Application Support/Codex/&lt;/code&gt;, plus &lt;code&gt;~/.codex&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Shared with Codex CLI&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Observed&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Codex CLI&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;${CODEX_HOME:-~/.codex}&lt;/code&gt;. &lt;code&gt;CODEX_SQLITE_HOME&lt;/code&gt;, &lt;code&gt;sqlite_home&lt;/code&gt; and &lt;code&gt;log_dir&lt;/code&gt; can move parts of it&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;auth.json&lt;/code&gt; in &lt;code&gt;file&lt;/code&gt; mode or as the &lt;code&gt;auto&lt;/code&gt; fallback; &lt;code&gt;.credentials.json&lt;/code&gt; for MCP OAuth without a keyring&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Documented and source&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/tbody&gt;&#10;&lt;/table&gt;&#10;&lt;p&gt;Desktop sessions that run Claude Code write to &lt;code&gt;~/.claude/&lt;/code&gt;, not to the Electron folder, so include both.&lt;/p&gt;&#10;&#10;&lt;h2 id="what-to-back-up-and-what-to-leave-out"&gt;What to back up and what to leave out &lt;/h2&gt;&#10;&lt;table&gt;&#10;&#9;&lt;thead&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Treatment&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Claude Code&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Codex&lt;/th&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/thead&gt;&#10;&#9;&lt;tbody&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Back up, versioned&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;settings.json&lt;/code&gt;, &lt;code&gt;CLAUDE.md&lt;/code&gt;, &lt;code&gt;rules/&lt;/code&gt;, &lt;code&gt;skills/&lt;/code&gt;, &lt;code&gt;commands/&lt;/code&gt;, &lt;code&gt;agents/&lt;/code&gt;, &lt;code&gt;workflows/&lt;/code&gt;, &lt;code&gt;output-styles/&lt;/code&gt;, &lt;code&gt;keybindings.json&lt;/code&gt;, &lt;code&gt;projects/*/memory/&lt;/code&gt;, &lt;code&gt;agent-memory/&lt;/code&gt;, &lt;code&gt;plugins/installed_plugins.json&lt;/code&gt; as a manifest&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;config.toml&lt;/code&gt;, &lt;code&gt;*.config.toml&lt;/code&gt;, &lt;code&gt;hooks.json&lt;/code&gt;, &lt;code&gt;rules/&lt;/code&gt;, &lt;code&gt;skills/&lt;/code&gt;, agent-role files&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Back up after a secret check&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;~/.claude.json&lt;/code&gt; (personal MCP servers, trust decisions)&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;MCP &lt;code&gt;env&lt;/code&gt; blocks in &lt;code&gt;config.toml&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Optional, encrypted, short retention&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;projects/*/*.jsonl&lt;/code&gt;, &lt;code&gt;subagents/&lt;/code&gt;, &lt;code&gt;tool-results/&lt;/code&gt;, &lt;code&gt;history.jsonl&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;sessions/YYYY/MM/DD/rollout-*.jsonl&lt;/code&gt;, &lt;code&gt;archived_sessions/&lt;/code&gt;, &lt;code&gt;history.jsonl&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Exclude&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;.credentials.json&lt;/code&gt;, &lt;code&gt;file-history/&lt;/code&gt;, &lt;code&gt;paste-cache/&lt;/code&gt;, &lt;code&gt;debug/&lt;/code&gt;, &lt;code&gt;shell-snapshots/&lt;/code&gt;, &lt;code&gt;session-env/&lt;/code&gt;, &lt;code&gt;backups/&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;auth.json&lt;/code&gt;, &lt;code&gt;.credentials.json&lt;/code&gt;, &lt;code&gt;log/&lt;/code&gt;, &lt;code&gt;logs_2.sqlite&lt;/code&gt;, &lt;code&gt;shell_snapshots/&lt;/code&gt;, &lt;code&gt;packages/&lt;/code&gt;, caches&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/tbody&gt;&#10;&lt;/table&gt;&#10;&lt;p&gt;For Claude Desktop, keep &lt;code&gt;claude_desktop_config.json&lt;/code&gt; after checking its MCP &lt;code&gt;env&lt;/code&gt; blocks, plus a list of installed extensions and where each came from. Reinstall extensions instead of restoring their executable caches. The ChatGPT app&amp;rsquo;s local databases are caches of a server-side account; use the account&amp;rsquo;s data export for conversations you want to keep.&lt;/p&gt;&#10;&#10;&lt;h2 id="why-copying-the-whole-folder-is-the-wrong-default"&gt;Why copying the whole folder is the wrong default &lt;/h2&gt;&#10;&lt;p&gt;Transcripts are plaintext. &lt;a&#10; href="https://code.claude.com/docs/en/claude-directory.md"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;Anthropic&amp;rsquo;s docs&lt;/a&gt;&#10; say that anything passing through a tool is written to the session &lt;code&gt;.jsonl&lt;/code&gt;, so a &lt;code&gt;.env&lt;/code&gt; file a tool reads or a token a command prints is in there. Codex rollouts record tool activity the same way.&lt;/p&gt;&#10;&lt;p&gt;Your backup also keeps transcripts after the client has deleted them. Claude Code deletes transcripts after &lt;code&gt;cleanupPeriodDays&lt;/code&gt; (default 30), but keeps sessions started, or last continued in Claude Desktop or Cowork at any age unless you set &lt;code&gt;desktopSessionCleanupPeriodDays&lt;/code&gt;. A backup with a year of history still holds a transcript eleven months after the client deleted it.&lt;/p&gt;&#10;&lt;p&gt;A restore can also run code. Hooks, MCP server commands, skills, plugins, desktop extensions, and shell snapshots all execute something. If malware edited them, a full restore puts it back on the clean machine.&lt;/p&gt;&#10;&lt;p&gt;The Keychain protects less than it seems to. It stops someone who copies the disk offline, but malware running as your user can still request the item. The Linux desktop build can also fall back to Chromium&amp;rsquo;s basic password store when no wallet is running.&lt;/p&gt;&#10;&#10;&lt;h2 id="dead-ends-worth-skipping"&gt;Dead ends worth skipping &lt;/h2&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;Guessing Linux paths from macOS ones.&lt;/strong&gt; Replacing &lt;code&gt;Library/Application Support&lt;/code&gt; with &lt;code&gt;.config&lt;/code&gt; works for Claude Desktop, which has a Linux beta. OpenAI ships no Linux desktop app for ChatGPT or Codex. The Codex source has Linux branches, but there is no supported Linux build. On Linux, your ChatGPT state lives in the browser profile.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Copying only the &lt;code&gt;.sqlite&lt;/code&gt; files.&lt;/strong&gt; Codex keeps state in &lt;code&gt;state_5.sqlite&lt;/code&gt;, &lt;code&gt;thread_history_1.sqlite&lt;/code&gt;, &lt;code&gt;memories_1.sqlite&lt;/code&gt; and &lt;a&#10; href="https://github.com/openai/codex/blob/67f0994b532b8740f14b5b738af7b7b2a915b1d6/codex-rs/state/src/sqlite.rs#L48-L53"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;others&lt;/a&gt;&#10;. A live copy without the &lt;code&gt;-wal&lt;/code&gt; and &lt;code&gt;-shm&lt;/code&gt; sidecars can be incomplete or corrupt. Stop the client and copy all three.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Treating the desktop database as the archive.&lt;/strong&gt; The Codex desktop app can rebuild its index from the rollout JSONL files, but metadata that lives only in the database is lost, back up the JSONL files.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Assuming &lt;code&gt;claude purge&lt;/code&gt; cleaned everything.&lt;/strong&gt; It doesn&amp;rsquo;t touch &lt;code&gt;shell-snapshots/&lt;/code&gt;, &lt;code&gt;backups/&lt;/code&gt; (older copies of &lt;code&gt;~/.claude.json&lt;/code&gt;) or files written by &lt;code&gt;/heapdump&lt;/code&gt;. According to Anthropic, a heap snapshot contains the full conversation and any credentials the process held.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&#10;&lt;h2 id="restore-into-a-staging-profile"&gt;Restore into a staging profile &lt;/h2&gt;&#10;&lt;ol&gt;&#10;&lt;li&gt;Reinstall the signed apps, plugins, and extensions from their sources.&lt;/li&gt;&#10;&lt;li&gt;Restore your config, then read every hook, MCP command, and executable path before the first launch.&lt;/li&gt;&#10;&lt;li&gt;Restore transcripts or memory only if you need them.&lt;/li&gt;&#10;&lt;li&gt;Log in again. Don&amp;rsquo;t restore &lt;code&gt;auth.json&lt;/code&gt;, &lt;code&gt;.credentials.json&lt;/code&gt;, cookies, or Keychain exports.&lt;/li&gt;&#10;&lt;li&gt;Resume one session and confirm that it uses the right account and repository.&lt;/li&gt;&#10;&lt;/ol&gt;&#10;&#10;&lt;h2 id="verify-the-backup-set"&gt;Verify the backup set &lt;/h2&gt;&#10;&lt;p&gt;To list what exists and how large it is, without printing any contents, run this on the machine you back up:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;for&lt;/span&gt; p in &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CLAUDE_CONFIG_DIR&lt;/span&gt;&lt;span class="k"&gt;:-&lt;/span&gt;&lt;span class="nv"&gt;$HOME&lt;/span&gt;&lt;span class="p"&gt;/.claude&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$HOME&lt;/span&gt;&lt;span class="s2"&gt;/.claude.json&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CODEX_HOME&lt;/span&gt;&lt;span class="k"&gt;:-&lt;/span&gt;&lt;span class="nv"&gt;$HOME&lt;/span&gt;&lt;span class="p"&gt;/.codex&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;XDG_CONFIG_HOME&lt;/span&gt;&lt;span class="k"&gt;:-&lt;/span&gt;&lt;span class="nv"&gt;$HOME&lt;/span&gt;&lt;span class="p"&gt;/.config&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/Claude&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$HOME&lt;/span&gt;&lt;span class="s2"&gt;/Library/Application Support/Claude&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$HOME&lt;/span&gt;&lt;span class="s2"&gt;/Library/Application Support/Codex&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$HOME&lt;/span&gt;&lt;span class="s2"&gt;/Library/Application Support/com.openai.chat&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$HOME&lt;/span&gt;&lt;span class="s2"&gt;/Library/Group Containers/2DC432GLL2.group.com.openai.chat&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;do&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="o"&gt;[&lt;/span&gt; -e &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$p&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; du -sh &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$p&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;done&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Then check that no login file ended up in the backup:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;find /path/to/mounted/backup &lt;span class="se"&gt;\(&lt;/span&gt; -name .credentials.json -o -name auth.json &lt;span class="se"&gt;\)&lt;/span&gt; -print&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;It should print nothing. If it prints anything, that backup is a credential store: encrypt it, shorten its retention, or rotate the logins.&lt;/p&gt;&#10;&#10;&lt;h2 id="if-malware-may-already-have-run"&gt;If malware may already have run &lt;/h2&gt;&#10;&lt;p&gt;Don&amp;rsquo;t treat a restore as the cleanup. From a clean device, revoke your Claude and ChatGPT sessions, then rotate API keys, MCP and connector credentials, and any secret that appeared in a transcript or in tool output. Any backup the malware could reach counts as exposed. Rebuild the user profile, restore only config you have reviewed, and log in again.&lt;/p&gt;&#10;&#10;&lt;h2 id="caveats"&gt;Caveats &lt;/h2&gt;&#10;&lt;p&gt;Paths and retention rules change between releases. The ChatGPT and desktop paths come from build inspection, not from a published schema, so check them against your installed versions before you write a backup job. Sources: the &lt;a&#10; href="https://code.claude.com/docs/en/claude-directory.md"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;&lt;code&gt;.claude&lt;/code&gt; directory reference&lt;/a&gt;&#10;, &lt;a&#10; href="https://code.claude.com/docs/en/authentication.md#credential-management"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;Claude Code credential management&lt;/a&gt;&#10;, &lt;a&#10; href="https://learn.chatgpt.com/docs/config-file/config-advanced.md#config-and-state-locations"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;Codex config and state locations&lt;/a&gt;&#10;, &lt;a&#10; href="https://learn.chatgpt.com/docs/auth.md#credential-storage"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;Codex credential storage&lt;/a&gt;&#10;, &lt;a&#10; href="https://modelcontextprotocol.io/quickstart/user"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;Claude Desktop MCP config paths&lt;/a&gt;&#10;, and &lt;a&#10; href="https://www.electronjs.org/docs/latest/api/app#appgetpathname"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;Electron&amp;rsquo;s &lt;code&gt;app.getPath()&lt;/code&gt;&lt;/a&gt;&#10;.&lt;/p&gt;&#10;&lt;p&gt;Back up the config you wrote, and leave out the files the app uses for login. Run the &lt;code&gt;find&lt;/code&gt; check above against your latest backup tonight.&lt;/p&gt;&#10;</description></item><item><title>'Downgrading is unsupported' after a config restore: the update guard was right</title><link>https://eghuro.cz/posts/update-guard-refusal-after-config-restore-was-correct/</link><pubDate>Fri, 16 Oct 2026 00:00:00 +0000</pubDate><guid>https://eghuro.cz/posts/update-guard-refusal-after-config-restore-was-correct/</guid><description>&lt;p&gt;After a configuration restore, an updater refused with:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;that release predates the gateway runtime these session stores now use. Downgrading is unsupported.&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The operator requested an update. The restored configuration selected an older code line than the databases required. Restoring the intended per-install channel corrected target selection; bypassing the guard would have put incompatible code over newer state.&lt;/p&gt;&#10;&#10;&lt;h2 id="a-newer-release-number-can-select-older-capabilities"&gt;A newer release number can select older capabilities &lt;/h2&gt;&#10;&lt;p&gt;The checkout followed a development branch. Its session databases already contained runtime-ledger history. The restored configuration backup lacked the install&amp;rsquo;s channel record, so release selection fell back to the stable channel.&lt;/p&gt;&#10;&lt;p&gt;The stable release had a higher patch version, but it came from a release branch without those runtime-ledger tables. Version ordering did not describe the relevant capability ordering.&lt;/p&gt;&#10;&lt;p&gt;The historical guard checked two things: whether the proposed target descended from the running commit, and whether a non-descendant target contained the runtime schema marker. It also read the session stores for ledger rows. Its refusal named the affected stores and stopped before moving the checkout.&lt;/p&gt;&#10;&lt;p&gt;The implementation explained the consequence: older code could not delete or prune sessions with that runtime history. This was a specific compatibility check, not proof that every forward commit or every future schema change would be safe.&lt;/p&gt;&#10;&#10;&lt;h2 id="repair-target-selection-before-changing-code"&gt;Repair target selection before changing code &lt;/h2&gt;&#10;&lt;p&gt;The diagnosis followed the configuration lookup:&lt;/p&gt;&#10;&lt;ol&gt;&#10;&lt;li&gt;Identify the actual install path and the channel record associated with it.&lt;/li&gt;&#10;&lt;li&gt;Compare that record with the backup that was restored. Check defaults when the record is absent.&lt;/li&gt;&#10;&lt;li&gt;Resolve the updater&amp;rsquo;s proposed target to a commit, rather than relying on its release label.&lt;/li&gt;&#10;&lt;li&gt;Compare the target&amp;rsquo;s ancestry and required state capabilities with the running checkout.&lt;/li&gt;&#10;&lt;li&gt;Reassert the intended channel through the updater&amp;rsquo;s supported configuration command, then run its check-only mode.&lt;/li&gt;&#10;&lt;/ol&gt;&#10;&lt;p&gt;The corrected check selected the development branch and reported available forward commits—that verified channel selection. The update itself was deliberately not run, so the record does not establish a completed runtime upgrade.&lt;/p&gt;&#10;&lt;p&gt;For a Git-based installation, this read-only check is useful once &lt;code&gt;TARGET&lt;/code&gt; is set to the exact resolved commit. Run it inside that installation&amp;rsquo;s checkout:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-sh" data-lang="sh"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;git merge-base --is-ancestor HEAD &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;TARGET&lt;/span&gt;&lt;span class="p"&gt;:?Set TARGET to the resolved update commit&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Exit status 0 means the target descends from the current checkout. Status 1 means it does not; other failures require investigation. A non-descendant can be a valid release-branch target, so this command alone is not a compatibility verdict. Check the particular schema capability named by the refusal.&lt;/p&gt;&#10;&#10;&lt;h2 id="a-config-only-restore-can-create-a-mixed-generation"&gt;A config-only restore can create a mixed generation &lt;/h2&gt;&#10;&lt;p&gt;The database kept its newer state while the restored configuration lost an additive setting. In this implementation, channel records were keyed by install path identity. Restoring an older backup, or moving an install without recreating its record, could therefore change how later updates were selected.&lt;/p&gt;&#10;&lt;p&gt;Deployment automation should assert the intended channel after install or restore, then read it back through the supported interface. Do not hard-code a path-derived identifier from another machine. Following a development branch is appropriate only when that is the deliberate operating policy; it is not a universal fix for update errors.&lt;/p&gt;&#10;&lt;p&gt;For an intentional rollback, follow the application&amp;rsquo;s documented procedure for compatible code and pre-upgrade data together. Preserve the current state first, and account for work absent from the older backup. Editing a schema-version value or removing ledger rows merely to silence the guard would discard the evidence it uses to protect the store.&lt;/p&gt;&#10;&lt;p&gt;Add an update-target check to your restore drill before allowing the restored installation to update itself.&lt;/p&gt;&#10;</description></item><item><title>Rootless Podman backup without app.ini, and a restore that fails with 'Cannot change ownership to uid 100999'</title><link>https://eghuro.cz/posts/rootless-podman-backup-missing-app-ini/</link><pubDate>Thu, 15 Oct 2026 17:23:00 +0200</pubDate><guid>https://eghuro.cz/posts/rootless-podman-backup-missing-app-ini/</guid><description>&lt;p&gt;Restoring a Gitea backup to a new host failed on every directory:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;tar: data/gitea: Cannot change ownership to uid 100999, gid 100999: Invalid argument&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;tar: data/gitea/queues: Cannot change ownership to uid 100999, gid 100999: Invalid argument&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;That one is easy to fix. The worse problem made no noise at all, and I found it just before: the nightly backup didn&amp;rsquo;t contain &lt;code&gt;app.ini&lt;/code&gt;. If you back up a rootless Podman volume with a plain &lt;code&gt;tar&lt;/code&gt; as the host user, yours may be missing files too.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-setup"&gt;The setup &lt;/h2&gt;&#10;&lt;p&gt;Gitea 1.27 and PostgreSQL ran in rootless Podman on a VPS, with Gitea&amp;rsquo;s data in a bind mount under &lt;code&gt;~/gitea/data&lt;/code&gt;. The nightly backup ran as the user who owns the containers:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;podman &lt;span class="nb"&gt;exec&lt;/span&gt; gitea_db_1 pg_dump -U gitea gitea &lt;span class="p"&gt;|&lt;/span&gt; bzip2 -z9 &amp;gt; ~/backup/gitea-db-&lt;span class="k"&gt;$(&lt;/span&gt;date +%y%m%d&lt;span class="k"&gt;)&lt;/span&gt;.sql.bz2&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;tar c -C ~/gitea/data gitea &lt;span class="p"&gt;|&lt;/span&gt; bzip2 -z9 &amp;gt; ~/backup/gitea-data-&lt;span class="k"&gt;$(&lt;/span&gt;date +%y%m%d&lt;span class="k"&gt;)&lt;/span&gt;.tar.bz2&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The target was Gitea in Docker, inside an unprivileged LXC container on Debian.&lt;/p&gt;&#10;&lt;p&gt;The archive looked fine. It unpacked, and it contained all the repositories.&lt;/p&gt;&#10;&#10;&lt;h2 id="compare-the-archive-with-the-live-tree"&gt;Compare the archive with the live tree &lt;/h2&gt;&#10;&lt;p&gt;Before restoring, I compared the archive&amp;rsquo;s file list with the live data. List the live side from inside the user namespace, so that nothing is hidden from you:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;ssh vps.example.com &lt;span class="s1"&gt;&amp;#39;cd ~/gitea/data &amp;amp;&amp;amp; podman unshare find gitea -type f&amp;#39;&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; sort &amp;gt; live.txt&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;tar tjf gitea-data.tar.bz2 &lt;span class="p"&gt;|&lt;/span&gt; grep -v &lt;span class="s1"&gt;&amp;#39;/$&amp;#39;&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; sort &amp;gt; backup.txt&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;comm -23 live.txt backup.txt &amp;gt; missing.txt&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;wc -l live.txt backup.txt missing.txt&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; 12359 live.txt&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; 12342 backup.txt&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; 17 missing.txt&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Among the 17 were the files Gitea can least do without:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;gitea/gitea/conf/app.ini&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;gitea/gitea/jwt/private.pem&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;gitea/git/.ssh/authorized_keys&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;gitea/gitea/avatars/... (3 files)&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;gitea/gitea/indexers/... (4 files)&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;gitea/gitea/sessions/... (2 files)&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;app.ini&lt;/code&gt; holds Gitea&amp;rsquo;s &lt;code&gt;SECRET_KEY&lt;/code&gt;, its internal tokens, and the mail settings. &lt;code&gt;private.pem&lt;/code&gt; signs its OAuth2 tokens. A restore from this backup would have started with fresh secrets: anything encrypted with the old &lt;code&gt;SECRET_KEY&lt;/code&gt;, such as two-factor secrets, would no longer decrypt, and the mailer would be unconfigured.&lt;/p&gt;&#10;&#10;&lt;h2 id="why-tar-skipped-them"&gt;Why tar skipped them &lt;/h2&gt;&#10;&lt;p&gt;Rootless Podman maps the container&amp;rsquo;s UIDs onto the subordinate range in &lt;code&gt;/etc/subuid&lt;/code&gt;. With the usual entry:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;deploy:100000:65536&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;container root is you, and container UID 1000 (Gitea&amp;rsquo;s &lt;code&gt;git&lt;/code&gt; user) is host UID 100000 + 999 = 100999. Files that Gitea creates with mode &lt;code&gt;0600&lt;/code&gt; belong to 100999, and you can&amp;rsquo;t read them from the host. The repository objects were &lt;code&gt;0644&lt;/code&gt; in &lt;code&gt;0755&lt;/code&gt; directories, so tar read those, and the archive looked complete.&lt;/p&gt;&#10;&lt;p&gt;tar does complain, on stderr, and exits with status 2. In &lt;code&gt;tar c ... | bzip2 &amp;gt; file&lt;/code&gt;, though, the pipeline&amp;rsquo;s exit status is bzip2&amp;rsquo;s, which is 0. Without &lt;code&gt;set -o pipefail&lt;/code&gt; and someone reading the job&amp;rsquo;s stderr, nothing tells you.&lt;/p&gt;&#10;&#10;&lt;h2 id="fix-the-backup"&gt;Fix the backup &lt;/h2&gt;&#10;&lt;p&gt;Run tar inside the user namespace, where you are root over every mapped UID:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;set&lt;/span&gt; -o pipefail&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;podman unshare tar c -C ~/gitea/data gitea &lt;span class="p"&gt;|&lt;/span&gt; bzip2 -z9 &amp;gt; ~/backup/gitea-data-&lt;span class="k"&gt;$(&lt;/span&gt;date +%y%m%d&lt;span class="k"&gt;)&lt;/span&gt;.tar.bz2&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The archive then records the IDs as the container sees them (1000), not the host&amp;rsquo;s 100999.&lt;/p&gt;&#10;&lt;p&gt;To rescue the files without touching the live host, fetch only the missing ones the same way, then merge them into the old archive with bsdtar&amp;rsquo;s &lt;code&gt;@archive&lt;/code&gt; syntax:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;ssh vps.example.com &lt;span class="s1"&gt;&amp;#39;cd ~/gitea/data &amp;amp;&amp;amp; podman unshare tar -c -T -&amp;#39;&lt;/span&gt; &amp;lt; missing.txt &amp;gt; missing.tar&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;bsdtar -czf gitea-complete.tar.gz @gitea-data.tar.bz2 @missing.tar&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&#10;&lt;h2 id="fix-the-restore"&gt;Fix the restore &lt;/h2&gt;&#10;&lt;p&gt;GNU tar run as root restores the owners recorded in the archive (&lt;code&gt;--same-owner&lt;/code&gt; is its default for root). Inside an unprivileged LXC container, root is itself a mapped user, and the container&amp;rsquo;s UIDs usually cover only 0 to 65535. UID 100999 doesn&amp;rsquo;t exist there, so &lt;code&gt;chown&lt;/code&gt; fails with &lt;code&gt;EINVAL&lt;/code&gt;, which tar reports as &lt;code&gt;Invalid argument&lt;/code&gt;.&lt;/p&gt;&#10;&lt;p&gt;Don&amp;rsquo;t restore owners from an archive made somewhere else. Unpack with &lt;code&gt;--no-same-owner&lt;/code&gt; and set the owners the new Gitea expects:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;tar -C /var/tmp/gitea-restore --no-same-owner -xzf gitea-complete.tar.gz&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;cp -a /var/tmp/gitea-restore/gitea/. /srv/gitea/data/&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;chown -R 1000:1000 /srv/gitea/data/git /srv/gitea/data/gitea&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;chown -R root:root /srv/gitea/data/ssh&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This also covers the merged archive, which carries 100999 for some files and 1000 for others.&lt;/p&gt;&#10;&lt;p&gt;After the restore, check against the live instance: the repository count, &lt;code&gt;git fsck&lt;/code&gt; on every repository, and the SSH host key fingerprint on the Git SSH port, so existing clones still trust the new host. All three matched.&lt;/p&gt;&#10;&#10;&lt;h2 id="takeaway"&gt;Takeaway &lt;/h2&gt;&#10;&lt;p&gt;Under rootless Podman, a host-side &lt;code&gt;tar&lt;/code&gt; leaves out the files the container keeps private, which are often its secrets, and the pipeline still exits 0. Comparing the archive&amp;rsquo;s listing with &lt;code&gt;podman unshare find&lt;/code&gt; takes a minute and shows the gap.&lt;/p&gt;&#10;&lt;p&gt;Run the &lt;code&gt;podman unshare find&lt;/code&gt; and &lt;code&gt;comm&lt;/code&gt; check above against your own latest backup and see what comes back.&lt;/p&gt;&#10;</description></item><item><title>APFS sparse bundle on SMB: rsync "Operation not supported (45)" and F_FULLFSYNC</title><link>https://eghuro.cz/posts/apfs-sparse-bundle-smb-fullfsync/</link><pubDate>Tue, 29 Sep 2026 12:55:00 +0200</pubDate><guid>https://eghuro.cz/posts/apfs-sparse-bundle-smb-fullfsync/</guid><description>&lt;p&gt;A backup job that rsyncs into encrypted disk images on a NAS started printing this for every file (paths elided):&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;rsync: [generator] failed to set times on …&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;rsync: [receiver] mkstemp …&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;rsync: [generator] delete_file: unlink … failed: Operation not supported (45)&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Then &lt;code&gt;hdiutil attach&lt;/code&gt; for the next image hung for over ten minutes, and the two images whose rsyncs had finished stayed mounted.&lt;/p&gt;&#10;&lt;p&gt;If you keep APFS sparse bundles on an SMB share and see error 45 inside them, the files aren&amp;rsquo;t the problem. The share can&amp;rsquo;t do &lt;code&gt;F_FULLFSYNC&lt;/code&gt;, and APFS can&amp;rsquo;t commit a single transaction without it.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-setup"&gt;The setup &lt;/h2&gt;&#10;&lt;p&gt;macOS 27.0.1 dropped AFP, so an old NAS that speaks only SMB 2.1 got mounted over SMB instead. Each backup source has its own AES-256 sparse bundle on the share, created like this:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;hdiutil create -encryption AES-256 -type SPARSEBUNDLE -size 50g &lt;span class="se"&gt;\&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -fs APFS -volname app -stdinpass /Volumes/backup/staging/app.sparsebundle&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The job attaches a bundle, rsyncs a server&amp;rsquo;s files into it, and detaches it.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-kernel-log"&gt;The kernel log &lt;/h2&gt;&#10;&lt;p&gt;rsync names the files, so the obvious suspects are permissions or the files themselves. The kernel log points at the image instead (timestamps shortened, process column removed):&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;/usr/bin/log show --last 90m --style compact &lt;span class="se"&gt;\&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; --predicate &lt;span class="s1"&gt;&amp;#39;process == &amp;#34;kernel&amp;#34; AND eventMessage CONTAINS &amp;#34;apfs&amp;#34;&amp;#39;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;12:43:53.531 disk7s1 mount-complete volume app, requested by: mount_apfs (pid 82535); parent: mount (pid 82534)&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;12:43:53.532 disk7s1 Host is remote&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;...&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;12:47:45.393 (apfs) tx_flush_thread:1678: disk7 tx_flush(8311) failed, 45, # 297&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;12:47:45.931 (apfs) tx_flush_thread:1678: disk7 tx_flush(8311) failed, 45, # 298&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;APFS noticed the image lives on a remote host. From 24 seconds after mounting, it failed to flush a transaction about twice a second, with error 45 (&lt;code&gt;ENOTSUP&lt;/code&gt;). The second mounted image did the same. Nothing written inside them was being committed.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-probe"&gt;The probe &lt;/h2&gt;&#10;&lt;p&gt;Error 45 from a flush suggested the sync call, not the write. This script tries plain &lt;code&gt;fsync&lt;/code&gt; and the two stronger syncs macOS offers. Run it from a directory on the share:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;fcntl&lt;/span&gt;&lt;span class="o"&gt;,&lt;/span&gt; &lt;span class="nn"&gt;os&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;_synctest&amp;#34;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;fd&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;O_RDWR&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;O_CREAT&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mo"&gt;0o600&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;write&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fd&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sa"&gt;b&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;x&amp;#34;&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;4096&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;op&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;fsync&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;F_FULLFSYNC&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;51&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;F_BARRIERFSYNC&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;85&lt;/span&gt;&lt;span class="p"&gt;)):&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;fsync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fd&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;op&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="n"&gt;fcntl&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;fcntl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fd&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;op&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nb"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;OK&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="ne"&gt;OSError&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nb"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;-&amp;gt;&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;close&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fd&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;unlink&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;fsync OK&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;F_FULLFSYNC -&amp;gt; [Errno 45] Operation not supported&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;F_BARRIERFSYNC -&amp;gt; [Errno 45] Operation not supported&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;APFS asks the disk image to flush; the disk image passes that on as &lt;code&gt;F_FULLFSYNC&lt;/code&gt; on its band files on the share, and the SMB client says no.&lt;/p&gt;&#10;&lt;p&gt;The NAS also had a share called &lt;code&gt;TimeMachineBackup&lt;/code&gt;, and Time Machine over SMB needs full sync, so it seemed worth a try. Same result. Its Time Machine backups dated from the AFP days.&lt;/p&gt;&#10;&#10;&lt;h2 id="what-works"&gt;What works &lt;/h2&gt;&#10;&lt;p&gt;Keep the image off the share. The bundles moved to a USB drive, and every healthy one attaches, writes, deletes, and detaches fine there. That&amp;rsquo;s where the job runs now.&lt;/p&gt;&#10;&lt;p&gt;If the image has to stay on the share, use HFS+ inside it. A test bundle on the same share, created with &lt;code&gt;-fs HFS+J&lt;/code&gt; instead of &lt;code&gt;-fs APFS&lt;/code&gt;, passed everything the job does: rsync with &lt;code&gt;--delete-during&lt;/code&gt;, rename, setting times, deletes, detach and reattach.&lt;/p&gt;&#10;&lt;p&gt;If the server runs Samba and you control it, &lt;code&gt;fruit:time machine = yes&lt;/code&gt; on the share makes vfs_fruit advertise the FULLSYNC volume capability to macOS clients. That should let APFS images commit, but it wasn&amp;rsquo;t tested here.&lt;/p&gt;&#10;&#10;&lt;h2 id="check-the-images-you-already-have"&gt;Check the images you already have &lt;/h2&gt;&#10;&lt;p&gt;The failed commits also left damage. When the bundles were copied to the USB drive, one of them wouldn&amp;rsquo;t mount:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;hdiutil: attach failed - no mountable file systems&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;A read-only check of its volume (&lt;code&gt;fsck_apfs -n&lt;/code&gt;) said why:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;warning: (oid 0x74e6) om: btn: invalid o_cksum (0xdfbd1ccee26c9006)&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;** The volume /dev/rdisk8s1 with UUID … was found to be corrupt and cannot be repaired.&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Its band files had last been modified during the failed SMB run, and the original on the NAS was identical, so the copy wasn&amp;rsquo;t to blame. Since it was only a staging mirror, it got recreated empty, and the next run filled it again. After any such episode, attach each image and run &lt;code&gt;diskutil verifyVolume&lt;/code&gt; on it.&lt;/p&gt;&#10;&#10;&lt;h2 id="why-restic-didnt-mind"&gt;Why restic didn&amp;rsquo;t mind &lt;/h2&gt;&#10;&lt;p&gt;Restic repositories on the same share were unaffected. Restic is written in Go, and Go&amp;rsquo;s &lt;code&gt;Fsync&lt;/code&gt; on macOS tries &lt;code&gt;F_FULLFSYNC&lt;/code&gt; and falls back to plain &lt;code&gt;fsync&lt;/code&gt; on &lt;code&gt;ENOTSUP&lt;/code&gt;. The &lt;a&#10; href="https://github.com/golang/go/blob/master/src/internal/poll/fd_fsync_darwin.go"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;comment in the Go source&lt;/a&gt;&#10; names this exact case: &amp;ldquo;There are scenarios such as SMB mounts where fcntl will fail with ENOTSUP.&amp;rdquo; APFS inside a disk image has no such fallback: when the full flush fails, it stops committing.&lt;/p&gt;&#10;&#10;&lt;h2 id="takeaway"&gt;Takeaway &lt;/h2&gt;&#10;&lt;p&gt;An APFS disk image on an SMB share works only if the share supports &lt;code&gt;F_FULLFSYNC&lt;/code&gt;. Without it, every write inside the image fails with error 45, and the failed commits can corrupt the image.&lt;/p&gt;&#10;&lt;p&gt;If you&amp;rsquo;re moving sparse bundles from AFP to SMB, run the probe above on the new share before you mount anything on it.&lt;/p&gt;&#10;</description></item><item><title>macOS SMB: OSError: [Errno 9] Bad file descriptor after a silent reconnect</title><link>https://eghuro.cz/posts/macos-smb-ebadf-after-reconnect/</link><pubDate>Tue, 29 Sep 2026 12:15:00 +0200</pubDate><guid>https://eghuro.cz/posts/macos-smb-ebadf-after-reconnect/</guid><description>&lt;p&gt;A Python script that had encrypted backup archives on a NAS for months started failing on every run:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Traceback (most recent call last):&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; File &amp;#34;/Volumes/backup/./encrypt-one.py&amp;#34;, line 74, in &amp;lt;module&amp;gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; main()&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; File &amp;#34;/Volumes/backup/./encrypt-one.py&amp;#34;, line 38, in main&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; with open(f, &amp;#34;rb&amp;#34;) as src:&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;OSError: [Errno 9] Bad file descriptor&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Then the bash script that called it stopped too. Not intermittently: start the job and something fails sooner or later.&lt;/p&gt;&#10;&lt;p&gt;If you run long jobs from an SMB share on macOS and get &lt;code&gt;EBADF&lt;/code&gt; on files that are plainly there, the cause is probably a session reconnect you never saw, and the kernel log is the only place that records it.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-setup"&gt;The setup &lt;/h2&gt;&#10;&lt;p&gt;macOS 27 dropped AFP, so an old NAS that speaks only SMB 2.1 got mounted over SMB instead. A bash script on the share runs &lt;code&gt;encrypt-one.py&lt;/code&gt; on a few archives in parallel with &lt;code&gt;xargs -P&lt;/code&gt;. Each run streams a 600 MB archive from the share through &lt;code&gt;openssl enc&lt;/code&gt; and writes the result back to the same share.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-dead-end"&gt;The dead end &lt;/h2&gt;&#10;&lt;p&gt;The traceback points at &lt;code&gt;open()&lt;/code&gt;, so the first move was to open and read that archive by hand. It worked, and so did a read under &lt;code&gt;xargs -P 3&lt;/code&gt;. A file that opens fine and a job that fails anyway tells you the file isn&amp;rsquo;t the problem. Something happens to the handle while it&amp;rsquo;s open.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-kernel-log"&gt;The kernel log &lt;/h2&gt;&#10;&lt;p&gt;&lt;code&gt;smbutil&lt;/code&gt; gave the first hint:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;$ smbutil statshares -m /Volumes/backup&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;...&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;SMB_VERSION SMB_2.1&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;SESSION_RECONNECT_TIME 2026-09-29 12:01:17&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;SESSION_RECONNECT_COUNT 4&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Four reconnects; the last one was inside the failed run. The kernel log explains what each one does (timestamps shortened, process column removed):&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;/usr/bin/log show --last 1h --style compact &lt;span class="se"&gt;\&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; --predicate &lt;span class="s1"&gt;&amp;#39;process == &amp;#34;kernel&amp;#34; AND eventMessage CONTAINS[c] &amp;#34;smb&amp;#34;&amp;#39;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:15.502 (smbfs) smb_iod_sendall: id 0 ts &amp;gt; iod_lastrecv &amp;amp;&amp;amp; ts &amp;gt; sr_timesent.&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:15.502 (smbfs) smb_iod_reconnect: id 0 Starting reconnect with nas.local&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:15.502 (smbfs) smb2_smb_parse_change_notify: smb_rq_reply failed 60&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:16.523 (smbfs) smb_iod_reconnect: id 0 The negotiate succeeded to nas.local&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:17.065 (smbfs) smb2fs_reconnect_dur_handle: Missing durable handle on &amp;lt;archive.tar.bz2.enc.tmp&amp;gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:17.173 (smbfs) smb2fs_reconnect_dur_handle: Missing durable handle on &amp;lt;backup.sh&amp;gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:17.186 (smbfs) smb2fs_reconnect_dur_handle: Missing durable handle on &amp;lt;archive.tar.bz2&amp;gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:17.205 (smbfs) smb_iod_reconnect: id 0: Reconnect completed successfully.&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:17.212 (smbfs) smb2_smb_read_write_async: smb_rq_reply failed 9&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:17.212 (smbfs) smbfs_do_strategy: archive.tar.bz2: READ failed with an error of 9&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;11:58:17.283 (smbfs) smbfs_do_strategy: archive.tar.bz2.enc.tmp: WRITE failed with an error of 9&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The NAS stopped answering under sustained I/O (error 60 is &lt;code&gt;ETIMEDOUT&lt;/code&gt;). macOS dropped the session and reconnected, and that part worked. Then it tried to reattach each open file; the server had nothing to reattach them to, and every read and write on those files failed with error 9, &lt;code&gt;EBADF&lt;/code&gt;.&lt;/p&gt;&#10;&lt;p&gt;Look at the middle line: &lt;code&gt;backup.sh&lt;/code&gt; is on the list. Bash reads a script from disk as it runs, so the script file is open the whole time and goes bad with everything else. That&amp;rsquo;s why the shell stopped as well.&lt;/p&gt;&#10;&#10;&lt;h2 id="why"&gt;Why &lt;/h2&gt;&#10;&lt;p&gt;A durable handle is how an SMB 2.x client keeps an open file across a dropped connection: the server holds the open for a while, and the client reclaims it after reconnecting. This server granted none. So &amp;ldquo;Reconnect completed successfully&amp;rdquo; is true for the session and useless for every file that was open at the time. Nothing reaches the application except &lt;code&gt;EBADF&lt;/code&gt; on the next read or write.&lt;/p&gt;&#10;&#10;&lt;h2 id="what-fixed-it"&gt;What fixed it &lt;/h2&gt;&#10;&lt;p&gt;Stop running long I/O jobs from the share. The pipeline moved to a USB drive, and only finished files go to the NAS. That ended the failures.&lt;/p&gt;&#10;&lt;p&gt;If you can&amp;rsquo;t move off the share, two workarounds got the job past the failing step:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&#10;&lt;p&gt;Make bash read the whole script before it starts, so it never goes back to the share for the next line:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="cp"&gt;#!/usr/bin/env bash&#10;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="o"&gt;{&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;set&lt;/span&gt; -euo pipefail&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# ... the whole script ...&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;exit&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="o"&gt;}&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Bash parses the entire &lt;code&gt;{ }&lt;/code&gt; block before running any of it, and the &lt;code&gt;exit&lt;/code&gt; stops it from reading past the closing brace.&lt;/p&gt;&#10;&lt;/li&gt;&#10;&lt;li&gt;&#10;&lt;p&gt;Make the I/O survive a reconnect: process the file in segments (64 MiB worked), open, write and &lt;code&gt;fsync&lt;/code&gt; each segment separately, and on &lt;code&gt;EBADF&lt;/code&gt;, &lt;code&gt;EIO&lt;/code&gt; or &lt;code&gt;EAGAIN&lt;/code&gt; reopen and retry only that segment. With AES-CBC you can chain the segments by using the last ciphertext block as the next IV, and the output is byte-identical to one &lt;code&gt;openssl enc&lt;/code&gt; pass. In the test run on the share, no reconnect happened at all, probably because the per-segment &lt;code&gt;fsync&lt;/code&gt; slowed the writes enough for the NAS to keep up.&lt;/p&gt;&#10;&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;If the server is a Samba box you control, check whether it grants durable handles: the &lt;code&gt;durable handles&lt;/code&gt; option only takes effect with &lt;code&gt;kernel oplocks = no&lt;/code&gt;, &lt;code&gt;kernel share modes = no&lt;/code&gt; and &lt;code&gt;posix locking = no&lt;/code&gt;. Turning off SMB signing on the client (&lt;code&gt;signing_required=no&lt;/code&gt; in &lt;code&gt;/etc/nsmb.conf&lt;/code&gt;) might also take enough load off a weak NAS CPU to prevent the stalls. Neither was tried here.&lt;/p&gt;&#10;&#10;&lt;h2 id="takeaway"&gt;Takeaway &lt;/h2&gt;&#10;&lt;p&gt;On macOS, an SMB reconnect to a server without durable handles invalidates every open file on the share, including a running shell script, and the only trace is &lt;code&gt;Missing durable handle&lt;/code&gt; in the kernel log.&lt;/p&gt;&#10;&lt;p&gt;If you&amp;rsquo;ve seen &lt;code&gt;Errno 9&lt;/code&gt; on an SMB mount, run &lt;code&gt;smbutil statshares -a&lt;/code&gt; and look at &lt;code&gt;SESSION_RECONNECT_COUNT&lt;/code&gt; before you debug your code.&lt;/p&gt;&#10;</description></item><item><title>Uninstalling one Odoo module can remove its dependents and their data</title><link>https://eghuro.cz/posts/uninstalling-one-module-can-cascade/</link><pubDate>Sun, 30 Aug 2026 00:00:00 +0000</pubDate><guid>https://eghuro.cz/posts/uninstalling-one-module-can-cascade/</guid><description>&lt;p&gt;An Odoo module replacement passed a rehearsal&amp;rsquo;s record-count and accounting-total checks, yet the uninstall also removed a dependent addon that owned important business data. The application still loaded. The acceptance checks had missed the field that mattered.&lt;/p&gt;&#10;&lt;p&gt;Before replacing a module, inspect its reverse dependencies and data ownership. Rehearse the corrected transition on a restored copy, then compare the affected values by record identity before approving the live change.&lt;/p&gt;&#10;&#10;&lt;h2 id="uninstall-computes-a-reverse-dependency-set"&gt;Uninstall computes a reverse-dependency set &lt;/h2&gt;&#10;&lt;p&gt;In &lt;a&#10; href="https://github.com/odoo/odoo/blob/18.0/odoo/addons/base/models/ir_module.py"&#10; &#10; target="_blank" rel="noopener"&#10; &#10; &#10;&gt;Odoo 18&amp;rsquo;s module implementation&lt;/a&gt;&#10;, &lt;code&gt;downstream_dependencies()&lt;/code&gt; recursively finds modules that depend on the selected modules. &lt;code&gt;button_uninstall()&lt;/code&gt; marks both the selection and those dependents as &lt;code&gt;to remove&lt;/code&gt;.&lt;/p&gt;&#10;&lt;p&gt;The same source describes &lt;code&gt;module_uninstall()&lt;/code&gt; as removing database structures created by the module, including tables and columns. Actual deletion depends on the affected records and metadata; do not assume every business record disappears, or that surviving rows retain every field.&lt;/p&gt;&#10;&lt;p&gt;Generalize the replacement to a full addon, a smaller replacement, and an extension that depends on the full addon. Installing the smaller addon does not automatically redirect the extension&amp;rsquo;s dependency. Uninstalling the full addon still includes that extension in the removal set.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-first-rehearsal-had-the-wrong-acceptance-criteria"&gt;The first rehearsal had the wrong acceptance criteria &lt;/h2&gt;&#10;&lt;p&gt;The historical transition checked surviving records, aggregate totals, and registry health. Those checks passed while the dependent addon was also uninstalled. The user caught the missing preservation requirement and required a restore.&lt;/p&gt;&#10;&lt;p&gt;The subsequent investigation found a second problem: changing the extension&amp;rsquo;s manifest alone would be insufficient. The smaller replacement did not yet implement a field the extension required. That field&amp;rsquo;s behavior needed to be ported before changing the dependency.&lt;/p&gt;&#10;&lt;p&gt;The retained record includes the corrective restore and a later clone&amp;rsquo;s uninstall/registry-reload output. It ends with the corrected rehearsal and final verification still open. It does not prove a successful final transition or a completed before/after digest comparison.&lt;/p&gt;&#10;&#10;&lt;h2 id="rehearse-the-field-handoff-explicitly"&gt;Rehearse the field handoff explicitly &lt;/h2&gt;&#10;&lt;p&gt;A safer transition procedure is:&lt;/p&gt;&#10;&lt;ol&gt;&#10;&lt;li&gt;Quiesce writes and capture a consistent database and filestore backup. Verify checksums and prove that the pair restores.&lt;/li&gt;&#10;&lt;li&gt;Record installed module states, reverse dependencies, and the fields and records that must survive. Keep the addon source revisions with the backup.&lt;/li&gt;&#10;&lt;li&gt;Implement the required field behavior in the replacement, including compute and reversal behavior where applicable. Version-control the dependent addon too.&lt;/li&gt;&#10;&lt;li&gt;On an isolated restore, install the replacement and upgrade the dependent addon so the database&amp;rsquo;s module metadata reflects the new dependency. Inspect the removal set again before uninstalling the old addon.&lt;/li&gt;&#10;&lt;li&gt;Complete the uninstall, reload the registry, and compare the required values and behavior. Investigate field metadata and ownership during the handoff; identical field names alone do not establish safe transfer.&lt;/li&gt;&#10;&lt;/ol&gt;&#10;&lt;p&gt;Disable scheduled jobs and outbound integrations on the clone. A rehearsal should not send mail, contact payment services, or write to production storage.&lt;/p&gt;&#10;&#10;&lt;h2 id="compare-identity-and-value-not-only-counts"&gt;Compare identity and value, not only counts &lt;/h2&gt;&#10;&lt;p&gt;For a small affected dataset, compare canonical exports directly. For larger exports, a digest is convenient if the query scope, ordering, and serialization remain identical.&lt;/p&gt;&#10;&lt;p&gt;This synthetic Python check demonstrates why counts and non-null counts are insufficient. It models two records whose field values were exchanged; it is not an Odoo migration test:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-python" data-lang="python"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;hashlib&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;json&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;digest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;sorted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;separators&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;,&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;:&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;ensure_ascii&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hashlib&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sha256&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;utf-8&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;hexdigest&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;before&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;2026-01-10&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;2026-01-20&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;)]&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;after&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;2026-01-20&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;2026-01-10&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;)]&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;before&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;after&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nb"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;before&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="nb"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;after&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;digest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;before&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;digest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;list&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;reversed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;before&lt;/span&gt;&lt;span class="p"&gt;)))&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;assert&lt;/span&gt; &lt;span class="n"&gt;digest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;before&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;digest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;after&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;counts match; identity/value digest detects the change&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;For a real comparison, export the stable record ID alongside each affected field, preserve nulls, and normalize dates and other types explicitly. A missing column must fail the check. Also test compute behavior and important workflows: a value digest cannot prove those work.&lt;/p&gt;&#10;&lt;p&gt;Before your next uninstall, list which dependent addons and field values must remain, and make their survival a rehearsal acceptance criterion.&lt;/p&gt;&#10;</description></item></channel></rss>