<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Sandbox on Lessons learned</title><link>https://eghuro.cz/tags/sandbox/</link><description>Recent content in Sandbox on Lessons learned</description><generator>Hugo</generator><language>en-us</language><copyright>© Alexandr Mansurov · Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)</copyright><lastBuildDate>Sat, 26 Sep 2026 16:30:00 +0200</lastBuildDate><atom:link href="https://eghuro.cz/tags/sandbox/index.xml" rel="self" type="application/rss+xml"/><item><title>FSKit after mount -F: "ls: fts_read: Operation not permitted"</title><link>https://eghuro.cz/posts/fskit-mount-fts-read-operation-not-permitted/</link><pubDate>Sat, 26 Sep 2026 16:30:00 +0200</pubDate><guid>https://eghuro.cz/posts/fskit-mount-fts-read-operation-not-permitted/</guid><description>&lt;p&gt;My new FSKit filesystem mounted without complaint, but nothing inside it could be read:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;$ ls -la ~/mnt/storage-test&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;ls: fts_read: Operation not permitted&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;total 0&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;$ cat ~/mnt/storage-test/sub/a.txt&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;cat: /Users/me/mnt/storage-test/sub/a.txt: Operation not permitted&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;stat&lt;/code&gt; on the mount point worked. If your FSKit extension does the same after a &lt;code&gt;mount -F&lt;/code&gt; from the command line, check its sandbox before your code: the extension can see the source folder&amp;rsquo;s metadata but may not read its contents.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-setup"&gt;The setup &lt;/h2&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;macOS 26, a file system extension built on FSKit&amp;rsquo;s &lt;code&gt;FSUnaryFileSystem&lt;/code&gt; that exposes a local folder through a mount point&lt;/li&gt;&#10;&lt;li&gt;A small host app, &lt;code&gt;StorageHost.app&lt;/code&gt;, in &lt;code&gt;/Applications&lt;/code&gt;; its executable runs &lt;code&gt;mount -F -t storagefs &amp;lt;source&amp;gt; &amp;lt;mount point&amp;gt;&lt;/code&gt;&lt;/li&gt;&#10;&lt;li&gt;The extension &lt;code&gt;StorageFS.appex&lt;/code&gt; with the FSKit entitlement and the app sandbox&lt;/li&gt;&#10;&lt;li&gt;&lt;code&gt;Info.plist&lt;/code&gt; with &lt;code&gt;FSSupportsPathURLs&lt;/code&gt; set to true and &lt;code&gt;FSRequiresSecurityScopedPathURLResources&lt;/code&gt; set to false&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&#10;&lt;h2 id="what-went-wrong"&gt;What went wrong &lt;/h2&gt;&#10;&lt;p&gt;The first mount was over &lt;code&gt;~/Downloads&lt;/code&gt;, so the obvious suspect was privacy protection (TCC) for that folder. A fresh &lt;code&gt;~/storage-test&lt;/code&gt; with one file in it failed the same way, which ruled TCC out and pushed the search elsewhere.&lt;/p&gt;&#10;&lt;p&gt;My coding agent&amp;rsquo;s first log queries came back empty, and it told me there were no log entries. That was wrong. In zsh, &lt;code&gt;log&lt;/code&gt; is a builtin, so &lt;code&gt;log show ... 2&amp;gt;/dev/null&lt;/code&gt; runs the builtin, sends its &lt;code&gt;too many arguments&lt;/code&gt; error to &lt;code&gt;/dev/null&lt;/code&gt;, and prints nothing. Call &lt;code&gt;/usr/bin/log&lt;/code&gt; by its full path. With that, the kernel had the answer:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;kernel (Sandbox) 3 duplicate reports for Sandbox: StorageFS(83505) deny(1) file-read-data /Users/me/storage-test&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The extension&amp;rsquo;s own log showed the failures from FSKit&amp;rsquo;s side: &lt;code&gt;readDirectory:...reply:error:1&lt;/code&gt; and &lt;code&gt;readFrom:...reply:error:1&lt;/code&gt;. Error 1 is &lt;code&gt;EPERM&lt;/code&gt;, which &lt;code&gt;ls&lt;/code&gt; turns into &lt;code&gt;fts_read: Operation not permitted&lt;/code&gt;.&lt;/p&gt;&#10;&#10;&lt;h2 id="fix"&gt;Fix &lt;/h2&gt;&#10;&lt;p&gt;&lt;code&gt;mount -F&lt;/code&gt; gives the extension a plain path, not a security-scoped URL, so a sandboxed process can&amp;rsquo;t read an arbitrary path. For development, grant temporary sandbox exceptions in the extension&amp;rsquo;s &lt;code&gt;.entitlements&lt;/code&gt; file:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-xml" data-lang="xml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;key&amp;gt;&lt;/span&gt;com.apple.security.temporary-exception.files.home-relative-path.read-write&lt;span class="nt"&gt;&amp;lt;/key&amp;gt;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;array&amp;gt;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;&lt;span class="nt"&gt;&amp;lt;string&amp;gt;&lt;/span&gt;/&lt;span class="nt"&gt;&amp;lt;/string&amp;gt;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;/array&amp;gt;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;key&amp;gt;&lt;/span&gt;com.apple.security.temporary-exception.files.absolute-path.read-write&lt;span class="nt"&gt;&amp;lt;/key&amp;gt;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;array&amp;gt;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;&lt;span class="nt"&gt;&amp;lt;string&amp;gt;&lt;/span&gt;/Volumes/&lt;span class="nt"&gt;&amp;lt;/string&amp;gt;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;&amp;lt;/array&amp;gt;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This covers your home folder and external drives. It doesn&amp;rsquo;t override privacy protection, so &lt;code&gt;~/Downloads&lt;/code&gt;, &lt;code&gt;~/Documents&lt;/code&gt;, and &lt;code&gt;~/Desktop&lt;/code&gt; may still be refused; test with a folder elsewhere. Temporary exceptions are for development on your own machine. A shipping extension should use security-scoped resources instead. &lt;code&gt;FSRequiresSecurityScopedPathURLResources&lt;/code&gt; in &lt;code&gt;Info.plist&lt;/code&gt; requests that; I haven&amp;rsquo;t managed a clean test yet (see below).&lt;/p&gt;&#10;&lt;p&gt;The same logs flagged two more problems that I fixed along the way:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;-[FSModuleVolume(Project) getMaxFileSizeInBits]: Volume implements neither maxFileSizeInBits nor maxFileSize. One of them must be implemented.&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;-[FSVolumeConnector getStandardItemAttributesForItem:replyHandler:]_block_invoke: Reported attributes are incomplete&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The first needs &lt;code&gt;maximumFileSizeInBits&lt;/code&gt; on the volume. The second went away once the item attributes set &lt;code&gt;flags&lt;/code&gt;, which cleared the log entry.&lt;/p&gt;&#10;&#10;&lt;h2 id="reinstalling-switches-the-module-off"&gt;Reinstalling switches the module off &lt;/h2&gt;&#10;&lt;p&gt;To test each change, I replaced the app: &lt;code&gt;rm -rf /Applications/StorageHost.app&lt;/code&gt;, copied the new build, and launched it once. After that, &lt;code&gt;mount&lt;/code&gt; failed outright:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Module com.example.storage.fs is disabled!&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;mount: Unable to invoke task&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;pluginkit&lt;/code&gt; disagreed. &lt;code&gt;pluginkit -e use -i com.example.storage.fs&lt;/code&gt; succeeded, and &lt;code&gt;pluginkit -m&lt;/code&gt; listed the extension with a &lt;code&gt;+&lt;/code&gt;, which means enabled. FSKit still refused. It keeps its own switch in System Settings → General → Login Items &amp;amp; Extensions → File System Extensions, and I didn&amp;rsquo;t find a command-line way to flip it. After I turned StorageFS back on there, the next mount listed and read files, and appends, renames, truncation, and an 8 MB write-and-readback all worked.&lt;/p&gt;&#10;&lt;p&gt;The disabled module also spoiled my one test of security-scoped resources. With &lt;code&gt;FSRequiresSecurityScopedPathURLResources&lt;/code&gt; set to true, &lt;code&gt;mount&lt;/code&gt; failed with &lt;code&gt;Unable to invoke task&lt;/code&gt;, but the reinstall had just switched the module off, so the failure says nothing about the setting.&lt;/p&gt;&#10;&#10;&lt;h2 id="why"&gt;Why &lt;/h2&gt;&#10;&lt;p&gt;An FSKit module is an app extension, and FSKit requires it to run in the app sandbox. When an app hands a folder to the extension, sandbox access can travel with it as a security-scoped URL. &lt;code&gt;mount -F&lt;/code&gt; on the command line passes only a string. The sandbox lets the extension look up metadata for that path, which is why the mount and &lt;code&gt;stat&lt;/code&gt; worked, and denies &lt;code&gt;file-read-data&lt;/code&gt;, which every directory listing and file read needs.&lt;/p&gt;&#10;&#10;&lt;h2 id="takeaway"&gt;Takeaway &lt;/h2&gt;&#10;&lt;p&gt;If an FSKit mount works and every read returns &lt;code&gt;Operation not permitted&lt;/code&gt;, search the kernel log for &lt;code&gt;deny(1) file-read-data&lt;/code&gt; with &lt;code&gt;/usr/bin/log&lt;/code&gt;, then give the extension sandbox access to the source path. After every reinstall, check the File System Extensions switch before you trust a test result.&lt;/p&gt;&#10;&lt;p&gt;If you&amp;rsquo;ve got &lt;code&gt;FSRequiresSecurityScopedPathURLResources&lt;/code&gt; working with a command-line &lt;code&gt;mount -F&lt;/code&gt;, I&amp;rsquo;d like to hear how.&lt;/p&gt;&#10;</description></item><item><title>Sandbox-blocked tests called "environmental", then committed</title><link>https://eghuro.cz/posts/sandbox-blocked-tests-called-environmental/</link><pubDate>Mon, 13 Jul 2026 15:38:00 +0200</pubDate><guid>https://eghuro.cz/posts/sandbox-blocked-tests-called-environmental/</guid><description>&lt;p&gt;My coding agent finished a feature, ran the full test suite, and reported:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Full suite: 135 passed, 97 skipped; 22 DB suites failed due to no container runtime, plus sandboxed stub bind. Commit only handoff implementation files.&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Then it committed. I ran &lt;code&gt;pnpm test&lt;/code&gt; myself and got three real failures. The production build it had also skipped was broken too, and nobody found out for two months.&lt;/p&gt;&#10;&lt;p&gt;If your agent&amp;rsquo;s test run shows &lt;code&gt;Error: Could not find a working container runtime strategy&lt;/code&gt; or &lt;code&gt;Error: listen EPERM: operation not permitted 0.0.0.0:8080&lt;/code&gt;, those tests never ran, and a summary that counts them as &amp;ldquo;blocked&amp;rdquo; tells you nothing about the code.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-setup"&gt;The setup &lt;/h2&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;Codex CLI on macOS, with its default sandbox&lt;/li&gt;&#10;&lt;li&gt;A pnpm monorepo: a Next.js 15.5.19 app plus a few packages&lt;/li&gt;&#10;&lt;li&gt;Vitest 2.1.9, with Testcontainers 10.28.0 starting Postgres for the database tests and a local stub of an external service listening on port 8080&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;The agent split the feature into five slices and gave each to a parallel subagent. Each subagent ran its own focused tests, and the orchestrator ran the full suite and the build at the end.&lt;/p&gt;&#10;&#10;&lt;h2 id="what-went-wrong"&gt;What went wrong &lt;/h2&gt;&#10;&lt;p&gt;The sandbox blocks more than writes outside the project. Here it blocked three things the checks needed:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;Testcontainers couldn&amp;rsquo;t reach Docker: &lt;code&gt;Could not find a working container runtime strategy&lt;/code&gt;.&lt;/li&gt;&#10;&lt;li&gt;The service stub couldn&amp;rsquo;t bind its port: &lt;code&gt;listen EPERM: operation not permitted 0.0.0.0:8080&lt;/code&gt;.&lt;/li&gt;&#10;&lt;li&gt;&lt;code&gt;next build&lt;/code&gt; couldn&amp;rsquo;t write its compiler cache: &lt;code&gt;EPERM&lt;/code&gt; writing &lt;code&gt;~/Library/Caches/next-swc&lt;/code&gt;.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;Every database suite failed in &lt;code&gt;beforeAll&lt;/code&gt;, and Vitest reported the tests inside them as skipped. The agent&amp;rsquo;s final report was accurate:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;- Typecheck pass&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;- Focused tests pass&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;- Full: 135 passed; DB suites blocked by no Docker&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;- Build blocked by Next SWC cache permissions&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Every line of that is true. But the suite has 232 tests, and 135 + 97 = 232: over 40 percent of it never ran, including every test that touched the database, where this feature did most of its work.&lt;/p&gt;&#10;&lt;p&gt;On the host, the same suite printed:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Test Files 2 failed | 63 passed (65)&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; Tests 3 failed | 229 passed (232)&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;One new test forgot to start the service stub, so its client found no endpoint configured. Two older tests broke because the feature added an ownership check and their fixtures sent requests without a logged-in user (&lt;code&gt;Error: authentication required&lt;/code&gt;). The agent fixed all three in two minutes once it saw them.&lt;/p&gt;&#10;&lt;p&gt;The build was worse. The commit re-exported a server action from another &lt;code&gt;&amp;quot;use server&amp;quot;&lt;/code&gt; file:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-ts" data-lang="ts"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s2"&gt;&amp;#34;use server&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="kr"&gt;export&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;finalizeArtifact&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="kr"&gt;from&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;../../reports/actions.ts&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;tsc&lt;/code&gt; accepts this, and so do the tests. Only &lt;code&gt;next build&lt;/code&gt; rejects it:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Failed to compile.&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;./app/reports/[id]/actions.ts&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Error: x Only async functions are allowed to be exported in a &amp;#34;use server&amp;#34; file.&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Two months later, a different session ran a production build, hit this error, and deleted the line.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-same-sandbox-the-same-day"&gt;The same sandbox, the same day &lt;/h2&gt;&#10;&lt;p&gt;Half an hour before the bad commit, another session in the same repository hit the same two errors. At first it called them environmental too. Then it decided:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Full suite blocked by sandbox: Testcontainers cannot see Docker; stub bind gets `EPERM :8080`. Since Docker is reportedly running, retry full suite with host access.&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Two minutes later it reported &amp;ldquo;Host rerun green: 60 files, 222 tests&amp;rdquo;, and only then did it commit. The agent that fixed my three failures did the same: it reran the focused tests with host approval before committing the fix. Both sessions knew how to leave the sandbox; nothing required them to before a commit.&lt;/p&gt;&#10;&#10;&lt;h2 id="fix"&gt;Fix &lt;/h2&gt;&#10;&lt;p&gt;Treat &amp;ldquo;blocked&amp;rdquo; as &amp;ldquo;not run&amp;rdquo;, and make &amp;ldquo;not run&amp;rdquo; stop the commit.&lt;/p&gt;&#10;&lt;p&gt;In Codex, a command runs outside the sandbox when the agent requests &lt;code&gt;sandbox_permissions: &amp;quot;require_escalated&amp;quot;&lt;/code&gt; and you approve it. Put the rule in the instructions file the agent reads every session (&lt;code&gt;AGENTS.md&lt;/code&gt;, or &lt;code&gt;CLAUDE.md&lt;/code&gt; for Claude Code):&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-markdown" data-lang="markdown"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;A check that the sandbox blocked has not passed. Before committing, rerun every blocked test suite and build with host access (&lt;span class="sb"&gt;`sandbox_permissions: &amp;#34;require_escalated&amp;#34;`&lt;/span&gt;).&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;If that isn&amp;#39;t possible, don&amp;#39;t commit: report what didn&amp;#39;t run.&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Then read the agent&amp;rsquo;s numbers yourself. If passed plus skipped equals the total, and you never skip tests on purpose, every skipped test is one that didn&amp;rsquo;t run. If you have CI, have it run the full suite and the build before a merge.&lt;/p&gt;&#10;&#10;&lt;h2 id="why-it-happens"&gt;Why it happens &lt;/h2&gt;&#10;&lt;p&gt;The agent&amp;rsquo;s memory for this project already had a rule: &amp;ldquo;separate passed static or focused checks from runtime suites blocked by the environment, and reserve the final green claim for integrated verification.&amp;rdquo; The agent followed that rule to the letter: it never claimed green and listed exactly what had been blocked. Then it committed, because nothing in its instructions tied the commit to the green claim.&lt;/p&gt;&#10;&lt;p&gt;&amp;ldquo;Blocked by the environment&amp;rdquo; reads like an infrastructure problem that someone else will handle, so it ends up as a footnote in the report while the commit goes ahead. The rule made the report honest. It didn&amp;rsquo;t make anyone run the tests.&lt;/p&gt;&#10;&#10;&lt;h2 id="takeaway"&gt;Takeaway &lt;/h2&gt;&#10;&lt;p&gt;When a sandbox blocks a test or a build, nothing was verified. An agent that reports this honestly will still commit, unless its instructions say that a blocked check stops the commit.&lt;/p&gt;&#10;&lt;p&gt;Look at your agent&amp;rsquo;s last commit: if its summary says a suite or a build was &amp;ldquo;blocked&amp;rdquo;, rerun them on the host today.&lt;/p&gt;&#10;</description></item></channel></rss>