Swift on Lessons learned

Swift Testing withKnownIssue can hide snapshot failures

Fri, 30 Oct 2026 19:45:00 +0100

Two snapshot tests passed while reporting:

No reference was found on disk. Automatically recorded snapshot:

Their Swift Testing withKnownIssue wrapper accepted every issue inside the snapshot assertion, including missing references. Adding a matching: closure limited the exception to expected dependency reports. A deliberate wrong-state snapshot then failed, even though those reports remained known issues.

Restrict which issues count as known

The view had two states: a progress indicator before a check completed, and content afterward. Rendering the content also touched an unimplemented test dependency. The snapshot helper wrapped rendering and comparison in withKnownIssue(isIntermittent: true) to tolerate that report.

Without a matcher, the wrapper also accepted issues from the assertion it surrounded. The initial recording run exited successfully: two tests passed with three known issues. Two of those issues said references were missing; only one was the dependency report the wrapper was meant to tolerate.

The historical fix added this predicate:

String(describing: issue).contains("Unimplemented")

Here is the same policy as a generalized synchronous helper. Pass an existing, nonthrowing snapshot assertion as its body:

import Testing

func withExpectedDependencyIssue(_ body: () -> Void) {
 withKnownIssue(isIntermittent: true) {
 body()
 } matching: { issue in
 String(describing: issue).contains("Unimplemented")
 }
}

The public withKnownIssue matching overload takes a predicate over Issue. Returning false leaves an issue outside this exception. The synchronous overload is rethrows; this helper needs no try because its body is nonthrowing. A throwing assertion needs corresponding throwing handling.

isIntermittent: true allows the body to record no matching issue. That was useful for the progress state, which did not touch the dependency. It does not restrict which issues match. Without matching:, an intermittent wrapper still accepts everything recorded in its scope.

A mutation must change the rendered output

The first attempted verification changed a prefilled field. Both snapshot tests still passed. That was an ineffective mutation for this check; it did not establish that the assertion could detect the intended visual regression.

The stronger mutation completed the check before constructing the view in the test for the before-check reference. It deliberately compared the content state against the progress-state baseline.

That run exited with code 65. The before-check test failed, while the after-check test passed with its expected known issue. Across the two tests, the runner reported three issues, including two known issues. Restoring the original setup produced a successful run of 239 tests in 24 suites with one known issue.

This shows that the narrowed helper rejected an incorrectly rendered state. The session did not run that same wrong-state mutation against the original broad wrapper, so it does not provide an identical before-and-after mutation comparison. The missing-reference reports already demonstrate the broad wrapper’s excessive scope.

Keep the exception smaller than the assertion

The repair used here matched a description substring. It remains brittle: another report containing Unimplemented would also match, and diagnostic wording can change. Where the reporting library exposes a stable public discriminator, prefer that over a broad text search. Better still, provide the missing test dependency so the assertion needs no exception.

Keep that follow-up separate from what was verified. Other snapshot helpers retained broad wrappers; this change did not repair them. A passing suite using this helper says nothing about their failure detection.

These results came from the original Xcode simulator runs. The generalized helper follows Swift Testing’s public matching overload ; it has not been compiled separately.

Before trusting a snapshot helper with a known-issue scope, force a visibly different state against an existing reference and require a failing run. Restore the state and require green again.

A shared Swift timeout left an earlier task without a deadline

Fri, 30 Oct 2026 17:37:00 +0100

A Swift service had a timeout, but waiting for its in-flight work could still hang. A held-fetch regression test reached Swift Testing’s outer limit:

Time limit was exceeded: 60.000 seconds

Starting another check replaced the shared timer while the earlier fetch kept running. The fix was to give each tracked operation its own cancellation deadline and cancel that deadline when the operation ended. That still depends on the fetch cooperating with cancellation.

One timer could not cover overlapping fetches

The service could start another asynchronous check before the previous one finished. A separate tracker retained the in-flight tasks so a caller could wait for their completion. Its waiter took a snapshot of those tasks and awaited their results.

The mistaken assumption was that the service’s existing timeout covered everything in that snapshot. Review found that starting a check reset the shared timer without canceling the previous fetch. Finishing a check also reset the timer so that an older completion could interfere with a newer check’s deadline.

An illustrative sequence makes the ownership mismatch visible:

start A tracked: A shared timer: A
start B tracked: A, B shared timer: B; A's timer reset
finish B tracked: A shared timer: reset
wait for A A still running no cancellation deadline for A

The tracker remembered more work than the timer covered. Making that timer longer would leave the same mismatch.

Put the deadline beside the operation

The correction belonged where the tracker created each fetch task. That scope already knew which task it owned and when it finished.

For each operation, create a separate deadline task that sleeps for the configured duration, then calls cancel() on that operation’s task handle. Keep the deadline handle local. On exit, cancel the deadline and remove the operation from tracking. Forward cancellation from the caller to the fetch as well.

The resulting ownership is:

operation A owns deadline A; completion A cancels deadline A
operation B owns deadline B; completion B cancels deadline B

Canceling a deadline’s sleep must end the deadline task. Swallowing the sleep’s cancellation error and continuing to cancel() would turn normal cleanup into an unintended cancellation request.

Keeping the handles local also means A’s cleanup cannot accidentally cancel B’s deadline through a shared slot.

Prove that the waiter needs this deadline

The regression test used a fetch held until cancellation, a short injected operation timeout, and a signal confirming that the fetch had started. Nothing outside the tracker canceled it. The test then awaited completion and checked that the fetch ended before the waiting caller continued.

Before the implementation used the injected timeout, the test failed at the one-minute limit. After adding the operation-local deadline, the archived test output reports the held-fetch test passing and the focused test target exiting successfully.

That establishes the missing-deadline behavior for a cooperative fake. It does not reproduce the entire overlapping-check sequence live. For your own tracker, also hold A open while B starts and finishes, then confirm that A still receives its own cancellation request.

A cancellation deadline is not a hard timeout

Swift’s Task.cancel() documentation explicitly says cancellation is cooperative. Calling it flags the task and runs cancellation handlers; arbitrary work does not automatically stop. A fetch that ignores cancellation can outlive this deadline and keep a completion waiter suspended.

Later work therefore separated the operation’s cancellation deadline from the caller’s user-facing wait budget. It also added recovery after an unsuccessful check. Giving every operation a deadline did not settle how long a caller should wait or whether continuing after failure was safe.

Before relying on a service timeout, start two operations and finish the newer one first. Check that the older operation still has its own cancellation deadline.

SwiftUI large title disappears after tapping the selected tab again

Fri, 30 Oct 2026 14:38:00 +0100

The Settings large title disappeared after tapping the already-selected tab again. Its space stayed empty; the small title still appeared when scrolling, and the accessibility check passed:

titleBefore=true titleAfter=true

An opaque navigation-bar background was painting over the title. The eventual repair removed a screen modifier’s writes to UINavigationBar.appearance(). Hiding the affected screen’s bar background was an intermediate workaround that introduced a font regression.

The title still existed

The prototype varied the tab view, loading state, scroll-view wrapper, and swipe-back helper. Its accessibility existence checks stayed green. Adding the app’s opaque navigation-bar appearance also produced green checks, even though the title vanished visually.

Counting dark pixels in the prototype’s title region exposed the difference:

Appearance	After first tap	After re-tap
Default	12955	12955
Opaque	12955	0

Those are diagnostic counts from one fixed light-background rendering, not a portable visibility threshold. They contradicted the existence assertion directly.

Before-and-after hierarchy dumps from the running app showed the same title view moving from the navigation bar into the scroll view. The bar’s opaque background then covered it. Initially, the screen displayed a loader without a scroll view, so UIKit built the title inside the bar. Re-tapping the selected tab triggered the transition that exposed the problem on the investigated simulator.

This establishes a rendering mechanism in that environment. It does not establish identical title placement on every supported runtime.

The offending modifier intended to remove a divider and set one screen’s background. Its essential mistake looked like this generalized fragment inside a SwiftUI .onAppear closure:

let appearance = UINavigationBarAppearance()
appearance.configureWithOpaqueBackground()
appearance.backgroundColor = .secondarySystemBackground
appearance.shadowColor = .clear

UINavigationBar.appearance().standardAppearance = appearance
UINavigationBar.appearance().scrollEdgeAppearance = appearance

The closure belongs to a screen; the appearance proxy configures all instances of that class. UIKit applies appearance changes when a view enters a window. A different tab’s later navigation bar can inherit those settings, so screen order becomes part of the bug’s setup.

The repair deleted this global writer. The calling screens already declared their backgrounds through SwiftUI’s per-screen toolbar styling; the investigated runtimes did not need a replacement divider-hiding implementation. Their existing snapshots passed without re-recording. Appearance save-and-restore workarounds around a system-presented screen went away with the leak.

A focused test supplied stronger evidence than the original title lookup: show a styled screen, then create another navigation bar and check its background. It failed before the deletion because the next bar inherited a background, then passed afterward. Separate checks guarded the calling screen’s color and the next large title’s font.

Why the one-line hotfix was reversed

The initial workaround used .toolbarBackground(.hidden, for: .navigationBar) on Settings. It kept the prototype’s title visible under the opaque appearance.

The attempted app snapshot never reproduced the original disappearance. The real screen reloaded into a spinner first, unlike the prototype’s scroll-view-first case. Capturing the loaded window still showed the title without the fix. We dropped that test; it couldn’t serve as a regression guard.

Later checks found that hiding the background also made the large title use the system font instead of the intended custom font. A snapshot cropped to the bar failed with that modifier present and passed after its removal. Once the global opaque write was gone, the workaround no longer had a reason to stay.

Verify the interaction, not just the label

Builds and broad test plans passed after the cleanup. Final manual checks in the running app remained open, and older supported runtimes were untested. The font snapshot and next-bar test do not close that gap.

For release verification:

Visit the screens that previously wrote global appearance, then open Settings. Repeat with a different screen order.
Tap the selected Settings tab again. Capture the title before and after, then scroll through its large-to-inline transition.
Check actual title pixels and contrast against the background. Use the hierarchy to explain occlusion; accessibility existence, or isHittable alone, is not universal proof that text is painted visibly.
Repeat on supported runtimes, checking pushed screens’ bar styling too.

Audit screen-level UINavigationBar.appearance() writes before adding another visibility workaround.

Swift Testing: .serialized did not stop another suite deleting test files

Wed, 28 Oct 2026 14:36:00 +0100

A filesystem test failed while writing its fixture, even though its suite used @Suite(.serialized) and each fixture had a UUID directory. The underlying error was:

NSPOSIXErrorDomain Code=2 "No such file or directory"

Another suite was constructing the application core, whose initializer swept the shared cache directory. Injecting that startup operation, with an inert test value, stopped unrelated tests from deleting each other’s fixtures.

A unique child directory still has a shared parent

The failing suite created a directory, then wrote a file into it. Its own cleanup tests ran serially. Elsewhere in the same Swift Testing process, ordinary tests constructed the core as part of their setup. Every construction triggered a sweep of the common cache parent.

The destructive interleaving was small:

cleanup test: create cache/<UUID>/
unrelated test: construct core -> sweep cache children
cleanup test: write cache/<UUID>/fixture.bin -> missing directory

UUIDs prevented name collisions. They could not protect a child from an operation that deleted every child.

Swift Testing’s parallelization trait documentation describes the scope: .serialized orders a suite’s contained tests and sub-suites. It does not order those tests relative to unrelated suites. Putting the attribute on the failing suite left the other constructors free to run alongside it.

Make ordinary test construction inert

The fix moved the startup sweep behind an injected operation. Production kept the real sweep; the test dependency supplied a no-op. Tests that needed to verify sweeping explicitly selected the real operation.

Here is a generalized closure-based adaptation, independent of a dependency-injection library:

import Foundation

struct StartupSweep {
 var run: () throws -> Void

 static var inert: StartupSweep { StartupSweep(run: {}) }

 static func live(root: URL) -> StartupSweep {
 StartupSweep {
 let manager = FileManager.default
 for child in try manager.contentsOfDirectory(
 at: root, includingPropertiesForKeys: nil
 ) {
 try manager.removeItem(at: child)
 }
 }
 }
}

struct CacheCore {
 init(startupSweep: StartupSweep) throws {
 try startupSweep.run()
 }
}

The production composition root passes .live(root: cacheRoot). The ordinary test factory passes .inert; neither caller relies on an initializer default. This example propagates filesystem errors so a cleanup test cannot pass by silently ignoring a failed sweep.

For a test-owned root containing a prepared fixture file, the two assertions look like this. This is a test-body excerpt using Testing:

// Ordinary construction must preserve the existing fixture.
_ = try CacheCore(startupSweep: .inert)
#expect(FileManager.default.fileExists(atPath: fixture.path))

// Cleanup coverage explicitly opts into the real operation.
_ = try CacheCore(startupSweep: .live(root: root))
#expect(!FileManager.default.fileExists(atPath: fixture.path))

Use a disposable root for this check. Never point the illustrative live operation at a directory containing files you need. Injecting a separate root per test is another way to isolate filesystem work when the design permits it.

Prove the constructor contract, then repeat the broader target

The historical investigation used an isolated-suite control: the cleanup suite repeated 30 times with zero issues. The broader core target failed on its first run. That narrowed the search to interference from outside the serialized suite.

A regression test then created a fixture, built the core-under-test dependencies, and asserted that the file still existed. Before the injection fix, that assertion failed. It converted an intermittent write failure into a direct check of the destructive constructor behavior.

After the fix, the archived broader-target output reported:

exit: 0
** TEST SUCCEEDED **
issues:
0
max repetition:
repetition 30
✔ Test run with 165 tests in 28 suites passed after 2.806 seconds.

Those results verify the original fix under Xcode and iOS Simulator. The closure-based adaptation above has not been compiled separately. The reported test count is the runner’s summary, not a total multiplied by the repetition limit.

When a serialized suite loses files, check every initializer that can sweep their shared parent, including constructors called by unrelated suites.

A five-second Swift Testing wait timed out after its loop had run five times

Thu, 22 Oct 2026 16:12:00 +0200

Five concurrency tests passed on their own in 0.086 seconds. In the full target, they failed together at about 6.8 seconds:

✘ Test fetch_threeSources_asksForEverySourceAtOnce() recorded an issue at FetcherConcurrencyTests.swift:21:30:
 Expectation failed: ContinuousClock.now < deadline
✘ Test fetch_threeSources_asksForEverySourceAtOnce() failed after 6.816 seconds with 1 issue.

It looked like a deadlock in the code under test. It was a test helper measuring the wrong thing.

Setup

Swift Testing, an Xcode test target that mixes @MainActor unit suites with snapshot suites.
Two test plans run the same target. One sets "parallelizable": false for it, and the other doesn’t. Only the parallel plan failed.
A small polling helper that the concurrent-fetch tests use to wait for a gated fake service to see requests:

static func until(
 sourceLocation: SourceLocation = #_sourceLocation,
 _ condition: () -> Bool
) async throws {
 let deadline = ContinuousClock.now.advanced(by: .seconds(5))
 while !condition() {
 try #require(ContinuousClock.now < deadline, "Timed out waiting for the condition to hold",
 sourceLocation: sourceLocation)
 await Task.yield()
 }
}

The helper runs on the main actor. Each Task.yield() lets the queued work, in this case the fetcher opening its requests, run before the condition is checked again.

Reproduce before guessing

Running the target alone under the parallel plan with -test-iterations and -run-tests-until-failure failed 3 out of 3 times, always in the same tests and always at the wall-clock deadline. So it wasn’t a rare flake. It failed every time under that plan.

To check whether the waits were stuck or just slow, I temporarily replaced the deadline with a counter and printed what each wait used:

let start = ContinuousClock.now
var turns = 0
while !condition() {
 turns += 1
 await Task.yield()
}
print("WAITPROBE turns=\(turns) elapsed=\(ContinuousClock.now - start) at \(sourceLocation.line)")

5 5.766 FetcherConcurrencyTests.swift:54
5 5.712 FetcherConcurrencyTests.swift:21
4 6.411 FetcherConcurrencyTests.swift:37
3 5.704 SyncLoggingTests.swift:25
3 5.338 InFlightTests.swift:20

Every wait finished, but the slowest needed only 3 to 5 turns and took more than 5 seconds of wall time. The condition was met on almost the first chance the fetcher got to run. It just didn’t get that chance for seconds.

The reason showed up when I sorted tests by duration. In the parallel plan, snapshot suites share the process and the main actor, and each render holds the actor synchronously, some of them for over 7 seconds. While a render ran, the waiting test’s next turn just sat in the queue, and the clock kept running.

Dead end: a longer timeout

I tried a 60-second deadline as an experiment. Both runs passed, with the concurrency tests taking about 7.9 seconds each.

That shows the cause, but it isn’t a fix. 60 is no more principled than 5. A heavier snapshot suite brings the failure back, and a test that really hangs now takes a minute to report it. I reverted it.

Fix: spend both budgets before failing

The wait should fail only once it has had enough chances to make progress and enough time has passed:

static func until(
 sourceLocation: SourceLocation = #_sourceLocation,
 timeBudget: Duration = .seconds(5),
 _ condition: () -> Bool
) async throws {
 let deadline = ContinuousClock.now.advanced(by: timeBudget)
 var turns = 0
 while !condition() {
 let hasBudgetLeft = turns < Constants.turnBudget || ContinuousClock.now < deadline
 try #require(hasBudgetLeft, "Timed out waiting for the condition to hold",
 sourceLocation: sourceLocation)
 turns += 1
 await Task.yield()
 }
}

enum Constants {
 /// A wait under a fully loaded run of this target took at most 5 turns.
 static let turnBudget = 100
}

On an idle main actor, 100 turns go by in milliseconds, so a condition that never holds still fails just after the time budget. On a starved actor, the wait keeps going until it has had real chances to run.

Pin it with a controlled starvation test

The full target is a slow, noisy reproduction, so I wrote a test that intentionally causes starvation. It makes timeBudget a parameter, holds the main actor for longer than that budget, and makes the condition true only afterward:

@MainActor
@Suite
struct WaitTests {
 @Test
 func until_whenTheMainActorIsHeldPastTheTimeBudget_waitsForTheCondition() async throws {
 let holds = LockIsolated(false)
 Task { @MainActor in
 Thread.sleep(forTimeInterval: 0.3) // stands in for a snapshot render
 Task { @MainActor in holds.setValue(true) }
 }
 try await Wait.until(timeBudget: .milliseconds(100)) { holds.value }
 }

 @Test
 func until_whenTheConditionNeverHolds_fails() async {
 await withKnownIssue {
 try await Wait.until(timeBudget: .zero) { false }
 }
 }
}

Before the fix, it failed after 0.335 seconds with the same ContinuousClock.now < deadline error. After the fix, it passed in 0.303 seconds. The second test makes sure the helper can still fail.

Verify both directions

Check	Result
Parallel plan, before the fix	0 of 3 passed
Parallel plan, after the fix	5 of 5 passed
Serial plan, whole target	690 tests passed
Mutant: fetcher limited to one request at a time	all 5 concurrency tests failed, each at 5.0 s

The last row matters most. A longer timeout would also have made the parallel runs pass. What it wouldn’t keep is fast failure for a real bug. I changed the fetcher’s concurrency limit from 4 to 1, so the “all three requests are open at once” condition could never be true. All five tests caught it, each at the 5-second time budget, and I reverted the mutation.

One caveat: that mutant ran under the serial plan. Under heavy parallel load, a real hang fails after 100 turns, which takes longer than 5 seconds. That’s the trade-off I chose, and it applies to this helper and this workload. It isn’t a general fairness guarantee for Task.yield().

A second guess hiding in the same tests

Three tests waited for the first request and then gave the scheduler “five settling turns” before asserting that all three were open. That turn count was another guess, and it could fail under load the same way. The fix was to wait for the exact condition:

// before
try await Wait.until { service.pendingCount > 0 }
await Wait.untilSettled()
#expect(service.pendingCount == 3)

// after
try await Wait.until { service.pendingCount == 3 }

Takeaway

A wall-clock timeout in a polling helper measures how busy the process is, not whether your code is stuck. When a wait times out only in the big parallel run, count its turns before you suspect a deadlock. If it got only a handful of turns, the actor was starved. Make the helper fail only after both a time budget and a turn budget run out. Then check the fix with a mutation that should hang and make sure it still fails quickly.

Try it on your own polling helper: add a turn counter and run the full target in parallel once. If the worst wait used single-digit turns but took seconds, the timeout is measuring main-actor contention.

Snapshot precision 0.99 let a wrong text line pass all seven screen tests

Tue, 13 Oct 2026 17:04:00 +0200

A code review claimed that a list screen’s snapshot tests couldn’t fail on the very line they were written to pin: a small gray “Location” line under each search result. To check, I broke that line on purpose. The rows got the container’s short name instead of its full name, so every location line rendered the wrong text. Then I ran the suite:

✔ Test listScreenDeletedItemsShowTheirOrigin() passed after 0.296 seconds.
✔ Test listScreenSearchResultsShowTheirLocation() passed after 0.708 seconds.
...
✔ Test run with 7 tests in 1 suite passed after 0.793 seconds.

All seven passed against the broken code. If your snapshot tests record a whole screen at .device layout with a precision below 1.0, they may be just as blind.

Setup

The project used swift-snapshot-testing 1.19.2, with Swift Testing on the iOS simulator. A shared defaults file sets both tolerances:

public static let perceptualPrecision: Float = 0.90
public static let precision: Float = 0.99

The two gates do different jobs:

perceptualPrecision decides whether a single pixel counts as different. 0.90 absorbs anti-aliasing drift between developer Macs.
precision decides what fraction of pixels must match. 0.99 lets 1% of the image differ by any amount.

The 0.99 was there to unblock CI: one render drifted by about 3,600 pixels between machines. The comment next to it even warned that it was “wide enough to hide a small real regression”. Every screen test used it anyway.

The arithmetic

An iPhone 17 Pro device frame is 1179×2556 pixels.

1179 × 2556 = 3,013,524 px
1% of that = 30,135 px may differ freely
wrong location = ~900 px actually differed

A 1% budget sounds strict. On a full screen it’s about a 58-point square of arbitrary change, more than all the location lines on the screen combined. The test was recorded correctly and passed honestly. It just had no way to fail on the thing in its name.

Measuring the mutation

To size the gate, record the mutated renders, diff them against the committed references, then restore the references. A few lines of Pillow give a rough count (a pixel counts if any channel moves by more than 25):

import sys
from PIL import Image

a = Image.open(sys.argv[1]).convert("RGB")
b = Image.open(sys.argv[2]).convert("RGB")
pa, pb = a.load(), b.load()
w, h = a.size
n = sum(1 for y in range(h) for x in range(w)
 if max(abs(c1 - c2) for c1, c2 in zip(pa[x, y], pb[x, y])) > 25)
print(f"{n / (w * h):.2%} of {w}x{h}")

TEST_RUNNER_SNAPSHOT_TESTING_RECORD=all xcodebuild test ... -only-testing:MyTests/ListScreenSnapshotTests
git show HEAD:path/to/reference.png > /tmp/head.png
python3 pxdiff.py /tmp/head.png path/to/reference.png
git checkout -- path/to/__Snapshots__ # put the real references back

...DarkAppearance...: 0.03% of 1179x2556
...SearchResultsAtTheDynamicTypeCap...: 0.14% of 1179x2556
...SearchResultsShowTheirLocation...: 0.03% of 1179x2556
...DeletedItemsShowTheirOrigin...: 0.03% of 1179x2556

The smallest real defect was 0.03%, about 900 pixels, so the budget had to sit well below that. 0.9999 allows 301.

The fix

Give the screen suite its own precision and write the numbers down where the next person will see them:

as: .image(
 precision: Constants.precision,
 perceptualPrecision: SnapshotDefaults.perceptualPrecision,
 layout: .device(config: config)
)

private enum Constants {
 /// A device frame is 1179×2556 px, so the default 1% budget is 30,135 px: wider than every
 /// Location line on the screen together. The wrong name moves about 900 px; this leaves a
 /// 301 px budget, while `perceptualPrecision` still absorbs anti-aliasing drift per pixel.
 static let precision: Float = 0.9999
}

Then rerun with the mutation still in place. It has to go red:

✘ Test listScreenDeletedItemsShowTheirOrigin() failed after 0.594 seconds with 1 issue.
✔ Test listScreenSortMenuOpen() passed after 0.751 seconds.
✔ Test listScreenBrowsingTheRoot() passed after 0.807 seconds.
✘ Test listScreenSearchResultsAtTheDynamicTypeCap() failed after 1.220 seconds with 1 issue.
✘ Test listScreenSearchResultsShowTheirLocation() failed after 1.618 seconds with 1 issue.
✔ Test listScreenSearchWithNoResults() passed after 1.659 seconds.
✘ Test listScreenDarkAppearance() failed after 2.036 seconds with 1 issue.

The four cases that draw location or origin lines failed. The three cases without them passed, which is also correct. After reverting the mutation, all seven passed again at 0.9999.

A 301-pixel budget is tight, and I verified it on one machine. If cross-machine drift trips it, look at the drift before you loosen the number.

Shrink the frame when you can

For a single component, a tight frame beats a tight precision. The same project measured a toolbar button’s enabled-to-disabled color change at 1,808 pixels, or 0.06% of a device frame. Rendered at 140×60 with layout: .fixed(width:height:), those 1,808 pixels became 2.4% of the image, well past a 1% budget. Keep .device for tests that are really about the whole screen, and size their precision from the smallest change they claim to catch.

Takeaway

precision is a fraction of the image, so a fixed value forgives more pixels as the frame grows. Before you trust a green snapshot, break the thing it’s named after, count the changed pixels, and compare that with (1 - precision) × width × height.

Pick one snapshot test in your suite, break its subject on purpose, and see whether it goes red.

xcodebuild says TEST SUCCEEDED, the xcresult says "Issues recorded without an associated test or suite"

Mon, 28 Sep 2026 12:16:00 +0200

A branch passed locally. xcodebuild test exited 0 and printed:

** TEST SUCCEEDED **

The fastlane coverage lane on the same branch failed with Tests have failed and didn’t name a test. Scan’s xcpretty output doesn’t show Swift Testing results, so the log didn’t include any failing tests. The result bundle had the answer:

xcrun xcresulttool get test-results summary --path Test.xcresult

{'result': 'Failed', 'totalTestCount': 2320, 'passedTests': 2317, 'failedTests': 1, ...}
testName: Issues recorded without an associated test or suite
failureText: Issue recorded: failed - Unimplemented: Unimplemented AppVersionClient.currentVersion …
 This issue was emitted from tests running in a host application.
 This can lead to false positives, where failures could have emitted from live application code at launch time, and not from the current test.

If your console is green and fastlane, Xcode Cloud or anything else that reads the .xcresult is red, check the summary for that pseudo-test before you go looking for a flaky test.

Setup

Xcode 26, Swift Testing, an app-hosted unit test target (tests run inside the real app process).
swift-dependencies 1.14, with a client whose testValue calls IssueReporting.unimplemented(...). That’s the usual way to make sure a test sets every dependency it uses.
The app’s root view is a login screen. On appear, it asks a backend for the minimum supported version and compares it with currentVersion().

What happened

Because the app hosts the tests, it launches as it does for a user, and its login screen sends the version request. When the response arrives, the screen reads the version client inside a test process, which resolves to the unimplemented test value and records an issue. No test is running on that code path, so the issue gets attached to no test.

The console run only shows it as a warning:

⚠︎ Test «unknown» recorded a warning at VersionClient.swift:24:1: Issue recorded
↳ failed - Unimplemented: Unimplemented AppVersionClient.currentVersion …

The result bundle counts it as a failure, though, and fastlane reads it.

Main didn’t fail, and the new branch didn’t touch any of this code. Its only addition was a new app-hosted suite that took about seven seconds. That extra time let the response arrive while tests were still running. On main, the run was over before the response came back. This is timing-dependent, so a faster backend or a slower one can hide the failure or bring it back.

Reproduce one target and compare the result bundle summaries:

Run	Total	Failed
Hosted target, as is	21	1
Same, new suite skipped (`-skip-testing`)	16	0
Same, with the fix	20	0

The 21 in the first row is 20 real tests plus the pseudo-test.

Dead end: set the dependency at launch

The obvious fix is to give the app a live value in its entry point:

prepareDependencies {
 $0.appVersion = AppVersionClient { Bundle.main.shortVersion }
}

It built and changed nothing. The same failure showed up again. In a test process, swift-dependencies resets its cached values for each test: it registers an XCTest observer that clears the cache whenever a test case starts. The value you prepare at app launch is gone once the tests start, and the later response reads the test value again.

Fix: don’t show the app UI while hosting tests

isTesting from IssueReporting (re-exported by Dependencies) is true inside a test process. Gate the scene’s content on it:

import Dependencies
import SwiftUI

@main
struct MyApp: App {
 var body: some Scene {
 WindowGroup {
 // Unit tests run inside this app. Its login screen would start a
 // network request whose response reads an unimplemented test dependency.
 if !isTesting {
 RootView()
 }
 }
 }
}

This is the workaround in the library’s Testing gotchas docs. Before applying it, check that no hosted test needs the app’s window or a view in the scene. Here, none did, because the tests only needed a real UIApplication, which still exists with an empty WindowGroup. The shipping app is unchanged, since isTesting is false outside a test run.

With the fix, the hosted target passed 20 of 20, the whole coverage lane reported 0 failures on both schemes, and the test run stopped sending requests to the dev backend.

That last part is a nice bonus. A test host that shows its real UI talks to real servers from your CI, and anything it logs (request and response bodies, anti-forgery tokens) ends up in your test logs.

Takeaway

An app-hosted test run also runs your app. Anything the app starts on its own at launch can record a failure that belongs to no test and shows up only in the .xcresult. Keep the UI out of the test host with if !isTesting, and when the console and the lane disagree, read xcresulttool get test-results summary first.

Try it on your own app-hosted target: grep a recent test log for Test «unknown» recorded. Each hit is an issue recorded outside any test, and the result bundle may already count it as a failure.

FSKit after mount -F: "ls: fts_read: Operation not permitted"

Sat, 26 Sep 2026 16:30:00 +0200

My new FSKit filesystem mounted without complaint, but nothing inside it could be read:

$ ls -la ~/mnt/storage-test
ls: fts_read: Operation not permitted
total 0
$ cat ~/mnt/storage-test/sub/a.txt
cat: /Users/me/mnt/storage-test/sub/a.txt: Operation not permitted

stat on the mount point worked. If your FSKit extension does the same after a mount -F from the command line, check its sandbox before your code: the extension can see the source folder’s metadata but may not read its contents.

The setup

macOS 26, a file system extension built on FSKit’s FSUnaryFileSystem that exposes a local folder through a mount point
A small host app, StorageHost.app, in /Applications; its executable runs mount -F -t storagefs <source> <mount point>
The extension StorageFS.appex with the FSKit entitlement and the app sandbox
Info.plist with FSSupportsPathURLs set to true and FSRequiresSecurityScopedPathURLResources set to false

What went wrong

The first mount was over ~/Downloads, so the obvious suspect was privacy protection (TCC) for that folder. A fresh ~/storage-test with one file in it failed the same way, which ruled TCC out and pushed the search elsewhere.

My coding agent’s first log queries came back empty, and it told me there were no log entries. That was wrong. In zsh, log is a builtin, so log show ... 2>/dev/null runs the builtin, sends its too many arguments error to /dev/null, and prints nothing. Call /usr/bin/log by its full path. With that, the kernel had the answer:

kernel (Sandbox) 3 duplicate reports for Sandbox: StorageFS(83505) deny(1) file-read-data /Users/me/storage-test

The extension’s own log showed the failures from FSKit’s side: readDirectory:...reply:error:1 and readFrom:...reply:error:1. Error 1 is EPERM, which ls turns into fts_read: Operation not permitted.

Fix

mount -F gives the extension a plain path, not a security-scoped URL, so a sandboxed process can’t read an arbitrary path. For development, grant temporary sandbox exceptions in the extension’s .entitlements file:

<key>com.apple.security.temporary-exception.files.home-relative-path.read-write</key>
<array>
	<string>/</string>
</array>
<key>com.apple.security.temporary-exception.files.absolute-path.read-write</key>
<array>
	<string>/Volumes/</string>
</array>

This covers your home folder and external drives. It doesn’t override privacy protection, so ~/Downloads, ~/Documents, and ~/Desktop may still be refused; test with a folder elsewhere. Temporary exceptions are for development on your own machine. A shipping extension should use security-scoped resources instead. FSRequiresSecurityScopedPathURLResources in Info.plist requests that; I haven’t managed a clean test yet (see below).

The same logs flagged two more problems that I fixed along the way:

-[FSModuleVolume(Project) getMaxFileSizeInBits]: Volume implements neither maxFileSizeInBits nor maxFileSize. One of them must be implemented.
-[FSVolumeConnector getStandardItemAttributesForItem:replyHandler:]_block_invoke: Reported attributes are incomplete

The first needs maximumFileSizeInBits on the volume. The second went away once the item attributes set flags, which cleared the log entry.

Reinstalling switches the module off

To test each change, I replaced the app: rm -rf /Applications/StorageHost.app, copied the new build, and launched it once. After that, mount failed outright:

Module com.example.storage.fs is disabled!
mount: Unable to invoke task

pluginkit disagreed. pluginkit -e use -i com.example.storage.fs succeeded, and pluginkit -m listed the extension with a +, which means enabled. FSKit still refused. It keeps its own switch in System Settings → General → Login Items & Extensions → File System Extensions, and I didn’t find a command-line way to flip it. After I turned StorageFS back on there, the next mount listed and read files, and appends, renames, truncation, and an 8 MB write-and-readback all worked.

The disabled module also spoiled my one test of security-scoped resources. With FSRequiresSecurityScopedPathURLResources set to true, mount failed with Unable to invoke task, but the reinstall had just switched the module off, so the failure says nothing about the setting.

Why

An FSKit module is an app extension, and FSKit requires it to run in the app sandbox. When an app hands a folder to the extension, sandbox access can travel with it as a security-scoped URL. mount -F on the command line passes only a string. The sandbox lets the extension look up metadata for that path, which is why the mount and stat worked, and denies file-read-data, which every directory listing and file read needs.

Takeaway

If an FSKit mount works and every read returns Operation not permitted, search the kernel log for deny(1) file-read-data with /usr/bin/log, then give the extension sandbox access to the source path. After every reinstall, check the File System Extensions switch before you trust a test result.

If you’ve got FSRequiresSecurityScopedPathURLResources working with a command-line mount -F, I’d like to hear how.

Waiting for logout before login still left a race inside the handshake

Mon, 21 Sep 2026 14:49:00 +0200

Correction 2026-10-31: The earlier version attributed the reported failure to an idle timer. Later source review ruled out that timer for this app. The injected-teardown reproduction and exclusion-gate fix remain valid; the production trigger is unknown.

Users authenticated with Face ID, and the login failed anyway with a generic “login failed” error. The password was right, and the biometric enrollment was fine. The server’s log for the affected user showed the last handshake step arriving on a session in its initial state, with no user bound to it.

The app waited for any teardown already running before starting login. A dedicated test injected a new teardown after that wait and reproduced the same server-state signature. Holding an exclusion gate for the whole handshake fixed that reproduction, but did not establish the cause of the reported incident.

Setup

A stateful, multi-step login (SRP-style): start a session, send the identity, send the proof, finish. The server tracks progress per session cookie.
A session teardown: sign out on the server, then clear the local cookie jar. begin() returns immediately, and concurrent begin() calls join one teardown.
An earlier fix for a related bug: before its handshake, login called waitForCompletion() so a sign-out still in flight couldn’t destroy the new session.
The reproduction explicitly calls begin() during the handshake. It does not rely on an idle timer.

login: { credential in
 await sessionTeardown.waitForCompletion() // once, before the handshake
 return await client.login(credential) // four round trips
}

Reproduce with the real stack

A dedicated live test drove the production login code against a development server and fired begin() at a fixed delay into the handshake. At 500 ms:

start -> {"state":"new"}
send identity -> {"state":"identified", "salt":...}
send proof (M1) -> {"state":"proof-verified", "M2":...}
logout() <- the teardown fires here
POST /logout
finish (keys sent)
Logout success
Session teardown: cookie jar cleared
Error: state "new"
-> failed(.other)

This matched the server-state signature logged for the user. In the reproduction, the one-time wait had nothing to wait for: no teardown was running when login started. The test started one between two round trips.

The trap inside the trap: which effect does the damage

The code comments blamed the cookie-jar clear: the concluding step would go out without a session cookie and land on an anonymous session. That would suggest a small fix: defer only the local cleanup.

In the reproduction’s trace, the cookie jar was cleared after the finish step had already gone out with valid cookies. The server sign-out reset the half-authenticated session, and the concluding step then arrived at a session the server had put back to new. Deferring only the jar clear would still fail this exact trace.

Before you choose what to defer, read the ordering on the wire. The local effect you can see easily may not be the one that broke things.

Fix: hold exclusion for the whole protocol

The teardown gets a gate. Login holds it from the first round trip to the last. A begin() that arrives while the gate is held is honored but deferred—neither its sign-out nor its cookie clear runs until the handshake releases the gate.

private struct TeardownState {
 var inFlight: Task<Void, Never>?
 var handshakes = 0 // a count, so overlapping logins can't release each other's gate
 var isHeldBack = false
}

begin: {
 state.withValue { s in
 guard s.inFlight == nil else { return } // join the running one
 guard s.handshakes == 0 else { s.isHeldBack = true; return }
 s.inFlight = Task { await runTeardown() } // clears inFlight when done
 }
},
beginHandshake: {
 // Wait out a running teardown, then claim the gate. Loop, because the slot
 // is only free on a later pass.
 while let running = state.withValue({ s -> Task<Void, Never>? in
 guard let t = s.inFlight else { s.handshakes += 1; return nil }
 return t
 }) {
 await running.value
 }
},
endHandshake: {
 state.withValue { s in
 s.handshakes -= 1
 guard s.handshakes == 0, s.isHeldBack else { return }
 s.isHeldBack = false
 s.inFlight = Task { await runTeardown() }
 }
}

func excludingTeardown<T>(_ body: () async -> T) async -> T {
 await beginHandshake()
 defer { endHandshake() }
 return await body()
}

login: { credential in
 await sessionTeardown.excludingTeardown {
 await client.login(credential)
 }
}

The other properties stayed the same. begin() still returns immediately. A second begin() still means one teardown, whether that teardown is running or held back. A teardown whose sign-out throws still releases its waiters. Logout still happens: it runs right after login, which then ends in a signed-out state on purpose.

Later source review found that this app created its idle timer only after the full login completed and dropped it when the login screen appeared. The shipped revision had the same wiring. The earlier claim that this timer disproved a signed-in-only assumption was withdrawn. The gate provides exclusion without depending on that reachability argument.

A sibling app did retain its timer after logout and needed a signed-in guard. That was a separate timer-lifetime defect, not evidence that the timer caused this app’s reported failure.

Verification

Unit test for the new direction. A teardown begun inside excludingTeardown runs neither effect until the body returns. The expected order is [handshake, signOut, clearCookieJar]. Inside the body, the test awaits waitForCompletion() rather than Task.yield(), so a buggy teardown is guaranteed a chance to run. A yield would leave that to the scheduler.
Unit test for the old direction. A teardown already running still finishes before the handshake starts. That is the guarantee the earlier fix provided.
Injected-teardown sweep. The reproduction fires the teardown at 100, 200, 300, 400, 500 and 700 ms. It had been marked XCTExpectFailure. After the fix, every delay logged in:

teardown at 100ms -> loggedIn
teardown at 200ms -> loggedIn
...
teardown at 700ms -> loggedIn

The expected-failure marker was removed in the same commit.

What this doesn’t prove

The reproduction establishes a failure mechanism and verifies the exclusion fix. It does not prove that this mechanism caused the reported incident or identify its triggering call. We ruled out the idle timer for this app; the field trigger remained unknown.
The dedicated reproduction and the unit tests verify the fix. The live test class still exited with failures. They came from a separate host-app dependency problem in the test process, and another test in that class still tracks an unrelated second-login bug. Don’t read “the sweep passes” as “the live suite is green”.
The ordinary test plan passed, but live tests are opt-in there, so it says nothing about the server interaction.
Which delays hit the window depends on latency to the server. A sweep is better than a single delay, but it is still a sample.

Takeaways

Waiting once for cleanup before a multi-step protocol protects only that instant. If cleanup can start during the protocol, exclude it for the whole protocol.
Defer cleanup, don’t drop it. Queue the request and run it when the gate is released.
Count holders instead of using a flag, so two overlapping logins can’t release the gate from under each other.
Read the wire ordering before you choose what to defer. Here, the server sign-out did the damage, not the local cookie clear that everyone suspected.
A matching reproduction does not identify the production trigger. Trace the initiating call in the affected build before naming it as the cause of an incident.

Swift on Lessons learned

Swift Testing withKnownIssue can hide snapshot failures

Restrict which issues count as known

A mutation must change the rendered output

Keep the exception smaller than the assertion

A shared Swift timeout left an earlier task without a deadline

One timer could not cover overlapping fetches

Put the deadline beside the operation

Prove that the waiter needs this deadline

A cancellation deadline is not a hard timeout

SwiftUI large title disappears after tapping the selected tab again

The title still existed

A local modifier changed later navigation bars

Why the one-line hotfix was reversed

Verify the interaction, not just the label

Swift Testing: .serialized did not stop another suite deleting test files

A unique child directory still has a shared parent

Make ordinary test construction inert

Prove the constructor contract, then repeat the broader target

A five-second Swift Testing wait timed out after its loop had run five times

Setup

Reproduce before guessing

Dead end: a longer timeout

Fix: spend both budgets before failing

Pin it with a controlled starvation test

Verify both directions

A second guess hiding in the same tests

Takeaway

Snapshot precision 0.99 let a wrong text line pass all seven screen tests

Setup

The arithmetic

Measuring the mutation

The fix

Shrink the frame when you can

Takeaway

xcodebuild says TEST SUCCEEDED, the xcresult says "Issues recorded without an associated test or suite"

Setup

What happened

Dead end: set the dependency at launch

Fix: don’t show the app UI while hosting tests

Takeaway

FSKit after mount -F: "ls: fts_read: Operation not permitted"

The setup

What went wrong

Fix

Reinstalling switches the module off

Why

Takeaway

Waiting for logout before login still left a race inside the handshake

Setup

Reproduce with the real stack

The trap inside the trap: which effect does the damage

Fix: hold exclusion for the whole protocol

Verification

What this doesn’t prove

Takeaways