Bring test files and fixtures into typecheck scope - #578
Merged
TheGreatAxios merged 2 commits intoAug 23, 2026
Merged
Conversation
tests/, evals/, and scripts/ were outside tsconfig's include, so a field rename or deletion left stale references in tests unnoticed until runtime. Widen include to cover them, excluding only the hermetic capability-eval sandbox repos (tests/fixtures/**, evals/capability/cases/**), which are copied into per-run workdirs rather than imported and in one case deliberately contain invalid syntax as fixture content. Fixes the batch of pre-existing errors the widened scope surfaced: real field/signature drift (LastCycleSource.id -> sourceId, AgentProfile.prompt -> systemPromptRole, ApprovalScope gaining id/label, StringToolHandler requiring an AbortSignal, InboundMessage's actual wire shape, ProviderSettings.baseURL becoming required, PricingFetcherOptions.fetchImpl over-typed as `typeof fetch`) plus routine noUncheckedIndexedAccess/exactOptionalPropertyTypes fallout in test helpers, narrowly cast where a test needs a loose shape.
- Run prettier on the 7 files CI flagged. - Narrow the tests/fixtures/** typecheck exclusion so it no longer covers tests/fixtures/crash-run or tests/fixtures/plugins/implement-feature — both import real production modules by relative path and crash-run is spawned by tests/integration/crash-finalize.test.ts as a live exercise of that code. Excluding them would have recreated the gap this PR closes. List the remaining hermetic eval fixtures individually instead of a blanket glob. - Typechecking implement-feature surfaced exactly the kind of drift this PR targets: its relative imports were two directories short (stale), so CommandPlugin/WorkflowPlugin resolved to nothing. Fixed the path depth. The fixture isn't wired into any test today, so this was invisible until now.
TheGreatAxios
enabled auto-merge
August 23, 2026 20:54
TheGreatAxios
added a commit
that referenced
this pull request
Aug 23, 2026
Merging this branch with main exposed two typecheck failures that the branch alone could not show, because it never touches tsconfig.json and PR #578 widened typecheck scope to tests/ and evals/ after this branch was cut. parseCaseJson narrowed raw.tier with a `tier as EvalTier` cast, which is not a type guard, so it returned string where the EvalCase field is EvalTier. Replaced with a real isEvalTier predicate. The four tier-* sandbox fixtures now fall inside typecheck scope. They have their own package.json and tier-hard is intentionally buggy, so they join the existing eval-sandbox exclusions. crash-run and plugins/implement-feature stay in scope deliberately — they import production modules and #578 carved them out for that reason. Also swept the exclusion entries for fixtures this branch deletes.
TheGreatAxios
added a commit
that referenced
this pull request
Aug 27, 2026
…e-the-typecheck-scope-so-deleted-fields Bring test files and fixtures into typecheck scope
TheGreatAxios
added a commit
that referenced
this pull request
Aug 27, 2026
Merging this branch with main exposed two typecheck failures that the branch alone could not show, because it never touches tsconfig.json and PR #578 widened typecheck scope to tests/ and evals/ after this branch was cut. parseCaseJson narrowed raw.tier with a `tier as EvalTier` cast, which is not a type guard, so it returned string where the EvalCase field is EvalTier. Replaced with a real isEvalTier predicate. The four tier-* sandbox fixtures now fall inside typecheck scope. They have their own package.json and tier-hard is intentionally buggy, so they join the existing eval-sandbox exclusions. crash-run and plugins/implement-feature stay in scope deliberately — they import production modules and #578 carved them out for that reason. Also swept the exclusion entries for fixtures this branch deletes.
TheGreatAxios
added a commit
that referenced
this pull request
Aug 27, 2026
…e-the-typecheck-scope-so-deleted-fields Bring test files and fixtures into typecheck scope
TheGreatAxios
added a commit
that referenced
this pull request
Aug 27, 2026
Merging this branch with main exposed two typecheck failures that the branch alone could not show, because it never touches tsconfig.json and PR #578 widened typecheck scope to tests/ and evals/ after this branch was cut. parseCaseJson narrowed raw.tier with a `tier as EvalTier` cast, which is not a type guard, so it returned string where the EvalCase field is EvalTier. Replaced with a real isEvalTier predicate. The four tier-* sandbox fixtures now fall inside typecheck scope. They have their own package.json and tier-hard is intentionally buggy, so they join the existing eval-sandbox exclusions. crash-run and plugins/implement-feature stay in scope deliberately — they import production modules and #578 carved them out for that reason. Also swept the exclusion entries for fixtures this branch deletes.
TheGreatAxios
deleted the
cl-5759-test-files-are-outside-the-typecheck-scope-so-deleted-fields
branch
August 27, 2026 23:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes CL-5759.
Verified the gap was real
tsconfig's
includewassrc/**/*.tsandpackages/**/*.tsonly —tests/,evals/, andscripts/(159 test files) were outside it. Confirmed empirically: introduced a type error insrc/state.test.ts(caught bytsc --noEmit) and the same kind of error intests/unit/plugin-repo-locator.test.ts(not caught), then reverted both.Fix
Widened
includeto addtests/**/*.ts,evals/**/*.ts,scripts/**/*.ts. Excluded, by name, only the capability eval suite's hermetic sandbox repos undertests/fixtures/**and all ofevals/capability/cases/**— these are copied into per-run eval workdirs rather than imported by our code, own their own package.json, andtests/fixtures/buggy-servicedeliberately ships a bug as fixture content for an eval case.Two fixture directories are deliberately not in that exclusion list, even though they live under
tests/fixtures/:tests/fixtures/crash-runandtests/fixtures/plugins/implement-feature. Both import real production modules by relative path (src/index.ts,src/session/active-run.ts,src/session/state.ts,src/tui/runner.ts,src/workflows/definition.ts,src/tui/commands/registry.ts) andtests/integration/crash-finalize.test.tsspawnscrash-run's scripts as live exercises of that code — excluding them would recreate exactly the silent-drift gap this PR closes. Typecheckingimplement-featuresurfaced a real instance of that drift: its relative import paths were two directories short (stale since the fixture was last touched), soimport type { CommandPlugin } from "../../../src/tui/commands/registry.js"resolved nowhere. Fixed the path depth; the fixture isn't wired into any test today, so this was invisible until now.bun run typecheck(tsc --noEmit) is unchanged, so the wider scope runs in the same command contributors already use.Errors surfaced
116 pre-existing errors across ~28 files, plus the two broken import paths above. All fixed properly, none suppressed:
LastCycleSource.id→sourceId,AgentProfile.prompt→systemPromptRole,ApprovalScopegaining requiredid/label,StringToolHandlernow requiring anAbortSignalargument,InboundMessage's actual wire shape (not the informal{role, content}a test assumed),ProviderSettings.baseURLbecoming required,PricingFetcherOptions.fetchImplover-typed astypeof fetch(which also demands static members likepreconnectthat no test double implements, narrowed to a plain call signature), and the two stale relative-import paths intests/fixtures/plugins/implement-featuredescribed above.noUncheckedIndexedAccess/exactOptionalPropertyTypesfallout in test helpers (array-index access, optional-field passthrough), fixed with non-null assertions or conditional-spread construction where the code's own logic already guarantees the value.datafield, since a partial cast ondataalone doesn't line up with the surrounding discriminant).Full test suite (
bun test ./src ./tests ./evals) passes: 5281 tests, 0 failures.bun run checkpasses in full.Noticed but not touched
Nothing else in scope needed suppression or a config relaxation.