Skip to content

Bring test files and fixtures into typecheck scope - #578

Merged
TheGreatAxios merged 2 commits into
mainfrom
cl-5759-test-files-are-outside-the-typecheck-scope-so-deleted-fields
Aug 23, 2026
Merged

Bring test files and fixtures into typecheck scope#578
TheGreatAxios merged 2 commits into
mainfrom
cl-5759-test-files-are-outside-the-typecheck-scope-so-deleted-fields

Conversation

@TheGreatAxios

@TheGreatAxios TheGreatAxios commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

Closes CL-5759.

Verified the gap was real

tsconfig's include was src/**/*.ts and packages/**/*.ts only — tests/, evals/, and scripts/ (159 test files) were outside it. Confirmed empirically: introduced a type error in src/state.test.ts (caught by tsc --noEmit) and the same kind of error in tests/unit/plugin-repo-locator.test.ts (not caught), then reverted both.

Fix

Widened include to add tests/**/*.ts, evals/**/*.ts, scripts/**/*.ts. Excluded, by name, only the capability eval suite's hermetic sandbox repos under tests/fixtures/** and all of evals/capability/cases/** — these are copied into per-run eval workdirs rather than imported by our code, own their own package.json, and tests/fixtures/buggy-service deliberately ships a bug as fixture content for an eval case.

Two fixture directories are deliberately not in that exclusion list, even though they live under tests/fixtures/: tests/fixtures/crash-run and tests/fixtures/plugins/implement-feature. Both import real production modules by relative path (src/index.ts, src/session/active-run.ts, src/session/state.ts, src/tui/runner.ts, src/workflows/definition.ts, src/tui/commands/registry.ts) and tests/integration/crash-finalize.test.ts spawns crash-run's scripts as live exercises of that code — excluding them would recreate exactly the silent-drift gap this PR closes. Typechecking implement-feature surfaced a real instance of that drift: its relative import paths were two directories short (stale since the fixture was last touched), so import type { CommandPlugin } from "../../../src/tui/commands/registry.js" resolved nowhere. Fixed the path depth; the fixture isn't wired into any test today, so this was invisible until now.

bun run typecheck (tsc --noEmit) is unchanged, so the wider scope runs in the same command contributors already use.

Errors surfaced

116 pre-existing errors across ~28 files, plus the two broken import paths above. All fixed properly, none suppressed:

  • Real field/signature drift caught by the widened scope: LastCycleSource.idsourceId, AgentProfile.promptsystemPromptRole, ApprovalScope gaining required id/label, StringToolHandler now requiring an AbortSignal argument, InboundMessage's actual wire shape (not the informal {role, content} a test assumed), ProviderSettings.baseURL becoming required, PricingFetcherOptions.fetchImpl over-typed as typeof fetch (which also demands static members like preconnect that no test double implements, narrowed to a plain call signature), and the two stale relative-import paths in tests/fixtures/plugins/implement-feature described above.
  • Routine noUncheckedIndexedAccess/exactOptionalPropertyTypes fallout in test helpers (array-index access, optional-field passthrough), fixed with non-null assertions or conditional-spread construction where the code's own logic already guarantees the value.
  • Narrow, commented casts only where a test deliberately needs a loose shape (e.g. casting a whole mock event object rather than just its data field, since a partial cast on data alone doesn't line up with the surrounding discriminant).

Full test suite (bun test ./src ./tests ./evals) passes: 5281 tests, 0 failures. bun run check passes in full.

Noticed but not touched

Nothing else in scope needed suppression or a config relaxation.

tests/, evals/, and scripts/ were outside tsconfig's include, so a
field rename or deletion left stale references in tests unnoticed
until runtime. Widen include to cover them, excluding only the
hermetic capability-eval sandbox repos (tests/fixtures/**,
evals/capability/cases/**), which are copied into per-run workdirs
rather than imported and in one case deliberately contain invalid
syntax as fixture content.

Fixes the batch of pre-existing errors the widened scope surfaced:
real field/signature drift (LastCycleSource.id -> sourceId,
AgentProfile.prompt -> systemPromptRole, ApprovalScope gaining
id/label, StringToolHandler requiring an AbortSignal, InboundMessage's
actual wire shape, ProviderSettings.baseURL becoming required,
PricingFetcherOptions.fetchImpl over-typed as `typeof fetch`) plus
routine noUncheckedIndexedAccess/exactOptionalPropertyTypes fallout in
test helpers, narrowly cast where a test needs a loose shape.
@linear-code

linear-code Bot commented Aug 23, 2026

Copy link
Copy Markdown

CL-5759

- Run prettier on the 7 files CI flagged.
- Narrow the tests/fixtures/** typecheck exclusion so it no longer covers
  tests/fixtures/crash-run or tests/fixtures/plugins/implement-feature —
  both import real production modules by relative path and crash-run is
  spawned by tests/integration/crash-finalize.test.ts as a live exercise
  of that code. Excluding them would have recreated the gap this PR
  closes. List the remaining hermetic eval fixtures individually instead
  of a blanket glob.
- Typechecking implement-feature surfaced exactly the kind of drift this
  PR targets: its relative imports were two directories short (stale),
  so CommandPlugin/WorkflowPlugin resolved to nothing. Fixed the path
  depth. The fixture isn't wired into any test today, so this was
  invisible until now.
@TheGreatAxios
TheGreatAxios merged commit 6ea4085 into main Aug 23, 2026
5 checks passed
TheGreatAxios added a commit that referenced this pull request Aug 23, 2026
Merging this branch with main exposed two typecheck failures that the
branch alone could not show, because it never touches tsconfig.json and
PR #578 widened typecheck scope to tests/ and evals/ after this branch
was cut.

parseCaseJson narrowed raw.tier with a `tier as EvalTier` cast, which is
not a type guard, so it returned string where the EvalCase field is
EvalTier. Replaced with a real isEvalTier predicate.

The four tier-* sandbox fixtures now fall inside typecheck scope. They
have their own package.json and tier-hard is intentionally buggy, so
they join the existing eval-sandbox exclusions. crash-run and
plugins/implement-feature stay in scope deliberately — they import
production modules and #578 carved them out for that reason.

Also swept the exclusion entries for fixtures this branch deletes.
TheGreatAxios added a commit that referenced this pull request Aug 27, 2026
…e-the-typecheck-scope-so-deleted-fields

Bring test files and fixtures into typecheck scope
TheGreatAxios added a commit that referenced this pull request Aug 27, 2026
Merging this branch with main exposed two typecheck failures that the
branch alone could not show, because it never touches tsconfig.json and
PR #578 widened typecheck scope to tests/ and evals/ after this branch
was cut.

parseCaseJson narrowed raw.tier with a `tier as EvalTier` cast, which is
not a type guard, so it returned string where the EvalCase field is
EvalTier. Replaced with a real isEvalTier predicate.

The four tier-* sandbox fixtures now fall inside typecheck scope. They
have their own package.json and tier-hard is intentionally buggy, so
they join the existing eval-sandbox exclusions. crash-run and
plugins/implement-feature stay in scope deliberately — they import
production modules and #578 carved them out for that reason.

Also swept the exclusion entries for fixtures this branch deletes.
TheGreatAxios added a commit that referenced this pull request Aug 27, 2026
…e-the-typecheck-scope-so-deleted-fields

Bring test files and fixtures into typecheck scope
TheGreatAxios added a commit that referenced this pull request Aug 27, 2026
Merging this branch with main exposed two typecheck failures that the
branch alone could not show, because it never touches tsconfig.json and
PR #578 widened typecheck scope to tests/ and evals/ after this branch
was cut.

parseCaseJson narrowed raw.tier with a `tier as EvalTier` cast, which is
not a type guard, so it returned string where the EvalCase field is
EvalTier. Replaced with a real isEvalTier predicate.

The four tier-* sandbox fixtures now fall inside typecheck scope. They
have their own package.json and tier-hard is intentionally buggy, so
they join the existing eval-sandbox exclusions. crash-run and
plugins/implement-feature stay in scope deliberately — they import
production modules and #578 carved them out for that reason.

Also swept the exclusion entries for fixtures this branch deletes.
@TheGreatAxios
TheGreatAxios deleted the cl-5759-test-files-are-outside-the-typecheck-scope-so-deleted-fields branch August 27, 2026 23:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant