Skip to content

Codex cost is finally windowed, so scan and report agree - #281

Merged
node9ai merged 1 commit into
mainfrom
dev
Sep 2, 2026
Merged

Codex cost is finally windowed, so scan and report agree#281
node9ai merged 1 commit into
mainfrom
dev

Conversation

@node9ai

@node9ai node9ai commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

node9 scan --days 30 counted every Codex session ever recorded. Measured on real data: 41 sessions / $13.69 from scan against 25 / $1.04 from node9 report --period 30d — same shared pricing function, 13x apart, because one windowed and the other did not.

Why it hid

scanCodexHistory takes startDate and used it exactly once, on a guard that runs only for response_item / function_call rows. That filters findings and loops. Token counts arrive as event_msg and never reach it, and total_token_usage is cumulative — so the cost added was always the session's whole lifetime, regardless of the window.

Now windowed at session level, matching report-audit.ts:658.

scan --days 30   $13.69 -> $1.04    (report says $1.04)
scan --all       $13.69             (unchanged)

All three agents now agree across both paths:

claude   $12,867.95   $12,867.95
codex         $1.04        $1.04
gemini        $0.00        $0.00

Deliberately NOT the ccusage delta approach

Converting cumulative usage into per-event deltas is the more correct model for a session straddling the window edge, and I proposed exactly that yesterday. Then I measured: zero of 41 sessions straddle. It would have solved a case this data does not contain while leaving the real defect — a filter that was never applied — untouched. Recorded with the evidence in doc/roadmap/active/cost-accuracy-and-plans.md for when a straddling session actually shows up.

A session with no session_meta timestamp is excluded once a window is set, matching report-audit.ts:656. It cannot be placed in time, and the point of this change is that the two paths agree.

Verification

Both binaries built and run back to back. Seven fields differ; exactly one is the fix:

byAgent[4] codex   $13.69 -> $1.04     <- the change
generatedAt, lastDate                  <- run stamps
totalToolCalls  24024 -> 24025         <- this session generating activity
byAgent[0] claude  +$3.39              <- same

Claude went UP $3.39 between runs while Codex went down $12.65, and -12.65 + 3.39 matches the total delta exactly — a code change does not raise an agent it never touched.

An earlier attempt at this comparison was worthless: git stash found nothing to stash after the commit, so it diffed today against a leftover file from yesterday and showed a $12,880 -> $509 "regression".

4,290 tests, typecheck, lint, format clean. Mutation testing killed removing the filter, killed a variant that breaks --all (the trap flagged before any code was written), and exposed that the no-timestamp test asserted only "did not crash" — every variant passed it. It now asserts behaviour.

🤖 Generated with Claude Code

…gree

`node9 scan --days 30` counted EVERY Codex session ever recorded. Measured
on real data: 41 sessions / $13.69 from scan against 25 / $1.04 from
`node9 report --period 30d` — same shared pricing function, 13x apart,
because one windowed and the other did not.

`scanCodexHistory` takes `startDate` and used it exactly once, on a guard
that runs only for `response_item` / `function_call` rows. That filters
findings and loops. Token counts arrive as `event_msg` and never reach it,
and `total_token_usage` is cumulative, so the cost added was always the
session's whole lifetime regardless of the window.

Now windowed at session level, matching report-audit.ts:658.

  scan --days 30   $13.69 -> $1.04   (report says $1.04)
  scan --all       $13.69            (unchanged)

DELIBERATELY NOT the ccusage per-event-delta approach. Converting
cumulative usage into deltas is the more correct model for a session
straddling the window edge, and I proposed it yesterday. Then I measured:
ZERO of 41 sessions straddle. It would have solved a case this data does
not contain while leaving the real defect — a filter that was never
applied — untouched. Recorded with the evidence in
doc/roadmap/active/cost-accuracy-and-plans.md for when a straddling
session actually appears.

A session with no `session_meta` timestamp is excluded once a window is
set, matching report-audit.ts:656. It cannot be placed in time and the
whole point here is that the two paths agree.

Mutation testing earned its keep twice. It killed removing the filter and
killed a variant that breaks `--all` — the trap I flagged before writing
any code, since the obvious filter also silences full-history scans. And
it exposed that my no-timestamp test asserted only "did not crash", which
every variant passed; it now asserts the behaviour, and a variant that
counts an unplaceable session turns it red.
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

🛡️ node9 agent-security · ✅

No agent-security findings — no injectable workflows, unsafe agent configs, or unpinned MCP servers.

@node9ai
node9ai merged commit fc848db into main Sep 2, 2026
14 checks passed
@node9ai

node9ai commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

🎉 This PR is included in version 2.7.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant