Skip to content

fix(telemetry): record the executor's system_prompt_chars; compress OPERATING.md for headroom (#1784, #1783) - #1798

Merged
ozand merged 1 commit into
mainfrom
spc-1784
Sep 19, 2026
Merged

ozand merged 1 commit into
mainfrom
spc-1784

Conversation

@ozand

@ozand ozand commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Closes #1784. Closes #1783.

Two small, independent fixes that share one file's worth of context.

#1784 — the executor was the one role not measured

executor rows with system_prompt_chars: null    797 of 797
proposer / reflector / curator / strategist      non-null

Those four pass the field at their own call sites. The executor reaches telemetry only through BaseProvider._record, which never passed it.

The sting: the executor is the role whose prompt budget was just raised from 24,000 to 35,000 (#1753), so the role the change was made for is the one with no measurement. And the number was not unavailable — the phase: "system_prompt" ledger row carried 21,477 for the same cycle the telemetry row recorded null.

system_chars moves from nanobot/runtime/role_prompt to nanobot/observability/llm_telemetry, beside the field it feeds. role_prompt re-exports it, so its four #1729 callers are untouched (a test asserts the two names are the same object).

The move is the point. providers/base importing a runtime module inverts the dependency direction — that is the import closure that turned test_trainer_no_direct_mutation red in #1743, and I would rather not rediscover it.

Additive and unread: no dashboard panel, scorecard metric or guard consumes the field, so this cannot change behaviour.

#1783 — compress, do not raise a cap

OPERATING.md sat at 4,921 of 5,000 — 79 spare — and #1770 must move two rules from SOUL.md into it (+153 chars), putting it 74 over. The issue asks which of three budget levels to raise.

Measured before choosing:

blocks ever truncated in the live 7-day ledger AGENTS.md only, 53 times, resolved by #1752
assembled prompt 20,837
sum of block caps 21,500
ceiling (#1753) 35,000

The caps are barely load-bearing and the ceiling has 14,000 spare. Raising a cap buys nothing and spreads a constant across the tests that pin it.

So: raise nothing, compress. ## Iteration budget loses 94 chars of restatement and keeps all four of its rules — where the number is stated, pace early, verify on a candidate, keep reserve for the commit and the final response. OPERATING.md is now 4,827 with 173 spare, enough for #1770 with margin.

A test now asserts the file keeps ≥153 chars of headroom, so #1770 cannot discover mid-PR that its migration does not fit — which is precisely what #1783 was filed about.

The trap this hit

My first compression paraphrased "tool iterations" into "tool-iteration". ADR-022 rule 4 fingerprints this block on that literal phrase, so the rule momentarily had no owner at all — caught by test_fingerprint_once_per_system_block, exactly as designed.

The phrase is restored, and now pinned in test_operating_md.py too, at the place a future edit actually happens rather than only in the harness one directory away.

Tests

tests/test_system_prompt_chars_executor.py (7): measures the system message; first system message only; returns None rather than 0 when there is nothing to measure, and 0 when the system message really is empty — the no-data-versus-zero distinction this project keeps paying for; tolerates malformed entries; the role_prompt re-export is the same object; and the regression itself — chat_with_retry records the value, with a fake provider and a captured record_llm_call.

tests/test_operating_md.py (+2): the four Iteration-budget rules plus the fingerprint phrase are pinned individually, and the headroom assertion above.

Green: test_llm_telemetry, test_served_model_telemetry, test_model_window, test_role_prompt*, test_operating_md, the new file (121), plus the whole prompt/context/trainer/import selection (540 passed, 1 xfailed).

…PERATING.md for headroom (#1784, #1783)

## #1784

Measured on host eeepc: 797 of 797 executor rows in `state/llm_calls`
carried `system_prompt_chars: null`, while the proposer, reflector, curator
and strategist all carried real numbers. Those four pass the field at their
own call sites; the executor reaches telemetry only through
`BaseProvider._record`, which never passed it.

So the one role whose prompt budget was just raised from 24,000 to 35,000
(#1753) is the one role not measured. The real figure exists in the
`phase: "system_prompt"` ledger row (21,477 at measurement time), so the
number was reachable in one artifact and null in another for the same cycle.

`system_chars` moves from `nanobot/runtime/role_prompt` to
`nanobot/observability/llm_telemetry`, beside the field it feeds, and
`role_prompt` re-exports it so its four existing callers are untouched. The
move is the point: `providers/base` importing a runtime module would invert
the dependency direction -- the same import closure that turned
`test_trainer_no_direct_mutation` red in #1743.

Additive and unread: no dashboard panel, scorecard metric or guard consumes
`system_prompt_chars`, so this cannot change behaviour anywhere.

## #1783

`OPERATING.md` was 4,921 of its 5,000-char cap -- 79 spare -- and #1770 must
move two rules out of `SOUL.md` into it (+153 chars), which would have put it
74 over. Three levels could have absorbed that: the block cap, the sum of
caps, or the prompt ceiling.

Measured before choosing: only `AGENTS.md` has ever been truncated in the
live ledger (53 times, already resolved by #1752); the assembled prompt is
20,837 against a 35,000 ceiling; and the per-block caps are barely
load-bearing. Raising one would buy nothing and would spread a constant
across the tests that pin it.

So: raise nothing, compress. `## Iteration budget` loses 94 chars of
restatement while keeping all four of its rules -- where the number is
stated, pace early, verify on a candidate, keep reserve for the commit and
the final response. `OPERATING.md` is now 4,827 chars with 173 spare, enough
for #1770 with margin.

One trap found by the ontology harness: the first compression paraphrased
"tool iterations" into "tool-iteration", and ADR-022 rule 4 fingerprints this
block on that literal phrase -- so the rule momentarily had no owner at all.
The phrase is restored and now pinned in `test_operating_md.py` as well, at
the place a future edit will happen rather than only in the harness.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ozand
ozand merged commit aad6435 into main Sep 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant