System prompt budget: raise 24,000 to 35,000 chars, derived from the 98,304-token window, and restore the droppable reserve

[ozand]

Context

ContextBuilder (nanobot/agent/context.py) caps the self-evolving loop's executor system prompt at MAX_SYSTEM_PROMPT_CHARS = 24000 characters, with per-block caps in _RELEASE_BLOCK_CAPS and BOOTSTRAP_FILES: IDENTITY.md 1,500, SOUL.md 1,800, goals.md 3,200, USER.md 4,000, OPERATING.md 5,000, workspace AGENTS.md 4,000, plus a memory block (1,000) and a generated runtime block (400). Over the cap, the strict (loop) profile drops sections marked droppable and then raises SystemPromptOverflowError.

Measured on the host 2026-09-18, a representative cycle's phase: "system_prompt" ledger row:

{"chars": 21368, "cap": 24000, "rung": "full", "dropped": [], "missing": [],
 "sections": {"identity":1244,"soul":1590,"goals":3040,"user":1878,"operating":4335,
              "agents":3989,"skills_catalogue":4022,"memory":824,"runtime":390},
 "droppable_reserve_chars": 0, "truncated": ["AGENTS.md"]}

The prompt sits at 89% of the cap with zero droppable reserve, and one block is being truncated every cycle.

Problem

The cap is stated in characters and its value has no recorded derivation. At ~3.5 chars/token for this corpus, 24,000 chars is ≈6,100 tokens — while the serving model, un/qwen3.8-27b-gguf (llama-server on the operator's workstation, per T:/Code/localllm-kb), has a 98,304-token context with q8_0 KV and a LiteLLM route max_tokens of 32,768. The cap is not protecting a scarce resource at the value it holds; it is squeezing the one part of the prompt that is authored deliberately, while leaving no reserve for growth.

The naive correction — "make it 24,000 tokens" — is wrong for a different reason. The system prompt is a prefix resent on every turn of an agentic cycle that runs up to 80 tool iterations, so every character added to it is added to every later request in the cycle. Executor prompt_tokens over 1,867 calls (2026-09-16..18, state/llm_calls):

min	p50	p90	p99	max
7,099	33,080	62,154	71,672	79,804

(finish_reason: "length" on 19 of 1,867 calls, 1.0%.) The max already stands at 81% of the window. A 24,000-token system prompt would add ~18,000 tokens to every one of those figures and push the tail past the context window mid-cycle.

What to do

1. Raise MAX_SYSTEM_PROMPT_CHARS from 24,000 to 35,000 (≈10,000 tokens at the measured ratio). Derivation, to be written into the code as a comment beside the constant and kept with the number:
   • window 98,304 tokens;
   • observed max in-cycle prompt 79,804 tokens with the present ~6,100-token prefix;
   • a prefix grown to ~10,000 tokens moves that max to ≈83,700, leaving ≈14,600 tokens of headroom for completion and variance.
2. Raise the per-block caps that the deduplicated content needs: workspace AGENTS.md 4,000 → 8,000 and OPERATING.md 5,000 → 8,000. Leave the others unless the block is measurably being truncated.
3. Keep the unit of enforcement as characters — there is no tokenizer on the i386 host, and all existing fit arithmetic, telemetry and tests are in characters. State the budget in tokens in the comment, enforce in chars, and name the chars/token ratio the number was derived from.
4. Restore a real droppable reserve: after the raise, the assembled prompt should leave ≥3,000 chars of reserve so a growing block degrades instead of raising SystemPromptOverflowError.
5. Re-derive the ratio from production once system_prompt_chars is being written to llm_calls rows (that column exists and is currently null; it is populated by the pending role-prompt work). Until then the 3.5 chars/token figure is an estimate corroborated by the 7,099-token minimum observed at turn 1.

Order matters: this issue is to be done after the AGENTS.md deduplication, not before. Deduplication frees ~5,400 chars at no cost to the context window; raising the cap first buys window space for text that is about to be deleted.

Acceptance Criteria

• MAX_SYSTEM_PROMPT_CHARS is 35,000 and carries an in-code derivation naming the context window (98,304), the observed max in-cycle prompt, and the chars/token ratio used.
• Workspace AGENTS.md and OPERATING.md block caps are 8,000 each; every other block cap is unchanged or changed with its own stated reason.
• Over 50 consecutive cycles after deployment, ledger phase: "system_prompt" rows show rung: "full", truncated: [], dropped: [], and droppable_reserve_chars >= 3000.
• Over the same window, executor prompt_tokens p99 in state/llm_calls is ≤ 85,000 and finish_reason: "length" is not more frequent than the 1.0% baseline measured 2026-09-16..18.
• Existing overflow tests still pass, including the strict-profile SystemPromptOverflowError path; no test is relaxed to accommodate the new cap.
• Full pytest run passes.

Non-goals

• Introducing a tokenizer or token-based enforcement on the host.
• Changing the assembly order, the block list, or the degradation ladder.
• Changing the LiteLLM route's max_tokens.

[ozand]

Sequencing: after #1752 (deduplicate first, then buy window space) and after #1754 (the serving-layer ceiling the derivation is sized against). The chars/token ratio is re-derived from production once system_prompt_chars lands in llm_calls via #1729 / PR #1743.

[ozand]

Precondition #1754 is answered and closed; the ceiling arithmetic here needs one correction before this is built.

The issue sizes the raise against prompt + max_tokens where max_tokens is the route's 32,768. The executor route sets no max_tokens (model_info for un/qwen3.8-27b-gguf: litellm_params.max_tokens = None, max_input_tokens = 98304), so the client's value passes through — nanobot's schema default 8,192, confirmed empirically: over 2,876 executor calls the maximum completion_tokens is exactly 8,192 and not one call exceeds it.

Corrected ceiling, host eeepc, 2026-09-15..18:

prompt_tokens        max  79,804
completion           max   8,192  (the cap, exact)
prompt + completion  max  83,836
window                    98,304
headroom today            14,468 tokens

A prefix grown from ~6,100 to ~10,000 tokens adds ~3,900 to that max: ~87,700 used, ~10,600 headroom. The raise to 35,000 chars fits, and the derivation written beside the constant should say prompt + 8,192, not prompt + 32,768 — and should name the client-side default as the thing that makes it true, so that a later change to AgentDefaults.max_tokens is visibly a change to this ceiling.

One more input for the same comment block: AgentDefaults.context_window_tokens defaults to 65,536, which is not this model's window. Whatever this PR writes about the window must not read that field.

Order still holds: #1752 (deduplicate the instance file) lands before the per-block caps move, or raising AGENTS.md 4,000 → 8,000 simply un-truncates duplicated content.