Skip to content

Stop leaking exception text; make conversation identity and terminal persistence safe - #13

Merged
HartBrook merged 2 commits into
mainfrom
feature/stream-safety-and-conversation-identity
Aug 10, 2026
Merged

HartBrook merged 2 commits into
mainfrom
feature/stream-safety-and-conversation-identity

Conversation

@HartBrook

Copy link
Copy Markdown
Owner

Addresses three P0 defects in the streaming path. Each is reachable by an ordinary client, and each is covered by tests that fail against main.

1. Raw exception text reached the browser

classify_exception() attached details={"original_error": str(exc)} to every classified error, and that was serialized straight into the StreamChunk. Provider exceptions routinely embed prompt content, tool output, response bodies, or credentials in a URL.

Demonstrated on main:

create_error_chunk(RuntimeError(f"429 rate limit for {SECRET}")).model_dump_json()
# -> secret present: True

Classification still reads the raw text, but nothing derived from it escapes beyond a numeric retry-after hint. The client now receives only a stable code, a canned message, retryability, and the exception class name (a type identifier, not content):

{"code":"rate_limit","message":"The AI service is temporarily busy...",
 "is_retryable":true,"retry_after_seconds":30,
 "details":{"exception_type":"RuntimeError"}}

The original exception is logged with its traceback via the standard logging module (llmpane.errors), so operators lose nothing. set_expose_raw_errors(True) or LLMPANE_EXPOSE_RAW_ERRORS=1 restores the old payload for local debugging, documented as development-only.

Classification behaviour is unchanged — all ten semantic codes are covered by parameterized tests.

2. Conversation identity was caller-controlled

An unknown conversation_id was passed straight to create_conversation(conversation_id), so a client could choose the key its own data was stored under.

An unknown or absent ID now always starts a fresh server-generated conversation. Callers read the authoritative ID back from the terminal chunk's conversation_id. Also documented that a ConversationStore is persistence, not authorization — it does not check who may read a conversation.

3. The terminal chunk disagreed with storage

Two separate bugs:

  • The terminal carried a message_id minted in the adapter, while add_message independently assigned a different one. Clients were handed an ID that did not exist server-side.
  • The terminal was emitted before persistence ran, so a storage failure still reported success.

The terminal is now held back until persistence succeeds, and the single allocated ID is what gets both stored and reported. A persistence failure yields a classified error instead of a success terminal.

Cancellation now has a defined outcome rather than an accidental one: the user message is already durable, and partial assistant text is discarded rather than silently stored.

Verification

  • 233 Python tests pass (was 202); ruff, format, and mypy --strict clean.
  • The new tests were confirmed to fail against main. Reverting session.py alone fails exactly the four behavioural tests: unknown-ID persistence, terminal/persisted ID equality, and both persistence-failure cases. The redaction leak was demonstrated directly before and after.
  • Validated against a production consumer without modifying it: its full llmpane-dependent suite passes unchanged.

That downstream check produced a useful signal. The consumer had already built this exact conversation-ID policy by hand — its router comments read "Only the server creates conversation IDs. A supplied ID must already exist" — and layered its own ownership check on top. It had to, because the framework default was unsafe. That guard is now defence-in-depth rather than load-bearing. The consumer also forwards chunk.message_id to its client, which under this change becomes a real persisted ID instead of a phantom.

Compatibility

No API signatures change. Behavioural changes worth noting for anyone upgrading:

  • error_info.details.original_error is gone by default. If you were surfacing it in a UI, switch to error_info.code and error_info.message, or opt back in for local debugging.
  • An unknown conversation_id no longer round-trips; read conversation_id from the terminal chunk instead of assuming yours was used.
  • message_id on the terminal is now the persisted ID, so previously it could not be used to look a message up. Now it can.

The React-side follow-ups (single terminal SSE outcome, ModelUsage protocol parity) build on this wire contract and follow in a second PR.

Three related defects in the streaming path, all reachable by an ordinary
client.

Raw exception text reached the browser. classify_exception attached
details={"original_error": str(exc)} to every classified error, and that
was serialized into the StreamChunk. Provider exceptions routinely embed
prompt content, tool output, response bodies, or credentials in a URL.

Classification now reads the raw text but nothing derived from it escapes
beyond a numeric retry-after hint. The original exception is logged with a
traceback through the standard logging module, so operators keep full
diagnostics while clients receive a stable code, a canned message,
retryability, and the exception class name. LLMPANE_EXPOSE_RAW_ERRORS=1 or
set_expose_raw_errors(True) restores the old payload for local debugging.

Conversation identity was caller-controlled. An unknown conversation_id
was passed straight to create_conversation, so a client could choose the
key its data was stored under. An unknown or absent ID now always starts a
fresh server-generated conversation; callers read the real ID back from
the terminal chunk. Documented that a ConversationStore is persistence,
not authorization.

The terminal chunk disagreed with storage. It carried a message ID minted
in the adapter while add_message assigned a different one, so clients held
an ID that did not exist server-side. The terminal was also emitted before
persistence ran, so a storage failure still reported success.

The terminal is now held back until persistence succeeds, and the single
allocated ID is what gets both stored and reported. A persistence failure
yields a classified error instead. Cancellation has a defined outcome: the
user message is already durable, and partial assistant text is discarded
rather than silently stored.
@HartBrook
HartBrook merged commit 6937447 into main Aug 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant