Stop leaking exception text; make conversation identity and terminal persistence safe - #13
Merged
Conversation
Three related defects in the streaming path, all reachable by an ordinary
client.
Raw exception text reached the browser. classify_exception attached
details={"original_error": str(exc)} to every classified error, and that
was serialized into the StreamChunk. Provider exceptions routinely embed
prompt content, tool output, response bodies, or credentials in a URL.
Classification now reads the raw text but nothing derived from it escapes
beyond a numeric retry-after hint. The original exception is logged with a
traceback through the standard logging module, so operators keep full
diagnostics while clients receive a stable code, a canned message,
retryability, and the exception class name. LLMPANE_EXPOSE_RAW_ERRORS=1 or
set_expose_raw_errors(True) restores the old payload for local debugging.
Conversation identity was caller-controlled. An unknown conversation_id
was passed straight to create_conversation, so a client could choose the
key its data was stored under. An unknown or absent ID now always starts a
fresh server-generated conversation; callers read the real ID back from
the terminal chunk. Documented that a ConversationStore is persistence,
not authorization.
The terminal chunk disagreed with storage. It carried a message ID minted
in the adapter while add_message assigned a different one, so clients held
an ID that did not exist server-side. The terminal was also emitted before
persistence ran, so a storage failure still reported success.
The terminal is now held back until persistence succeeds, and the single
allocated ID is what gets both stored and reported. A persistence failure
yields a classified error instead. Cancellation has a defined outcome: the
user message is already durable, and partial assistant text is discarded
rather than silently stored.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses three P0 defects in the streaming path. Each is reachable by an ordinary client, and each is covered by tests that fail against
main.1. Raw exception text reached the browser
classify_exception()attacheddetails={"original_error": str(exc)}to every classified error, and that was serialized straight into theStreamChunk. Provider exceptions routinely embed prompt content, tool output, response bodies, or credentials in a URL.Demonstrated on
main:Classification still reads the raw text, but nothing derived from it escapes beyond a numeric retry-after hint. The client now receives only a stable code, a canned message, retryability, and the exception class name (a type identifier, not content):
{"code":"rate_limit","message":"The AI service is temporarily busy...", "is_retryable":true,"retry_after_seconds":30, "details":{"exception_type":"RuntimeError"}}The original exception is logged with its traceback via the standard
loggingmodule (llmpane.errors), so operators lose nothing.set_expose_raw_errors(True)orLLMPANE_EXPOSE_RAW_ERRORS=1restores the old payload for local debugging, documented as development-only.Classification behaviour is unchanged — all ten semantic codes are covered by parameterized tests.
2. Conversation identity was caller-controlled
An unknown
conversation_idwas passed straight tocreate_conversation(conversation_id), so a client could choose the key its own data was stored under.An unknown or absent ID now always starts a fresh server-generated conversation. Callers read the authoritative ID back from the terminal chunk's
conversation_id. Also documented that aConversationStoreis persistence, not authorization — it does not check who may read a conversation.3. The terminal chunk disagreed with storage
Two separate bugs:
message_idminted in the adapter, whileadd_messageindependently assigned a different one. Clients were handed an ID that did not exist server-side.The terminal is now held back until persistence succeeds, and the single allocated ID is what gets both stored and reported. A persistence failure yields a classified error instead of a success terminal.
Cancellation now has a defined outcome rather than an accidental one: the user message is already durable, and partial assistant text is discarded rather than silently stored.
Verification
mypy --strictclean.main. Revertingsession.pyalone fails exactly the four behavioural tests: unknown-ID persistence, terminal/persisted ID equality, and both persistence-failure cases. The redaction leak was demonstrated directly before and after.That downstream check produced a useful signal. The consumer had already built this exact conversation-ID policy by hand — its router comments read "Only the server creates conversation IDs. A supplied ID must already exist" — and layered its own ownership check on top. It had to, because the framework default was unsafe. That guard is now defence-in-depth rather than load-bearing. The consumer also forwards
chunk.message_idto its client, which under this change becomes a real persisted ID instead of a phantom.Compatibility
No API signatures change. Behavioural changes worth noting for anyone upgrading:
error_info.details.original_erroris gone by default. If you were surfacing it in a UI, switch toerror_info.codeanderror_info.message, or opt back in for local debugging.conversation_idno longer round-trips; readconversation_idfrom the terminal chunk instead of assuming yours was used.message_idon the terminal is now the persisted ID, so previously it could not be used to look a message up. Now it can.The React-side follow-ups (single terminal SSE outcome,
ModelUsageprotocol parity) build on this wire contract and follow in a second PR.