fix(pi): pass model samplingParams into the isolated models.json - #193
Merged
Conversation
The bridge copies only allowlisted fields from the operator's models.json
model entry into the isolated models.json each pi run reads. It dropped
`samplingParams`, the object pi merges into every request body
(`Object.assign(params, options.samplingParams)` in pi's openai-completions
transport). That object is the only way to send body-level gateway options
such as `{"gateway": {"streamIdleTimeout": 180000}}` to router.tangle.tools,
because the router reads gateway options from the body only.
Without it, a Claude model that pauses longer than the router's 30 s
stream-idle default has its stream closed by the router. Pi retries, and the
bridge's accounting check (`accountingMatched` requires
`failedRequests === 0`) then refuses the whole session.
Measured on 2026-09-02 on a fleet host running cli-bridge 0.3.0
(origin/main 0299595): 23 of 25 single-inference workers refused in one run,
with 2 stream-idle closures per refused session.
This change adds `samplingParams` to SAFE_MODEL_FIELDS and validates it is a
plain object at both copy sites. Pi spreads the value into the request body,
so an array or scalar would produce a malformed upstream request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Defect
src/backends/pi-inference-transport.tscopies onlySAFE_MODEL_FIELDSfrom the operator's~/.pi/agent/models.jsonmodel entry into the isolatedmodels.jsoneach pi run reads.That allowlist dropped
samplingParams.Pi merges a model's
samplingParamsobject into every request body (Object.assign(params, options.samplingParams)in pi'sopenai-completionstransport;Model.samplingParams?: Record<string, unknown>in pi's types).It is the only way to send body-level gateway options such as
{"gateway": {"streamIdleTimeout": 180000}}to router.tangle.tools, because the router reads gateway options from the body only.Without it, a Claude model that pauses longer than the router's 30 s stream-idle default has its stream closed by the router ("sent nothing for 30s and the router closed the stream").
Pi retries, and the bridge's accounting check (
src/backends/pi.tsaccountingMatchedrequiresfailedRequests === 0) then refuses the whole session.Measurement
2026-09-02, fleet host running cli-bridge 0.3.0 (origin/main 0299595): 23 of 25 single-inference workers refused in one run, 2 stream-idle closures per refused session.
Change
samplingParamstoSAFE_MODEL_FIELDS. Body-level request parameters only; auth never rides here.models.jsonpath and pi catalog path) intocopySafeModelFields, which rejects asamplingParamsvalue that is not a plain object with the file's existingBackendError('not_configured')style. Pi spreads the value into the request body, so an array or scalar would produce a malformed upstream request.Test
tests/pi-inference-isolation.test.ts:carries model samplingParams into the isolated models.json: a model entry withsamplingParams: { gateway: { streamIdleTimeout: 180000 } }survives intoresolved.modelConfigand into themodels.jsonwritten byprovisionPiInferenceTransport.rejects model samplingParams that are not a plain object: an array value is refused before pi auth is invoked.Local results (macOS): the two new tests pass;
pnpm build(tsc --noEmit) exits 0.The same file has 2 pre-existing failures on pristine
origin/mainon macOS (/proc/self/environmissing;/tmprealpath prefix), unrelated to this change.Note
The fleet host applied the same one-line allowlist change locally on 2026-09-02, pending this PR.