Skip to content

fix(pi): pass model samplingParams into the isolated models.json - #193

Merged
drewstone merged 1 commit into
mainfrom
fix/pi-pass-sampling-params
Sep 2, 2026
Merged

fix(pi): pass model samplingParams into the isolated models.json#193
drewstone merged 1 commit into
mainfrom
fix/pi-pass-sampling-params

Conversation

@drewstone

Copy link
Copy Markdown
Owner

Defect

src/backends/pi-inference-transport.ts copies only SAFE_MODEL_FIELDS from the operator's ~/.pi/agent/models.json model entry into the isolated models.json each pi run reads.
That allowlist dropped samplingParams.
Pi merges a model's samplingParams object into every request body (Object.assign(params, options.samplingParams) in pi's openai-completions transport; Model.samplingParams?: Record<string, unknown> in pi's types).
It is the only way to send body-level gateway options such as {"gateway": {"streamIdleTimeout": 180000}} to router.tangle.tools, because the router reads gateway options from the body only.

Without it, a Claude model that pauses longer than the router's 30 s stream-idle default has its stream closed by the router ("sent nothing for 30s and the router closed the stream").
Pi retries, and the bridge's accounting check (src/backends/pi.ts accountingMatched requires failedRequests === 0) then refuses the whole session.

Measurement

2026-09-02, fleet host running cli-bridge 0.3.0 (origin/main 0299595): 23 of 25 single-inference workers refused in one run, 2 stream-idle closures per refused session.

Change

  • Add samplingParams to SAFE_MODEL_FIELDS. Body-level request parameters only; auth never rides here.
  • Factor the two identical allowlist copy loops (explicit models.json path and pi catalog path) into copySafeModelFields, which rejects a samplingParams value that is not a plain object with the file's existing BackendError('not_configured') style. Pi spreads the value into the request body, so an array or scalar would produce a malformed upstream request.

Test

tests/pi-inference-isolation.test.ts:

  • carries model samplingParams into the isolated models.json: a model entry with samplingParams: { gateway: { streamIdleTimeout: 180000 } } survives into resolved.modelConfig and into the models.json written by provisionPiInferenceTransport.
  • rejects model samplingParams that are not a plain object: an array value is refused before pi auth is invoked.

Local results (macOS): the two new tests pass; pnpm build (tsc --noEmit) exits 0.
The same file has 2 pre-existing failures on pristine origin/main on macOS (/proc/self/environ missing; /tmp realpath prefix), unrelated to this change.

Note

The fleet host applied the same one-line allowlist change locally on 2026-09-02, pending this PR.

The bridge copies only allowlisted fields from the operator's models.json
model entry into the isolated models.json each pi run reads. It dropped
`samplingParams`, the object pi merges into every request body
(`Object.assign(params, options.samplingParams)` in pi's openai-completions
transport). That object is the only way to send body-level gateway options
such as `{"gateway": {"streamIdleTimeout": 180000}}` to router.tangle.tools,
because the router reads gateway options from the body only.

Without it, a Claude model that pauses longer than the router's 30 s
stream-idle default has its stream closed by the router. Pi retries, and the
bridge's accounting check (`accountingMatched` requires
`failedRequests === 0`) then refuses the whole session.

Measured on 2026-09-02 on a fleet host running cli-bridge 0.3.0
(origin/main 0299595): 23 of 25 single-inference workers refused in one run,
with 2 stream-idle closures per refused session.

This change adds `samplingParams` to SAFE_MODEL_FIELDS and validates it is a
plain object at both copy sites. Pi spreads the value into the request body,
so an array or scalar would produce a malformed upstream request.
@drewstone
drewstone merged commit 18f2a42 into main Sep 2, 2026
1 of 2 checks passed
@drewstone
drewstone deleted the fix/pi-pass-sampling-params branch September 2, 2026 05:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant