Repository navigation
Preserve Windows access benchmarks and verified MCP client evidence - #25
Merged
Merged
Conversation
- Add persistent C, Node, and Rust workers to compare all 21 supported Windows request paths with matched payloads and verified reads. - Separate in-memory keys from saved credentials, isolate the benchmark credential store, and measure CLI batches apart from single-process calls. - Record Windows loopback, separate-server, and no-install MCP results with raw latency data, artifact hashes, and reproduction steps. - Validate 138,000 timed operations and 504 final smoke operations on the Windows VM; pass just check and Python, Node, and C source checks.
- Explain local, direct MCP, and CLI performance in four sentences a developer can use in conversation. - Keep the precise native-versus-MCP figures beside the matched comparison and state the VM and timing limits.
- Exercise both Mac harnesses against the measured Windows Plasmite 1.0.0 artifact to verify remote access through real agent clients. - Record matching reads, waits, authorization reuse, expired-token renewal, and revocation with credential-free evidence. - Explain Claude's large-result file handling and the certificate and client-lifetime limits of the checks. - Preserve the existing transport timings so model connections and compatibility checks do not change the benchmark claims.
- Preserve the three original benchmark and harness evidence commits on current main. - Keep the README visuals and service work already merged into main.
- Revoke issued benchmark grants when MCP setup fails and surface failed revocation responses while closing the connection. - Cover failed authorization, handshake, notification, interruption, and revocation with socket-free regressions in just check. - Disclose the extra pool-open work included in MCP call measurements and require certificates trusted by each client. - Link the Windows results from the current README without changing existing prose or visuals.
- Collect cleanup errors while still closing every owned benchmark client, stopping the native helper, and disconnecting saved access. - Terminate an owned native helper that exceeds the shutdown timeout before reporting the cleanup failure. - Cover failed revocation and native shutdown timeout with socket-free tests. - Avoid Python bytecode artifacts in the required benchmark harness check.
sandover
marked this pull request as ready for review
October 2, 2026 21:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preserved Windows access measurements and verified Codex/Claude interoperability were missing from main. Add their report, recorded results, and reproduction harnesses, preserving the original three commits. Restore the report link in the README's performance section.
The report scopes results to the recorded Windows 11 Arm VM running x64 clients, distinguishes saved credentials from in-memory keys, and explains that MCP's extra pool-open request contributes to the measured interface cost. It also records the limits of headless client tests and the private certificate authority. Historical timings are retained; integration checks do not claim to reproduce Windows performance.
Fix benchmark authorization cleanup after initialization and revocation failures. Ten socket-free regression tests now run through
just checkand verify that teardown continues for other owned processes and saved access.Validation:
just release-gatepassed on the final tree, covering format, clippy, Rust tests, ten benchmark cleanup tests, ABI/conformance/bindings, UI, and package smokes.No release tag or publication is part of this PR. Historical timed harnesses differ from the current reproduction scripts; their recorded hashes and the report's claim boundaries remain explicit.