Skip to content

agmsgd beta: opt-in Codex delivery daemon and the agmsg command - #1529

Open
fujibee wants to merge 22 commits into
mainfrom
integration/agmsgd-beta
Open

fujibee wants to merge 22 commits into
mainfrom
integration/agmsgd-beta

Conversation

@fujibee

@fujibee fujibee commented Sep 30, 2026 •

Copy link
Copy Markdown
Owner

Summary

agmsgd beta for 1.6.0: an opt-in, per-machine resident process that delivers new-message notices to Codex CLI sessions through codex queue, plus the one agmsg command that controls it. Nothing changes for users who do not enable it.

  • Opt-in daemon: agmsg daemon start | stop | status | enable | disable. enable registers a resident unit (launchd on macOS, systemd --user on Linux, Task Scheduler on Windows); disable removes it and prints how to move each Codex seat back to the bridge.
  • Codex delivery: while the daemon is enabled, the Codex shim passes straight through (no app-server, dispatcher or bridge); the daemon notices new messages for each Codex seat and writes one queue item per seat per CODEX_HOME, confirming it by the queue row or by a per-notice nonce in the session rollout. Unreadable or unknown state keeps the notice pending instead of resending. Seats already running on the bridge stay on it until they restart. Teams on the JSONL storage driver are not served and are reported as such.
  • Health notices: when the daemon is enabled but not running, agmsg operations (rate-limited to once per ten minutes), the Codex shim, status and doctor say so.
  • One agmsg command: the npm entry runs install itself and hands every other verb to the installed bash runtime. Install places a marked launcher in a fixed candidate directory (~/.local/bin; ~/bin first under Git Bash on Windows) and never edits shell rc files; when that directory is not on PATH it prints the line to add. In 1.6.0 only daemon is public; other verbs are reserved.
  • Install safety: install.sh and uninstall.sh take a per-install operation lock and publish a completion manifest (file digests, generation, install id, bootstrap version) only after every write. An interrupted operation leaves an incomplete-operation record that blocks later operations (and daemon start) until it is cancelled cleanly or recovered explicitly with the printed --recover <id> command.
  • Codex profile per seat: each Codex seat records its effective CODEX_HOME, so notices go to the right profile.

Known limits

  • Linux: the systemd --user registration path is covered by tests with a stand-in, not yet run on a real Linux desktop session.
  • Windows: queue delivery is enabled only for the measured Codex release (0.157.0, native codex.exe resolved on PATH or under the npm vendor path); any other or unreadable version stays blocked with the reason shown in status. Delivery into a live session and Task Scheduler registration were verified on Windows.
  • The nonce in the session rollout was verified on Windows (a role: user message item); on macOS the same rollout shape is assumed, and an unreadable or unknown rollout keeps the notice pending.
  • When a 1.5.1-or-earlier global npm agmsg is ahead on PATH, it still receives agmsg daemon; install prints a note, and npm i -g agmsg@latest updates it.
  • The once-per-ten-minutes health notice can print twice when two operations straddle a window boundary.

Test plan

  • Unit and integration bats/node tests for the daemon, lock, manifest, queue channel, switching and launcher; the full CI matrix passes on the integration branch, including Windows install helpers.
  • macOS resident registration by hand (registered and removed); Windows Task Scheduler XML registration by hand (registered, queried, removed, no admin rights).

* Record the effective profile path with each seat

* Use selected profile for rollout discovery

* Normalize profile paths in resume tests
…s, daemon.sh CLI) (#1505)

## Scope

agmsgd beta skeleton: install.db schema (meta, daemon_owner, daemon_intent, daemon_start_attempts), executor liveness (pid + boot id), the daemon_owner CAS lifecycle, completion-record verification (install_id + digest against run/install-manifest.json), per-cycle drift watch (moved manifest / VERSION bump / in-place edit under scripts/), the control socket (stop/status only), log rotation, status.mjs (the owner/intent decision table, narrowed to the 3 beta-owned tables), main.mjs, the fixed bootstrap pair (agmsgd-launch.sh + agmsgd), and the top-level scripts/daemon.sh CLI (start/stop/status/enable/disable; resident registration for launchd/systemd --user/Task Scheduler).

Also extracted the #963 detector (collectInstallBaseline / installChangedAgainst) out of scripts/internal/remote-sync.mjs into scripts/internal/install-baseline.mjs, with remote-sync.mjs re-exporting both names unchanged; tests/remote_sync_engine.test.mjs passes without modification (115/115).

All plain .mjs, no build step, no external packages.

## Stop sequence verification

The focused stop-sequence test passed 20 consecutive runs without reproducing the intermittent hang after closing inherited file descriptors in both the lock-holding sqlite3 child and the launcher child, and removing the duplicate sqlite3 .quit write. These are candidate causes; the root cause is not established.

## Spawn environment

CODEX_HOME is intentionally inherited by spawned Codex seats because codex-record-session.sh records the active profile for resume, and the spawned Codex must use that same profile.

## Tested

- .github/scripts/check-enforced-assertions.sh: 621 unenforceable assertions, at the existing baseline; passed.
- Focused assertions in tests/test_agmsgd_daemon_sh.bats: 4/4 passed; tests/test_agmsgd_entrypoint.bats: 1/1 passed; tests/test_agmsgd_launch.bats: 1/1 passed.
- The stop-sequence test in tests/test_agmsgd_daemon_sh.bats passed 20 consecutive runs.
- launchd/systemd/schtasks commands are overridable (AGMSGD_LAUNCHCTL/AGMSGD_SYSTEMCTL/AGMSGD_SCHTASKS); automated tests never call the real resident-manager binaries, only a fake stand-in. Real registration is exercised by hand only, and always torn down right after.

## Not yet verified

- Linux (systemd --user) and Windows (Task Scheduler / schtasks XML) resident-registration code paths are written and shellchecked but not yet run on real Linux/Windows hardware.
…#1506)

## Summary

The installer and uninstaller use one per-install operation lock. Before the first protected write, each operation atomically publishes run/install-op-incomplete.json with an operation ID, operation kind and mode, target install path, actor PID, and start time. Later install and uninstall operations refuse to proceed while that record remains; the daemon entrypoint checks for it under the same exclusive lock before importing daemon code and exits with status 75 when it is present.

A refusal prints the exact recovery command for the selected install and describes the recorded operation separately from the requested next operation. Before using it, an operator must verify that no writer from the recorded operation is still running; the command displays the recorded PID but does not infer liveness. Recovery removes only the matching operation ID under the lock, then continues the requested install or uninstall.

Successful install publishes the completion manifest before removing its matching incomplete-operation record. A caught INT or TERM stops and reaps the tracked copy or removal command before clearing the record when the lock is still held; otherwise the record stays in place. Cancellation during the interval between writer spawn and PID publication also keeps the record, so later operations remain blocked until explicit recovery. A hard process death leaves the record as well.

A small recovery helper is atomically published under run/ before installation files are rewritten. It remains available if an interrupted uninstall has removed scripts/, allowing the installed uninstall command to recover and continue. During a full uninstall, the installed uninstall command is moved to an operation-ID-specific retired path before the matching incomplete-operation record is cleared. The final cleanup removes only that retired path, never the canonical uninstall.sh path a later install may have recreated; the recovery helper follows the same matching-operation retirement rule.

The installer preserves the last readable completion record before changing installation files. Generation 1 is used only when neither completion record exists; unreadable records without a valid fallback make the operation stop before changing installed files. The manifest records install identity, generation, source and bootstrap versions, timestamp, and SHA-256 digests for files under scripts/.

The manifest writer hashes all safely named scripts files in one checked SHA-256 tool invocation and inserts their records with one SQLite session. Pending-operation validation reads all seven fields in one SQLite invocation, with hex-encoded values so delimiters and control characters in paths remain unambiguous. Lock confirmations occur at phase boundaries because the incomplete-operation record blocks competing operations during each phase; fresh-install marker and version writes share one checked phase.

The tests/test_install.bats measurements progressed from 1614 seconds on the initial head to 171 seconds after the first batching change and 130 seconds after pending-record read batching; the integration baseline is 73 seconds.

Bash 4 and later use a coprocess for the SQLite lock connection because the Windows SQLite CLI treats FIFO input as interactive; Bash 3.2 retains the FIFO path for compatibility with the system Bash on macOS.

If the SQLite lock child exits during a copy or removal, the active command may finish, but the incomplete-operation record prevents later install/uninstall operations from entering concurrently; the next boundary check stops the original operation before subsequent writes. A daemon invocation that reaches its locked entry check while the record exists exits with status 75.

## Test plan

- [x] Focused install tests cover lock-child loss during scripts copy and full uninstall removal, refusal of competing operations while the incomplete-operation record exists, recovery through the installed uninstaller after scripts/ is removed, a competing install during final retired-path cleanup, and cancellation before and after writer PID publication.
- [x] Focused manifest and pending-record tests check batched hashes, per-file fallback for an ambiguous filename, refusal before publishing when the hash tool fails its known-answer probe, one SQLite read for all pending fields, Windows path conversion, CRLF normalization, and the legacy missing-mode default.
- [x] Focused checks confirm lock liveness uses the shared local-pid helper when available and both lock/writer spawns close Bats file descriptors 3 and 4.
- [x] The focused daemon-entrypoint test verifies status 75 while an incomplete-operation record exists.
- [x] Focused install tests, bash -n, and git diff --check pass for the affected files; node --check passes for the daemon entrypoint.
- [x] Windows CI (install helpers) passes, covering the coprocess lock path, CRLF normalization, and Windows path conversion.
…1513)

## Summary

The squash of #1506 into `integration/agmsgd-beta` carried stale copies of four files and reverted #1504 in full. The #1506 diff for these files is the exact inverse of #1504's diff.

Restores the state after #1505 for:

- `scripts/drivers/types/codex/codex-record-session.sh`: resolve and record the effective CODEX_HOME with each seat
- `scripts/lib/role-session.sh`: the `codex_home` field in the role-session record
- `scripts/drivers/types/codex/template.md`: the actas step text about the profile directory
- `tests/test_codex_resume.bats`: the tests for the above

No other part of #1506 touched these files.

## Test plan

- [x] `bats tests/test_codex_resume.bats`: 35/35 pass locally
… places a launcher) (#1515)

## Summary

One `agmsg` command for 1.6.0.

- `bin/agmsg.js` (the npm entry) runs `install` itself, as before, and hands every other verb to the installed bash runtime by absolute path, preserving arguments, stdio and the exit code. When no runtime is installed it prints how to install one. The old bootstrapper's guidance table is removed; compatibility with it is not kept.
- `scripts/agmsg` is the runtime entry. In 1.6.0 only `daemon` is public; reserved verbs print that they are not available in this version; anything else exits 2 with the runtime location.
- `scripts/lib/agmsg-launcher.sh` places a marked launcher in a fixed candidate directory (`~/.local/bin`; `~/bin` first on Windows Git Bash). It never edits shell rc files; when the directory is not on PATH it prints the one line to add. An existing entry is judged without following links: valid or broken symlinks, directories, foreign files, edited launchers and another install's launcher are left untouched.
- `install.sh` places the launcher inside the install operation lock phase, on fresh install and `--update`, and marks `scripts/agmsg` and `scripts/daemon/agmsgd` executable.
- `uninstall.sh` removes the launcher only when its marker, embedded install path and contents all match this install.

Known limit: when an npm entry from 1.5.1 or earlier is still first on PATH, `agmsg daemon` reaches the old installer and is rejected; install prints a one-line note when it finds one, and `npm i -g agmsg` updates it.

## Test plan

- [x] `tests/test_agmsg_command.bats` (new) and `tests/test_bin_agmsg.bats` (rewritten) pass locally; the launcher directory is redirected with `AGMSG_BIN_DIR`, never the real `~/.local/bin`.
- [x] Launcher placement and removal, the older-npm-entry note (including an `npx` entry first on PATH), and Windows bash selection are covered by the library and entry tests above.
The entrypoint captures the verified manifest and schema while holding the install-operation lock. Before ownership claim it reacquires the lock, refuses an incomplete operation, and rechecks the manifest generation, install ID, and full scripts tree; it then opens install.db and applies the captured schema while protected. The lock remains held through ownership claim and control-socket readiness, so install.db is never opened outside the lock.

A regression test injects the incomplete-operation marker immediately after the initial lock is released and verifies agmsgd exits with status 75 without claiming daemon ownership.

Focused validation: the new entrypoint race test, the existing incomplete-operation refusal test, and the real entrypoint start/status/stop test pass; startup unit tests pass 2/2; the enforced-assertions checker passes at the existing baseline.
)

Fixes a macOS CI hang in the install cancellation test. Its deliberately blocked copy writer inherited Bats file descriptors 3/4 and standard streams, and the test did not register the lock child with teardown. A process left with those descriptors could keep the test job output pipe open after the assertions finished.

The test now disconnects both background installer invocations from Bats descriptors and stdin, closes the fake writer descriptors, and registers the writer and SQLite lock child with teardown. Teardown checks each command before signaling and waits for confirmed exit.

Validation: the focused cancellation test passes locally, and a post-run process check found no matching writer, lock, or installer processes.
The daemon now tracks Codex seat destinations and reads existing message stores without selecting message bodies. For an addressable seat, it queues one nonce-bearing inbox nudge, records the queued item, and confirms delivery from the queue row or rollout; unreadable observations remain pending and visible in status. Per-seat status reports addressable, unaddressable, blocked, or bridged, including a reason when delivery cannot be established. Queue writes are serialized by CODEX_HOME. Windows queue delivery remains blocked with an explicit status reason until delivery to a live conversation is verified. Local checks: focused Codex queue, daemon main, and status Bats tests; syntax checks for changed modules.
When the daemon beta is enabled, new Codex launches now bypass the bridge app-server and dispatcher. Existing bridge sessions continue until restarted. Ordinary script operations warn when the enabled daemon is unavailable, with recovery commands and an install-wide ten-minute warning limit; daemon status and doctor also report the failure without requiring Node for the fallback check.

Start and enable show existing bridge sessions and missing destination records. Disable distinguishes sessions that already have a bridge from sessions that need restarting, preserves unread messages, and warns about unsupported JSONL storage. README and design notes describe the optional 1.6.0 beta, macOS/Linux support, Windows delivery remaining unsupported pending measurement, and the installed command path to use when agmsg is absent from PATH.

Known limitation: concurrent operations spanning a ten-minute slot boundary can emit two warnings less than ten minutes apart; cleanup preserves current and newer claims.

Validation: `bats tests/test_agmsgd_switch.bats tests/test_agmsgd_status.bats` passed all 17 checks, including enabled/off/missing/corrupt records, direct wrapper bypass, concurrent warning suppression, a rolling ten-minute interval, past-slot-only cleanup, helper-free non-daemon operations, suppression in the watcher, hooks, one-shot polling, and their child operations, skipping repeated health queries, Node-unavailable status and doctor, executor boot evidence, and disable inventory. Shellcheck at warning severity passed with the existing SC1091 and SC2034 exclusions; `git diff --check` passed. Fixtures retain HOME and use disposable scripts, data, and CODEX_HOME. No live user service or live Codex queue was exercised. The isolated entrypoint fixture exercises a real daemon control socket.

CI follow-up: watcher and hook entrypoints export notice suppression before starting child operations, ordinary operations skip loading the daemon helper when install.db is absent, and a recent notice skips the health query. The entrypoint readiness fixture now gives its concurrent SQLite read a bounded five-second busy timeout. macOS CI timing logs showed watch continuing through test 37 at job timeout, rather than hanging: the new test file changed shard assignment, placing the 604-second Jev fixture and the active watch suite together while the weighting table still estimated watch at its historical quarantined two seconds. The table now records Jev at 604 seconds (run 36657411860), watch at 215 seconds (an unchanged 40-test run on local macOS at 96b43b5), and the new daemon switching fixture at 10 seconds (16 checks in 9.74s on local macOS). This targeted weighting update will be replaced by the rebuilt main timing table when main is brought into the integration branch.

Follow-up validation: entrypoint 3/3 PASS, CI sharding 20/20 PASS, and unchanged watch 40/40 PASS on macOS. Initial sequential timing was 215.18s at 96b43b5 versus 197.90s at 4c49efe (39/40; the fixed-three-second DB-health fixture failed with empty stdout). Its subsequent full run passed 40/40 in 338.71s while the sharding tests ran concurrently, so that rerun is not a like-for-like timing comparison. The job-timeout test 38 also passed alone. The shard balance and coverage checks pass; weighting estimates are not a claim of a completed CI run.

Targets `integration/agmsgd-beta`, based on `96b43b51`.
Brings in the #485 launcher test fix (#1519) and the refreshed shard
weight table (#1527). The weight table takes main's version, plus the
one integration-only file main does not list (test_agmsgd_switch.bats,
10s measured).
## Summary

Merge `main` into `integration/agmsgd-beta` so the integration branch picks up:

- the `#485` launcher test fix (#1519), which removes the intermittent macOS shard 1 failure seen on this branch;
- the refreshed shard weight table (#1527).

Conflict: `.github/scripts/bats-file-seconds.tsv`. Resolved by taking main's table and adding the one integration-only file it does not list, `test_agmsgd_switch.bats` (10s, measured locally on macOS).

## Test plan

- [x] `bats tests/test_ci_sharding.bats`: 20/20 locally.
- [x] `.github/scripts/check-enforced-assertions.sh`: at baseline (612).
#1530)

* fix: wait for install database readers during daemon startup

* fix: allow daemon stop before polling timer initialization
Brings in the inbox mark-read test fix (#1535) and the other changes on
main since the last sync. Conflict in scripts/doctor.sh: the daemon
health block from this branch and the orphaned run/ record scan from
main were both added at the same spot; both are kept, daemon health
first.
## Summary

Merge `main` into `integration/agmsgd-beta` again so the branch picks up the changes on main since the last sync, including the inbox mark-read test fix (#1535).

Conflict: `scripts/doctor.sh`. This branch added the daemon health report and main added the orphaned `run/` record scan at the same place. Both are kept, daemon health first.

## Test plan

- [x] `bats tests/test_agmsgd_switch.bats` (16) and `bats tests/test_doctor.bats` (40) pass locally; `bash -n scripts/doctor.sh` passes.
Brings in the changes on main since the last sync, including the
elevated-Windows plain Codex launch (#1534). Conflict in
scripts/drivers/types/codex/codex-shim.sh: main factored the plain launch
into exec_plain_launch (which adds --no-daemon on an elevated Windows
shell), and this branch added the pass-through used while agmsgd is
enabled. Both are kept: the agmsgd pass-through now goes through
exec_plain_launch for session launches, and still runs other subcommands
directly.
## Summary

Merge `main` into `integration/agmsgd-beta` again, picking up the changes on main since the last sync, including the elevated-Windows plain Codex launch (#1534).

Conflict: `scripts/drivers/types/codex/codex-shim.sh`. Main factored the plain launch into `exec_plain_launch` (adds `--no-daemon` on an elevated Windows shell); this branch added the pass-through used while agmsgd is enabled. Both are kept: while agmsgd is enabled, session launches go through `exec_plain_launch`, and the listed non-session subcommands still run the real binary directly.

## Test plan

- [x] `bats tests/test_codex_shim.bats` (29) and `bats tests/test_agmsgd_switch.bats` (16) pass locally; `bash -n` passes on the shim.
The held-lock launcher test could fail before it exercised the launcher: its independent `BEGIN EXCLUSIVE` readiness probe could acquire the lock first, making the holder's own nonblocking `BEGIN` fail with `database is locked`. The holder then waited on its FIFO without owning the lock, while the polling loop eventually reached its unconditional cutoff. The launcher correctly classified the free lock and `.prev` manifest as an incomplete previous update.

Wait for an acquisition acknowledgement from the holder itself instead. Enable SQLite bail-on-error so a failed `BEGIN` cannot produce a false acknowledgement, and bound the acknowledgement wait to five seconds. Release the holder before checking the launcher result, and assert exit 75, the update-in-progress diagnostic, and no failed-update attempt record. Production launcher behavior is unchanged.

Validation: all eight launcher bats tests pass on macOS, and the held-lock case passes five additional runs. A forced probe-first reproduction produced `Runtime error near line 1: database is locked (5)` while the holder continued without owning the lock. Local verification used a scratch helper copy without HOME reassignment or scratch directory deletion. `git diff --check` passes.
## Summary

Permit new Windows queue operations only when a native codex.exe resolves and that executable reports codex-cli 0.157.0. This release has measured evidence that native queue creates no descendants and that its notice reaches a logged-in live conversation. Missing executables, unreadable version output, and every unlisted version remain blocked with a reason. The version probe has a two-second timeout and bounded output. Queue launches use the resolved native path directly, including the measured npm global installation layout, rather than its JavaScript or shell shim.

Apply the same gate while reconciling pending Windows operations, preserving confirmation and expiry observations even when a changed or unavailable executable blocks new delivery.

Add an anonymized, measured Windows response_item fixture from 2026-10-01, preserving field types and positions, including metadata.client_authored=false and metadata.user_input_order. One regression verifies that the fixture's user input nonce is observed as present.

## Validation

- `node --test tests/agmsgd_codex_queue.test.mjs`: 9/9 PASS.
- `bats tests/test_agmsgd_codex_queue.bats`: 1/1 PASS (runs the Node support suite).
- `git diff --check`: PASS.
- Tests cover executable resolution, the measured npm layout, unlisted and unreadable versions, native executable forwarding, pending-to-confirmed Windows delivery, and preventing duplicate nudges for a confirmed snapshot.

Validation ran on macOS with disposable profiles and mocked native version/queue processes. It did not launch a live Windows service or modify a user queue. The measured rollout fixture and native path layout were supplied from the existing Windows live-delivery measurement.

Targets integration/agmsgd-beta. Merge is handled by the integration coordinator.
Brings in 1.5.2 from main, including the optional Codex profile field in
the seat record (#1549). That change and this branch's profile recording
(#1504) touched the same code:

- scripts/drivers/types/codex/codex-record-session.sh: keep a single
  profile resolution. A missing, relative or malformed profile no longer
  skips the seat record (main's behavior); the record is written without
  the profile and rollout discovery falls back to $HOME/.codex as before.
  A resolved profile is still used for discovery and stored for delivery.
- scripts/drivers/types/codex/template.md: keep this branch's wording,
  since on this branch the profile is used to route notices.
- tests/test_spawn.bats: take main's identical CODEX_HOME keep entry.
## Summary

Merge `main` (1.5.2) into `integration/agmsgd-beta`. Main now carries the optional Codex profile field in the seat record (#1549), which overlaps this branch's profile recording (#1504).

Conflict resolution:

- `scripts/drivers/types/codex/codex-record-session.sh`: one profile resolution instead of two. A missing, relative or malformed profile no longer skips the seat record (main's behavior): the record is written without the profile, and rollout discovery falls back to `$HOME/.codex` as before. A resolved profile is still used for rollout discovery and stored for delivery.
- `scripts/drivers/types/codex/template.md`: keep this branch's wording, since here the profile routes notices.
- `tests/test_spawn.bats`: take main's equivalent `CODEX_HOME` keep entry.

## Test plan

- [x] `bats tests/test_codex_resume.bats`: 37/37 locally, including main's "unresolved profile omits only the field" case and this branch's profile discovery cases.
- [x] `bats tests/test_role_session.bats` (22) and the spawn reader-inventory test pass locally.
Merge main into integration/agmsgd-beta (5)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant