Gastown Cloud: Cloud-first rewrite of Gastown as a Kilo platform feature

> **Disclaimer**: This is completely experimental. We are currently exploring multiple angles for integration.

## Overview

Cloud-first rewrite of Gastown's core tenets as a Kilo platform feature (Proposal D — Revised). All orchestration state lives in Durable Objects (SQLite). Agents interact with Gastown via **tool calls** backed by DO RPCs — no filesystem coordination, no gt/bd binaries. Each town gets a **Cloudflare Container** that runs all agent processes (Kilo CLI instances) — one container per town, not one per agent.

**Product vision:** An absurdly simple product. Create a town, connect repos via existing integrations, talk to the Mayor in a chat interface. The full Gastown machine operates behind the scenes — agents spawn, communicate, merge code — and the UI shows everything happening in real time. Every object on screen is clickable, every connection is traceable.

See `plans/gastown-cloud-proposal-d.md` for the full architecture proposal, UI vision, and architecture assessment.

---

## Workflow

### For each sub-issue:

1. **Branch**: Create a branch off main using the naming scheme {issue_number}-{short-description}
2. **Implement**: Work through the issue's acceptance criteria. Write tests alongside the implementation.
3. **Validate**: Run the test suite and type checks (pnpm typecheck) to confirm everything passes.
4. **Commit**: Stage all changes and create a commit with a descriptive message referencing the sub-issue number.
5. **Push & PR**: Push the branch and create a pull request targeting main. Link the sub-issue in the PR body with Closes #NNN.

---

## Phase 1: Core Loop + Mayor (COMPLETED ✅)

Foundation: Rig DO state machine, HTTP API, tool plugin, container with kilo serve, alarm scheduler, tRPC routes, basic dashboard, MayorDO.

- [x] #208 — **PR 1: Rig Durable Object**
- [x] #209 — **PR 2: HTTP API Layer**
- [x] #210 — **PR 3: Tool Plugin**
- [x] #211 — **PR 4: Town Container — Execution Runtime**
- [x] #212 — **PR 5: Rig DO Alarm — Work Scheduler**
- [x] #305 — **PR 5.5: Container — Adopt kilo serve**
- [x] #268 — **PR 6: tRPC Routes**
- [x] #213 — **PR 7: Basic Dashboard UI** (skeleton)
- [x] #335 — **PR 7.5: Agent Lifecycle & Container Reliability Fixes**
- [x] #338 — **PR 8: MayorDO — Town-Level Conversational Agent**

## Phase 1.5: Close the Loop + Usable UI (NEW — Priority Sprint)

> The core loop is built but doesn't close: the Mayor can't delegate, agents can't be watched, merges 404, and the dashboard is a skeleton. These issues unblock a product that actually works end-to-end.

**P0 — Must complete before product is usable:**

- [x] #339 — **Mayor Tools — Cross-Rig Delegation** ⚡ Highest priority. Without tools, the Mayor is a chatbot that can't do anything. The chat-first product vision depends entirely on this.
- [x] #343 — **Agent Streaming Endpoint** ⚡ WebSocket streaming from container through TownContainerDO to browser. PR #348.
- [x] #349 — **Mayor Tool Execution Failures** ⚡ Mayor agent starts and has tools registered but completes without using them. Need to diagnose container-side errors when mayor tools (gt_sling, gt_list_rigs, etc.) execute — likely auth or network issue with GASTOWN_API_URL from inside the container.
- [x] #214 — **Manual Merge Flow** ⚡ The container has no `/merge` endpoint — the call 404s. The polecat→done→merge→closed loop is broken. Beads in the review queue sit forever.
- [x] #344 — **Rig DO Event Log** ⚡ No append-only event stream means no activity feed, no bead timeline, no real-time dashboard. Required for the UI to show "what happened."

**P1 — High impact, required for decent UI:**

- [x] #345 — **Wire Detailed Polecat System Prompt** — Low effort, high impact. A GUPP-aware prompt with tool docs exists but isn't used.
- [x] #346 — **Dashboard UI Overhaul** — Split-pane town home (Mayor chat + dashboard), rig workbench (bead board, agent roster, stream panel), clickable detail panels. Depends on #339, #343, #344.

## Phase 2: Multi-Agent Orchestration (COMPLETED ✅)

All Phase 2 issues implemented in PR #389.

- [x] #215 — **Town Durable Object** — Cross-rig coordination: convoy lifecycle, escalation routing, watchdog heartbeat.
- [x] #385 — **Town Configuration — Environment Variables & Settings** — Env var editor, git auth tokens, agent defaults, refinery config, settings page UI.
- [x] #386 — **Integrations-Based Repo Connection** — Repo picker via GitHub/GitLab integrations, platform_integration_id on rigs, self-hosted GitLab support.
- [x] #216 — **Multiple Polecats per Rig** — 20-name pool, configurable concurrency cap, bead-ID branch isolation.
- [x] #218 — **Refinery Agent** — AI-powered merge review with quality gates, rework requests. Falls back to deterministic merge.
- [x] #219 — **Molecule/Formula System** — Multi-step workflows with gt_mol_current / gt_mol_advance tools, auto-completion.
- [x] #220 — **Convoy Lifecycle** — Cross-rig batch work tracking with automatic landing detection on bead close.
- [x] #221 — **Escalation System** — Severity routing (including critical), auto-re-escalation with 4h spacing, acknowledge flow.

**Known issues from Phase 2 testing:**

- #411 — Polecat stuck in zombie hook state after clone failure causes cascading sling failures

## Phase 2.5: Town-Centric Refactor

> Simplify the DO topology, replace subprocess-based agent management with the SDK, and establish a clean WebSocket event pipeline. See `plans/gastown-town-centric-refactor.md` for the full plan.

- [x] #419 — **Town-Centric Refactor** — Consolidate all control-plane data into TownDO (eliminate Rig DO + Mayor DO), SDK-based agents (`createOpencode()` replaces `Bun.spawn`), WebSocket streaming (eliminate SSE + polling + tickets), per-agent AgentDO for event storage, config-on-request (eliminate stale container injection), proactive container/mayor startup. PR #421.
- [x] #441 — **Beads-Centric Data Model** — Unify all object types (mail, molecules, convoys, escalations, merge requests, agents) into the universal beads primitive with satellite metadata tables. Drop `rig_mail`, `rig_molecules`, `rig_review_queue`, `town_convoys`, `town_convoy_beads`, `town_escalations`, `rig_agents`. Add `parent_bead_id`, `bead_dependencies` table, entity-prefixed column naming.
- [x] #443 — **Fix: Polecat git push credentials** — Git credentials are never populated in town config. The tRPC `createRig` doesn't set `git_auth.github_token`, so `GIT_TOKEN` is never passed to the container. Agents clone public repos fine but can't push. Fix: populate git credentials from integration tokens on rig creation, add token refresh, fix merge dispatch for self-hosted GitLab.
- [x] #450 — **Agent Terminal UI (xterm.js)** — Embed live terminal view of agent sessions via xterm.js connected to the container's existing PTY infrastructure. Proxy PTY WebSocket through control-server → TownContainerDO → worker → browser. Enables direct visibility into agent work and interactive agent sessions from the dashboard.
- [x] #225 — **Dashboard Deep Drill-Down** — Convoy visualization, conversation history/replay, topology view, cost tracking.

## Phase 3: Multi-Rig + Scaling

- [x] #537 — **Gate Gastown UI to Kilo Admins** — Restrict all Gastown pages, tRPC procedures, and nav items to `is_admin` users only. Temporary gate until Gastown is ready for beta/GA.
- [x] #686 — **PR #544 Review Hardening** — Fire-and-forget error logging, SDK response validation, rig_id registration verification, thread-safe config, escalation notification events, atomic rig+credential creation, git-credentials route, 403 for cross-town access.
- [x] #657 — **Bug: Refinery dispatched to wrong rig** — Four compounding bugs cause the refinery to clone the wrong rig's repo in multi-rig towns: singleton reuse ignores rig_id, processReviewQueue hardcodes rigList[0], merge_request beads have null rig_id, ReviewQueueInput has no rig_id field.
- [x] #473 — **Configurable Merge Strategy** — Support `direct` (Refinery pushes to main) and `pr` (Refinery creates a GitHub/GitLab PR for human review) merge strategies. Configurable at town level with per-rig overrides. PR strategy includes quality gate results, agent attribution, and webhook-based status tracking. PR #701.
- [x] #717 — **Replace CF Access with Direct JWT Validation** — Remove Cloudflare Access from the Gastown worker and validate Kilo user JWTs directly using the shared signing secret. Fixes production container→worker auth without leaking CF Access credentials into containers. PR #701.
- [x] #540 — **Wire Convoys End-to-End** — Add `gt_sling_batch` tool for atomic multi-bead + convoy creation, `gt_list_convoys` and `gt_convoy_status` visibility tools, Mayor prompt updates, and dead code cleanup. Backend convoy infrastructure (createConvoy, onBeadClosed, landing detection) already works — this connects it to the Mayor.
- [x] #901 — **Feature Flags & Progressive Rollout** — Replace the binary `is_admin` gate (#537) with progressive rollout: admin-only → allowlist → percentage → GA. Kill-switch for instant revert. Sub-feature flags for convoys, PR merge strategy, multi-rig. Technology-agnostic — defines integration points, not vendor choice.
- [x] #442 — **Witness & Deacon: Alarm-Driven Orchestration** — Run all mechanical witness/deacon patrol behaviors (protocol mail flow, zombie detection, GUPP violations, convoy completion, stale hooks, gate evaluation) as deterministic code in the TownDO alarm. Spawn short-lived LLM triage agents only for ambiguous situations (dirty polecat triage, stuck agent inspection, help requests). Replaces persistent AI patrol loops with alarm + on-demand reasoning.
- [x] #223 — **Multi-Rig Support** — Closed. Implemented across #419, #441, #657. Remaining gaps (per-bead rig targeting in convoys, production dashboard) tracked by #540 and #346/#225.
- [x] #269 — **Container Resilience — Checkpoint/Restore** — Ephemeral disk recovery, git state restoration.
- [x] #925 — **Mayor: `gt_rig_browse` tool** — Let the Mayor inspect rig code on demand. New container endpoint clones-on-demand or fetches-if-stale, returns file tree or file contents. Intermediate solution for post-restart empty workspaces (full recovery in #269).
- [x] #923 — **Bug: Agent JWT expires after 8h with no refresh** — `GASTOWN_SESSION_TOKEN` is minted once per `startAgentInContainer()` with 8h expiry. No refresh mechanism exists. Persistent agents (Mayor) that run >8h get 401s on all tool calls, heartbeats, and event persistence. Affects production now.
- [x] #965 — **Bug: Triage agent feedback loop** — `detectCrashLoops()` counts triage agent's own bead failures as crash loops, creating new triage requests that fail and feed the loop. Compounds with: no dispatch cooldown after failure, unnecessary git clone for triage role, no cap on triage request creation. Actively happening in production.
- [x] #962 — **Bug: Convoy landing merge never triggers** — `updateConvoyProgress()` is never called after MR beads close because `completeReview()` uses raw SQL (bypasses `updateBeadStatus`) and `closeBead()` on the already-closed source bead is a no-op. The `ready_to_land` flag is never set, so `processConvoyLandings()` has nothing to process. Fix: explicitly re-trigger `updateConvoyProgress` in `completeReviewWithResult()`.
- [ ] #1236 — **Persist Agent Conversation Across Restarts** — Reconstruct conversation history from AgentDO streaming events (message.created/completed/part.updated) and inject as context on re-dispatch. Fix sendMayorMessage passing checkpoint: null. Mayor maintains conversational continuity across container restarts. Transcript truncated/summarized to fit context window.
- [x] #1195 — **Fix Terminal Stability** — Three compounding bugs: (1) PTY WebSockets have no reconnection logic — container sleep/restart = permanent freeze. (2) Undebounced ResizeObserver fires resize storms during CSS transitions — PTY/xterm dimension mismatch causes blacked-out cells. (3) Missing 0x00 control frame filter — SDK cursor metadata written as raw JSON to xterm. Reference fix exists in the native Kilo app.
- [x] #1182 — **Bug: Town deletion leaks DOs and containers** — tRPC deleteTown only removes rows from GastownUserDO, never calls TownDO.destroy() or stops the container. TownDO alarm re-arms unconditionally (no exit condition), keeps pinging the container (preventing sleep), and can resurrect after destroy() via constructor re-initialization. 6 compounding root causes. Actively burning Cloudflare resources in production.

## Phase 4: Hardening

- [x] #897 — **User-Scoped Admin Panel** — User-scoped admin panel for debugging and intervening in Gastown towns. Entry point on existing admin user detail page: GastownUserDO lookup → town list → Town Inspector with bead/agent/container/config drill-down and admin interventions (force-reset, force-close, credential refresh, container restart). Immutable audit log for all actions.
- [x] #895 — **In-Review Bead State** — Add `in_review` state between `in_progress` and `closed`. Polecats mark beads `in_review` on completion; only the Refinery transitions to `closed` (on merge) or back to `in_progress` (on rework). Bead board gets an "In Review" column.
- [x] #1006 — **Staged Convoys** — `gt_sling_batch` with `staged=true` creates convoy + beads without hooking agents or dispatching. Users review the plan, iterate on beads (add/remove/edit), then `gt_convoy_start` to kick off execution.
- [x] #996 — **Mayor Edit Capabilities + Manual Editing** — Give the Mayor surgical edit tools: `gt_bead_update` (status, title, body, priority), `gt_bead_reassign`, `gt_agent_reset`, `gt_convoy_close`, `gt_bead_delete`, `gt_escalation_acknowledge`. Add PATCH endpoints for bead fields and agent status with user auth. Add edit controls to the dashboard UI (status dropdown, editable fields, unhook/reset buttons).
- [x] #228 — **Observability** — Structured logging, event streaming, alerting, usage metrics.
- [x] #958 — **Polecat Pre-Submission Gates** — Inject the town's quality gates (`townConfig.refinery.gates`) into the polecat system prompt so polecats self-validate before calling `gt_done`. Catches lint/test/build failures in-session instead of wasting a refinery round-trip.
- [x] #1032 — **Cloud-Native Nudge System** — Reliable real-time message delivery to agents. Non-mayor agents enter “idle-but-available” state on `session.idle` instead of exiting, check for pending nudges, and stay alive to process them. Nudge queue in TownDO with wait-idle/immediate/queue delivery modes. `gt_nudge` tool for agents. Patrol functions use nudge instead of mail for time-sensitive messages.
- [x] #1154 — **Gastown Feature Attribution** — Add `gastown` to `FEATURE_VALUES` and have the container send `X-KILOCODE-FEATURE: gastown` on all LLM requests. Currently all Gastown agent usage is unattributed (NULL) in `microdollar_usage_metadata`.
- [ ] #963 — **Bug: Shift+Enter doesn't insert newline in xterm.js terminal** — xterm.js 6.0.0 doesn't support the Kitty keyboard protocol. Shift+Enter sends `
` (same as Enter) instead of `^[[13;2u`. Quick fix: `attachCustomKeyEventHandler` to inject the CSI u sequence. Proper fix: upgrade xterm.js to 6.1+ or switch to ghostty-web.
- [x] #1100 — **Bug: Mayor cannot edit bead dependencies** — Dependency graph is immutable after creation — no write functions, no HTTP endpoints, no Mayor tools exist for adding/removing `bead_dependencies` rows. Full vertical slice needed: core functions with cycle detection, TownDO RPC methods, HTTP endpoints, `gt_bead_add_dependency` / `gt_bead_remove_dependency` tools, and `depends_on` param on `gt_sling`. Missed in #1076.
- [ ] #1145 — **Sort agents by status** — Agent cards on overview and rig pages sorted with working agents at top, then starting, idle, and exited/failed at bottom. Secondary sort by last_activity_at descending.
- [ ] #229 — **Onboarding Flow** — Zero to agent in under 2 minutes wizard.
- [ ] #995 — **Detect misdirected messages to polecats/refinery** — Prompt-level heuristic that detects when a user message looks like it was meant for the Mayor (town status, work delegation, questions about other agents) and responds with a redirect. Prevents polecats from misinterpreting coordination messages as work instructions.
- [ ] #825 — **Revisit bead metadata column vs. satellite metadata tables**
- [ ] ~#1001~ — ~~Git Commit Co-Authorship~~ — Superseded by #1350. — Human as `GIT_AUTHOR`, agent as `GIT_COMMITTER`. User name/email resolved from Kilo user record and passed through dispatch. `Co-authored-by` and `Bead-ID` trailers in commit messages. GitHub contribution graphs credit the human.
- [ ] #1002 — **Automatic PR Comment Resolution** — Extend PR polling to detect unresolved review comments, dispatch agents to address them in batches (fix code + push + reply + resolve thread), escalate architectural concerns to Mayor. GitHub and GitLab support.
- [ ] #1003 — **Track Commits in Bead History** — Capture commits produced by agents per bead (on `gt_done`), store SHA/message/author/timestamp with SCM URLs. Bead detail panel shows commit history with clickable links to GitHub/GitLab.
- [ ] #1004 — **Automatic Merge Conflict Resolution** — Detect merge conflicts on open PRs via GitHub/GitLab API, dispatch agent to resolve conflicts, run gates, and push. Escalate complex conflicts.
- [ ] #1005 — **Bead Archival** — Add `archived_at` timestamp to beads. Archived beads hidden from default views but preserved for history. Archive/unarchive via API, Mayor tool, and dashboard UI. Bulk archive for closed beads and completed convoys.
- [ ] #1073 — **Parallel Refinery: Per-Convoy Refinery Agents** — One refinery per active convoy for intermediate merges (polecat branch → feature branch), running in parallel across convoys. Rig singleton refinery stays for standalone beads and final landing merges to main. `processReviewQueue()` pops multiple items per tick and routes by convoy vs standalone. Configurable `refinery.maxConvoyConcurrency`.
- [ ] #1040 — **Integrating with the Wasteland** — Dolt binary in the container, local MySQL server for fast queries, outbound sync (TownDO → Dolt → DoltHub) and inbound sync (Wasteland commons → Mayor). DoltHub OAuth onboarding, fork creation, PR submission to commons on bead completion. Full sync for existing towns via GastownDoltSyncDO. Incremental sync on cadence + push before eviction.
- [ ] #231 — **Documentation & API Reference**
- [ ] #224 — **Agent CVs & Performance Analytics** — Agent Identity DO, work ledger, performance cards.
- [ ] #226 — **Stress Testing** — 30 concurrent polecats, latency benchmarks, resource limits.
- [ ] #227 — **Edge Case Handling** — Split-brain, container OOM, crash recovery.
- [ ] #1307 — **Parse Agent Status from H1 Headers** — Replace the `gt_status` tool (which agents often forget to call) with automatic H1 parsing from the agent’s text output. Parse `message.part.updated` events in `broadcastEvent()`, extract first H1 line, post to existing `updateAgentStatusMessage` API. Status bubbles work automatically. Remove `gt_status` tool and prompt references.
- [ ] #1297 — **Real-Time Throughput Gauges & Timeseries** — Live gauges on town overview: tokens/sec, cost/sec, active agents, and a gas tank fuel gauge (balance → empty/full with time-remaining estimate). Timeseries charts on observability tab: token throughput, cost over time, agent utilization, bead velocity, review queue depth. Zoomable 1h/6h/24h/7d windows.
- [ ] #1249 — **Lazy Agent Assignment** — Move agent assignment from bead creation time to schedule time. Currently slingBead/slingConvoy eagerly create and hook agents to ALL beads including blocked ones. The scheduler should assign agents only when beads are unblocked and ready. Reduces wasted agent slots, preserves the name pool, enables serial convoy reuse, and makes staged/non-staged convoys consistent.
- [ ] #1335 — **Reconciler Phase 5: Debug endpoints, Grafana integration, and alerting** — Complete remaining reconciler hardening deliverables: event replay debug endpoint (replay town_events to diagnose stuck states), reconcile dry-run endpoint, Grafana dashboard panels for reconciler metrics, Sentry alerting on invariant violations, container_status event noise filtering.
- [ ] #1350 — **GitHub OAuth Device Flow for User-Scoped Git Identity** — One-click GitHub authorization via Device Flow. User token provides commit attribution (human as Author, agent as Committer), PR authorship, contribution graph credit, and gh CLI auth. Supersedes #1001.
- [ ] #1248 — **Gastown Bug Reporting** — GitHub issue template for structured bug reports, "Report a Bug" link in the dashboard UI, and a `gt_report_bug` Mayor tool that searches existing issues for duplicates and files new ones with structured context (town ID, rig, recent errors). UI link is the fallback when the Mayor/container is unavailable.
- [ ] #1237 — **Configurable Terminal Orientation + Resize** — Terminal bar can be positioned at bottom/top/left/right with drag-to-resize. Page content adjusts dynamically (replacing static pb-[340px]). Collapsed state only reserves the tab bar strip. Position and size persisted to localStorage. Vertical orientation adapts tab bar, status pane, and DrawerStack.
- [ ] #1173 — **Redesign Merge Queue Page** — Split into two sections: “Needs Your Attention” (open PRs, failed reviews, merge conflicts blocking on humans) and “Refinery Activity Log” (human-readable timeline of what bots did). Convoy reviews grouped under convoy headers. Every bead reference links to stackable drawer. New tRPC procedure returning full merge_request bead data.
- [ ] #1174 — **Upstream Branch Cleanup** — Delete merged polecat and convoy branches from the remote. Refinery deletes source branch after successful merge (deterministic, via git manager). Convoy feature branches deleted after landing or close. Alarm-based sweep catches orphaned branches from crashed agents.
- [ ] #1172 — **Bead Failure Reasons** — 8 of 16 failure paths record no reason for why a bead failed. Add structured `failure_reason` (code, message, details, source) to every failure event. Fix `completeReview()` raw SQL paths to use `updateBeadStatus()`. Persist container’s `reason` param for polecat crashes (currently discarded). Render failure reasons in UI with red banner + expandable details. Stretch: LLM-written plain-language summaries for complex failures.
- [ ] #230 — **Postgres Replication Layer** — Read replica, sync-on-write, migrate dashboard reads.
- [ ] #1136 — **Gastown Sync Update: 2026-03-16** — Analysis of Gastown Local v0.7.0–v0.12.1 changes. Key findings: ACP (Agent Communication Protocol), Agent Provider abstraction, Mountain-Eater (autonomous epic execution), Convoy Multi-Store, Swarm removal, Generic Hook Installer. Not a commitment to implement — a reference for tracking ecosystem evolution.

---

## Future Ideas

- [ ] #447 — **Cloud-Unique Capabilities** (org-level towns shipped in PR #1153) — Ideas for features that are impossible or dramatically better in the cloud: shared towns, agent marketplace, cross-session intelligence, cost attribution/budget controls, speculative execution, webhook-driven beads, persistent agent reputation, local CLI bridge.

---

## Closed/Superseded Issues

- ~#217~ — **Witness Alarm** — Folded into #212 (Rig DO Alarm). The witness patrol is part of the alarm handler.
- ~#222~ — **Mayor Agent (old design)** — Superseded by #338 (MayorDO).

Navigation Menu

Uh oh!

Gastown Cloud: Cloud-first rewrite of Gastown as a Kilo platform feature #204

Overview

Workflow

For each sub-issue:

Phase 1: Core Loop + Mayor (COMPLETED ✅)

Phase 1.5: Close the Loop + Usable UI (NEW — Priority Sprint)

Phase 2: Multi-Agent Orchestration (COMPLETED ✅)

Phase 2.5: Town-Centric Refactor

Phase 3: Multi-Rig + Scaling

Phase 4: Hardening

Future Ideas

Closed/Superseded Issues

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Uh oh!

Gastown Cloud: Cloud-first rewrite of Gastown as a Kilo platform feature #204

Description

Overview

Workflow

For each sub-issue:

Phase 1: Core Loop + Mayor (COMPLETED ✅)

Phase 1.5: Close the Loop + Usable UI (NEW — Priority Sprint)

Phase 2: Multi-Agent Orchestration (COMPLETED ✅)

Phase 2.5: Town-Centric Refactor

Phase 3: Multi-Rig + Scaling

Phase 4: Hardening

Future Ideas

Closed/Superseded Issues

Activity

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions