feat(agent): extract lifecycle management into bootstrap process - #579
rmlearney-digicatapult wants to merge 12 commits into
Conversation
Signed-off-by: rmlearney-digicatapult <robert.learney@digicatapult.org.uk>
Signed-off-by: rmlearney-digicatapult <robert.learney@digicatapult.org.uk>
Signed-off-by: rmlearney-digicatapult <robert.learney@digicatapult.org.uk>
There was a problem hiding this comment.
Pull request overview
Executive Summary
This PR extracts cloud-agent startup and shutdown orchestration into bootstrap.ts, preparing the service for additional applications such as OpenID4VC. It adds coordinated server cleanup, externally managed DIDComm WebSockets, lifecycle tests, and a patch-version bump. Several startup failure paths can still leak resources or bypass cleanup, so changes are required.
Changes:
- Added reusable
startCloudagent()lifecycle management. - Simplified
index.tsto signal handling. - Added shutdown/restart tests and DID:web stop support.
Blockers (Must-Fix)
src/bootstrap.ts:99-102: DIDComm WebSocket bind errors escape startup handling.src/bootstrap.ts:110-111: partially initialized agents cannot be cleaned up.src/bootstrap.ts:176: DID:web startup does not await successful binding.src/bootstrap.ts:262: failed admin binding aborts remaining cleanup.src/agent.ts:255-256: multiple WS transports reuse one server and duplicate handlers.tests/unit/bootstrap.test.ts:189-190: shared wallet store can be deleted twice.
Targeted Suggestions
- Await
listening/errorfor every listener within the guarded startup path. - Make cleanup best-effort so one close failure does not skip later resources.
- Either map external WS servers by configured port or reject multiple WS entries.
Test Gap Analysis
Add regression tests for occupied admin, DID:web, and DIDComm ports; partial setupAgent() failure; and multiple WS transport entries.
Integration Risks
Failed startup can leave wallets or ports open, affecting restart and deployment rollback. No API or database schema migration is introduced.
Scores
| Category | Score | Reason |
|---|---|---|
| Correctness | 4/10 | Multiple lifecycle failure paths remain broken |
| Security | 7/10 | No direct regression, but leaked listeners are undesirable |
| Testing | 5/10 | Happy paths covered; failure paths are absent |
| Maintainability | 6/10 | Better separation, but cleanup contracts remain fragmented |
| Performance | 8/10 | No material steady-state regression identified |
| Integration/Infra | 4/10 | Port conflicts and restart cleanup are unreliable |
| Overall | 5.5/10 | Lifecycle design is promising but not merge-ready |
Release Notes Draft
- Extract cloud-agent lifecycle orchestration into
startCloudagent(). - Add coordinated shutdown for REST, DID:web, and WebSocket services.
- Add lifecycle restart and active DIDComm WebSocket tests.
- Bump package version to
0.20.36.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
src/bootstrap.ts |
Adds lifecycle orchestration and cleanup |
src/index.ts |
Delegates startup and shutdown |
src/agent.ts |
Supports externally managed DIDComm WS server |
src/didweb/server.ts |
Adds explicit server stop method |
tests/unit/bootstrap.test.ts |
Adds lifecycle tests |
package.json |
Bumps package version |
package-lock.json |
Synchronizes package version |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Signed-off-by: rmlearney-digicatapult <robert.learney@digicatapult.org.uk>
Signed-off-by: rmlearney-digicatapult <robert.learney@digicatapult.org.uk>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 2 comments.
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
src/bootstrap.ts:178
- This lifecycle creates a Knex-backed
Database, but neithershutdown()norDidWebServer.stop()destroys its connection pool. With DID:web enabled,generateAndRegister()performs an upsert and opens PostgreSQL connections; every in-process restart then leaks another pool, undermining the new reusable lifecycle. Add a databasedestroy()operation and invoke it during normal and failed startup cleanup.
const database = new Database({
host: env.get('POSTGRES_HOST'),
database: env.get('DID_WEB_DB_NAME'),
user: env.get('POSTGRES_USERNAME'),
password: env.get('POSTGRES_PASSWORD'),
port: env.get('POSTGRES_PORT'),
})
Signed-off-by: rmlearney-digicatapult <robert.learney@digicatapult.org.uk>
Signed-off-by: rmlearney-digicatapult <robert.learney@digicatapult.org.uk>
Signed-off-by: rmlearney-digicatapult <robert.learney@digicatapult.org.uk>
| const shutdown = async () => { | ||
| if (!shuttingDownPromise) { | ||
| shuttingDownPromise = (async () => { | ||
| await closeServer(adminServer) |
There was a problem hiding this comment.
[BLOCKER] closeServer(adminServer) waits before admin WebSocket clients are terminated. adminSocketServer uses noServer, so its upgraded sockets belong to this HTTP server and Node does not close them automatically; an active admin WebSocket can therefore block shutdown indefinitely. Terminate/close adminSocketServer before awaiting the HTTP server close, and add an active admin WebSocket regression test.
|
|
||
| if (didcommSocketServer) { | ||
| // Remove once upstream websocket transport closes connected clients during stop. | ||
| for (const client of didcommSocketServer.clients) { |
There was a problem hiding this comment.
[MAJOR] This terminates clients only on the first externally managed DIDComm WebSocket server. Additional WS transports create private servers in setupAgent, so an active client on a later transport makes agent.shutdown() wait until the backstop timeout. Manage and terminate every configured WS server, or reject multiple WS transports. The multiple-transport test currently terminates both clients before shutdown and does not cover this path.
| agent.didcomm.registerInboundTransport(new DidCommWsInboundTransport({ port: inboundTransport.port })) | ||
| } | ||
|
|
||
| await agent.initialize() |
There was a problem hiding this comment.
[MAJOR] Credo 0.7.0 does not roll back modules when Agent.initialize() fails. DIDComm transports start before Askar opens the wallet, so a wallet initialisation failure leaves those transports running while the agent is never returned to startCloudagent(). Move agent.initialize() inside this cleanup try.
|
|
||
| const listen = async (server: HttpServer) => { | ||
| await new Promise<void>((resolve, reject) => { | ||
| server.once('listening', () => resolve()) |
There was a problem hiding this comment.
[MINOR] Remove the opposite event listener when either listening or error fires. After successful startup, this stale error listener silently consumes the server's first runtime error; the DIDComm listener at line 120 has the same problem.
Pull Request
Checklist
PR Type
Please delete options that are irrelevant.
Linked tickets
High level description
Extract lifecycle (startup & shutdown) into single
bootstrap.tsDetailed description
Veritablecurrently mixes startup and shutdown logic with signal handlers inindex.ts, which only comprisesawaitlogic and no conditional branching.OpenID4VC requires a more complex init pattern which creates an Express app, reconciles resources depending on role config after init, registers 404/other handlers after Credo-ts and begins listening only after success
Refactored
index.tsto extract startup and shutdown orchestration intobootstrap.tsso that theindex.tsentrypoint focuses on underlying signal logic.startCloudagent()now:shutdown()method to prevent double-close if clients still connectedDrpcReceiveHandlerto prevent resolving stale registration on stopped agentAlso removed
@singletonfromDrpcController.ts(the only controller with@singleton) so that the controller attaches to a new agent on restart. The controller is already stateless.Improvements:
startCloudagent()agent.shutdown()to prevent hangs if WS still connected. New shutdown logic prevents race conditions if shutdown called inline from signal handlersTests to demonstrate clean restart behaviour, shutdown with active DIDComm WebSocket connection, restart reconnects to existing wallet store
Describe alternatives you've considered
Operational impact
No changes for existing consumers
Additional context
Precursor to introducing OpenID4VC support which requires its own Express app and would be harder to wire into existing logic
NB Not a new behaviour introduced by the changes to the
DrpcControllerbut worth noting that on process shutdown/restart just after DRPC message receipt, Credo-ts may contain staleDrpcRecordentries without an ability to respond to them. No on-startup rehydration/erase/error-report logic was implemented for this edge case.