diff --git a/.agent-loop/CURRENT_STATE.md b/.agent-loop/CURRENT_STATE.md index 054bb0019..02c2c84f4 100644 --- a/.agent-loop/CURRENT_STATE.md +++ b/.agent-loop/CURRENT_STATE.md @@ -33,7 +33,7 @@ authority; these records do not grant or withhold it. | [WS-AUTH-001](initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md) | Active delivery initiative; project-policy authority and unified compilation authorization are merged through `12I` | POL-03B consumes 12I next; remaining AUTH activation chunks wait for their exact hidden owner behavior | | [WS-CON-001](initiatives/WS-CON-001-contribution-compensation-boundary/STATUS.md) | Active delivery initiative; CP03B completes adapter-binding activation and CP04A-CP04B complete hidden ContributionPolicy behavior; shared lifecycle audit foundations are merged | Prepare CP05 policy activation, then complete guide-activation validation/persistence before task readiness | | [WS-AUTH-003](initiatives/WS-AUTH-003-module-boundary-recovery/STATUS.md) | AUTH boundary foundation and first public-capability proof through POL-03A are merged | Repair each touched AUTH capability through `authorization.api` and shrink the canonical AUTH ledger | -| [WS-POL-003](initiatives/WS-POL-003-unified-project-guide-compilation/STATUS.md) | Active delivery initiative; hidden compilation custody and AUTH-12I activation are merged | Implement POL-03B authorized compilation persistence, then continue the reviewed dependency order | +| [WS-POL-003](initiatives/WS-POL-003-unified-project-guide-compilation/STATUS.md) | Active delivery initiative; `WS-POL-003-03B` authorized compilation persistence is complete | `WS-POL-003-04A` is the next eligible dependency; it remains unstarted | | [WS-REV-001](initiatives/WS-REV-001-review-revision-lifecycle/STATUS.md) | Independent foundations through queue admission and reviewer-lease persistence are merged through `03A2`; schema/packet foundations may continue behind their own gates | Live admission/claim requires canonical 04E `allow_review`; ReviewLease copies the admitted Submission's immutable attempt policy version, and the first Review commit requires CON-03C/07 atomic contribution/award behavior | | [WS-QUAL-002](initiatives/WS-QUAL-002-behavior-ownership-catalogue/STATUS.md) | Catalogue foundation `01` and local context evidence `02` are merged through PRs #297 and #303 | Populate subsystem ownership through `03A`-`03D` before completeness or changed-line-aware mutation work | | [WS-XINT-002](initiatives/WS-XINT-002-art-auth-end-to-end/STATUS.md) | Guide and pre-submit materialization activation is merged | Continue only remaining ART and AUTH activation edges required by the artifact delivery path | diff --git a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/CHUNK_MAP.md b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/CHUNK_MAP.md index 5daff6b14..dc5c99701 100644 --- a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/CHUNK_MAP.md @@ -10,7 +10,7 @@ transient work, and no chunk starts automatically. | `WS-POL-003-02` | One `compile_project_guide` adapter method and fake-runtime proof. Merged PR #301. | 01 | | `WS-POL-003-03A` | Hidden immutable attempt/compilation schema, validator, repository, crash fence, and deny-by-default authorization seams. | 02 | | `WS-AUTH-001-12I` | Register and activate exact PM compilation request/recovery plus fixed-service compilation execute authority. | 03A exact resource/action manifest | -| `WS-POL-003-03B` | Consume 12I to make immutable compilation parent/result persistence usable; no policy projection or setup-service cutover. | 03A + AUTH-12I | +| `WS-POL-003-03B` | Complete authorized immutable compilation persistence; no policy projection or setup-service cutover. POL-04A is the next boundary. | 03A + AUTH-12I satisfied | | `WS-POL-003-04A` | Hidden one-attempt setup orchestrator over the complete result, with all three legacy inference methods denied/unreachable in the candidate call graph. | 03B | | `WS-AUTH-001-12B2` | Activate only setup-ledger mutation and its fixed-service adapter for the reviewed unified setup-service manifest. | 04A | | `WS-POL-003-04B` | Live one-call setup cutover; persist complete result, sufficiency, and artifact-policy projections; remove every legacy inference call from live reachability. | 04A + AUTH-12B2 | diff --git a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/DECISIONS.md b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/DECISIONS.md index a6057ca33..3d31988d5 100644 --- a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/DECISIONS.md +++ b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/DECISIONS.md @@ -2,9 +2,12 @@ 1. One logical structured model attempt is used per immutable source snapshot, catalogue snapshot, and setup generation. A durable attempt row and provider - idempotency key serialize dispatch. Retry/recovery uses that exact key and - retrieves or reuses its accepted result; it never creates another call. - Invalid or unsafe output consumes the attempt and blocks that generation. + idempotency key serialize dispatch. The current provider boundary cannot + prove retrieval or same-key replay after an unknown outcome, so recovery + remains `provider_outcome_unresolved` and never redispatches. A future + provider capability may retrieve or reuse the accepted result only after it + proves exact same-key observation. Invalid or unsafe output consumes the + attempt and blocks that generation. 2. `ProjectGuideCompilation` is immutable provenance/proposal evidence, not a canonical policy replacement. 3. Existing policy objects and Project Manager approval gates remain separate. @@ -29,7 +32,7 @@ 10. Representative task context is optional and bounded; its absence cannot block project guide compilation. 11. Setup failures, capability gaps, timeouts, and retries create no - ContributionRecord, payment, award, or negative reputation evidence. + ContributionRecord, settlement, award, or negative reputation evidence. 12. No backward-compatibility aliases or dual model-inference paths survive final cleanup. 13. Pre-submit has no standalone feedback/execution API. One canonical diff --git a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/STATUS.md b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/STATUS.md index 64ac67dd1..e2b776fde 100644 --- a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/STATUS.md +++ b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/STATUS.md @@ -45,10 +45,14 @@ before any is started, its contract must be expanded on then-current main with explicit allowed/not-allowed paths, runnable verification commands, and named reviewer tracks. They cannot authorize implementation in their current form. -`WS-POL-003-03A` merged through PR #307 at `5e459a8f`. It installs hidden -compilation custody and the first public AUTH-capability consumer proof. -AUTH-12I merged through PR #312 at `98eae13e`; POL-03B is now the next eligible -POL implementation boundary. +`WS-POL-003-03A` merged through PR #307 at `5e459a8f`, and AUTH-12I merged +through PR #312 at `98eae13e`. `WS-POL-003-03B` is complete. It installs the +hidden internal coordinator, migration 0008 request custody, SQL digest and +authorization trigger, explicit dispatch-versus-recovery receipts, +current-setup lineage checks, real-PostgreSQL concurrency and crash proof, and +semantic-lane registration. POL-04A is the next eligible dependency and has +not started. The open pull request remains transient until a human merges it; +protected main does not gain this behavior before that merge. WS-POL-003-08 is planned only after the canonical WS-ARCH-001-04E manifest and is not a prerequisite for WS-ARCH-001-03A. diff --git a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/chunks/WS-POL-003-03B-authorized-compilation-persistence.md b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/chunks/WS-POL-003-03B-authorized-compilation-persistence.md index 45bc2b058..d4ebfbf7c 100644 --- a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/chunks/WS-POL-003-03B-authorized-compilation-persistence.md +++ b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/chunks/WS-POL-003-03B-authorized-compilation-persistence.md @@ -1,34 +1,656 @@ # Chunk Contract: WS-POL-003-03B - Authorized Compilation Persistence -Status: Proposed after 03A and AUTH-12I; inactive. Risk: L1. +Status: Complete. +Risk: L1. -## Goal +## Merge state -Consume the exact compilation request/execute authorization adapters so PM -dispatch custody and fixed-service immutable compilation persistence are usable. +- Outcome on merge: `complete` + +## Outcome + +Make the merged hidden POL-03A compilation custody usable under the exact +merged AUTH-12I request and execute capabilities. The chunk adds one +PROJECTS-owned transactional coordinator that: + +1. atomically binds an authorized Project Manager request to the only attempt + for one setup generation; +2. gives the fixed `workstream.project.setup` service a short-transaction + execute preflight and commits a durable may-have-dispatched fence before any + later provider call; +3. records one known provider result under the original attempt and provider + key; and +4. consumes fresh, result-bound execute authority in the same root transaction + that inserts the immutable compilation and completes the attempt. + +The capability remains hidden. This chunk does not call a model, enqueue work, +add a route, project policy, approve a guide, or make setup live. + +## Authoritative starting point + +- Contract refresh base: `c716fa424c1a86bda9e0f85c77c307fa07172bca`. +- POL-03A prerequisite: merge commit `5e459a8f`; its hidden attempt, + compilation, validator, repository, and AUTH public facts are present. +- AUTH-12I prerequisite: merge commit `98eae13e`; the exact request/execute + catalogue entries and production authorization adapter are present. +- Current schema head: `0007_contribution_policy_publication_custody`. +- If schema custody described below is required, this chunk alone allocates + `0008_guide_compilation_authorized_persistence` with `down_revision` equal to + the current sole head. The same bounded change updates Alembic's existing + `_CURRENT_HEAD_REVISION` constant from 0007 to this exact 0008 identifier so + later runs recognize an already-migrated database. Implementation must stop + and re-plan if main changes the sole head before work starts. +- GitHub had no open pull request at contract refresh. Open pull requests, not + this document, remain authoritative for transient ownership. + +## Scope and ownership + +### PROJECTS/POL owns + +- the exact request-operation receipt and its database constraints; +- attempt reservation, state locking, result validation, terminal failure, + recovery classification, append-only compilation persistence, and lineage + compare-and-swap; +- the coordinator's root transaction and rollback behavior; and +- server-derived request facts, attempt identity, provider key, accepted result + hashes, and bounded receipts. + +The coordinator is internal to +`app.modules.projects.guide_compilation`. POL-03B adds **no public Projects +port** because it has no cross-module or delivery consumer. POL-04A, not this +chunk, owns the later asynchronous-execution Projects API and owner composition-root +wiring. Tests may construct the internal coordinator with the production AUTH +adapter; application delivery code may not import it yet. + +### AUTH owns + +- `ProjectGuideCompilationAuthorizationPort` and all public request, + preflight, and persist fact types in + `app.modules.authorization.api.project_guide_compilation`; +- Project Manager grant evaluation, fixed-service matrix evaluation, prepared + handle custody, and allowed/denied authority evidence; and +- the production adapter in `app.modules.authorization.guide_compilation`. + +POL consumes only that frozen public AUTH Protocol. It must not import an AUTH +model, repository, kernel, prepared service, concrete handle, concrete adapter, +or private resource context. This chunk changes no AUTH public method or fact +shape and no AUTH catalogue, evaluator, policy, runtime, service matrix, or +schema vocabulary. + +The POL-03A `DenyProjectGuideCompilationAuthorization` was an intentionally +temporary seam while AUTH-12I was unavailable. AUTH-12I is now merged, so 03B +deletes that class/module and replaces its old behavior test with a +syntax-aware assertion that no production or test consumer imports the retired +seam. It is not retained as an alias, fallback, or second adapter. + +### Composition boundary + +The coordinator constructor accepts the frozen AUTH public Protocol and a +caller-provided `AsyncSession`/session factory only as required by the exact +operation. It creates repositories inside its own root transaction. No +repository, session, prepared handle, authorization context, workspace, raw +guide, provider result, or model client is returned or serialized. + +There is no application composition-root change in this chunk. POL-04A must +compose the internal capability behind a typed Projects public API from +`app/adapters/projects/__init__.py`; asynchronous execution must consume that future API rather +than these private files. + +## Exact durable request-operation receipt + +POL-03A reserved attempts but deliberately did not persist the request +operation or the allowed request decision. AUTH-12I therefore cannot yet prove +that concurrent prepared request handles commit at most one mutation and one +allowed event. POL-03B closes only that gap with the exact +`project_guide_compilation_request_operations` table. + +Each immutable row contains only: + +- `operation_id` as the primary key; +- `request_id` and `idempotency_key`; +- requesting human `actor_profile_id` and `identity_link_id`; +- exact `project_id`, `guide_id`, `source_snapshot_id`, `setup_run_id`, and + positive `setup_generation` selectors; +- optional exact `expected_predecessor_compilation_id`; +- `request_facts_digest` from the merged public AUTH helper; +- the unique bound `attempt_id`; +- the exact allowed request `authorization_decision_event_id`; and +- database-owned `created_at`. + +Database uniqueness covers the operation ID, the actor-scoped request ID, the +actor-scoped idempotency key, and the attempt ID. Foreign keys bind the exact +project/guide/snapshot/setup/attempt identities. A database trigger validates +that the referenced audit event is allowed, uses action and permission +`project.guide_compilation.request`, resource type +`project_guide_compilation_request`, resource ID equal to `operation_id`, the +same project/actor/link, and a resource-context digest containing the exact +request-facts digest. The row is insert-only; update, delete, and truncate fail +closed. + +Migration 0008 installs two PROJECTS-owned, domain-specific SQL digest +functions: one reconstructs the exact no-whitespace, sorted-key UTF-8 preimage +used by `project_guide_compilation_facts_digest` from the operation and bound +attempt columns; the second reconstructs the exact preimage used by +`project_guide_compilation_request_authority_digest` from that facts digest, +the operation actor/link/project/operation values, and the audit event's +matched Project Manager grant. Both use PostgreSQL's built-in +`sha256(bytea)` plus `encode(..., 'hex')`; no extension or general digest +framework is added. The custody trigger first requires the stored facts digest +to equal the first function, then requires the event's +`after_facts.resource_context_digest` to equal the second. Golden parity tests +compare both SQL functions with the merged Python public helpers for null and +non-null predecessors and independently mutate every input. Thus PostgreSQL, +not an ORM or narrative assertion, proves the digest preimage and exact +identity-link/request-facts binding. + +This is not a generic operation framework. It has no JSON request body, status +machine, response cache, arbitrary action, arbitrary resource, delivery state, +or reusable outbox abstraction. + +## Transaction protocol + +Every method requires a session with no active transaction, opens exactly one +root transaction for its mutation, and lets that root context commit or roll +back. Repositories and AUTH never commit independently. + +### A. Authorized request or exact recovery + +Input is the authenticated human actor, complete +`ProjectGuideCompilationRequestFacts`, and the server-derived +`CompilationAttemptIdentity` whose values must match the facts exactly. + +1. Read an existing operation by operation ID and actor-scoped request and + idempotency identities. +2. If one committed row exists, compare every immutable selector and the + request-facts digest. Exact replay returns only its bounded attempt/provider + identifiers and current recovery classification. A changed or crossed + replay fails closed. Recovery creates no AUTH event, attempt, outbox row, or + external effect. +3. For a new operation, prepare request authority in the root transaction. +4. Reserve the exact attempt through the merged POL-03A repository. +5. Reject `mismatch`. Reject an `existing` attempt without the exact immutable + operation receipt; POL-03B must not adopt pre-authorized or legacy custody. +6. Consume the prepared request handle and insert the operation receipt bound + to the returned allowed decision-event UUID. +7. Commit the attempt, operation receipt, and allowed event atomically. + +If a concurrent identical caller wins after preparation, the losing root +transaction, including its tentative allowed event, rolls back. A fresh root +transaction then reloads and returns the winner's exact receipt. Constraint +errors are classified by exact constraint name; unknown storage errors remain +storage errors and are never reported as successful replay. + +An exact committed replay is receipt recovery, not a new privileged mutation. +It does not reauthorize or expose compilation content. Revocation is still +checked before any future dispatch fence or final persistence. + +### B. Fixed-service execute preflight and dispatch fence + +Input is the fixed service actor plus complete +`ProjectGuideCompilationExecutePreflightFacts`. + +1. Open a fresh root transaction and lock the attempt and its exact request + operation. +2. Rebuild facts from durable server-owned state and reject caller/fact drift, + stale setup/source/catalogue/agent lineage, missing request custody, wrong + predecessor, wrong attempt/key, wrong actor/link/service, or a state other + than `compilation_reserved`. +3. Call AUTH's non-evidencing `authorize_execute_preflight`. +4. Transition `compilation_reserved` to + `compilation_provider_uncertain` and commit. +5. Return only a bounded dispatch receipt containing operation, attempt, and + provider-idempotency UUIDs, the recovery classification, and an explicit + `dispatch_permitted=true` signal. + +An already-uncertain replay returns the same bounded identifiers with +`dispatch_permitted=false`. It performs no authorization or state change. A +caller may invoke the provider only when the committed receipt explicitly +permits dispatch. + +`compilation_provider_uncertain` is deliberately conservative: after the +commit, provider dispatch **may** have begun. A future caller may attempt the +external call only after receiving this committed receipt. A crash between the +commit and the call can strand the attempt, but cannot create a duplicate +provider attempt. Safety is preferred to silent redispatch. + +The current `ProjectGuideAgentRuntime.compile_project_guide` boundary is a +one-shot call: it accepts no application idempotency key and exposes no +retrieve/reconcile operation. The current OpenAI adapter therefore does not +prove application-level same-key replay across process failure. POL-03B must +not pretend otherwise. From `compilation_provider_uncertain`, this chunk +returns `provider_outcome_unresolved` and never allocates a new key or directs a +redispatch. Any future same-key reconciliation requires a separately reviewed +provider-port capability and proof before POL-04A may rely on it. + +### C. Known provider outcome custody + +POL-03B itself never calls the provider. After a future POL-04A caller has a +known result, it invokes one of two fixed-service methods in a fresh root +transaction: + +- `record_accepted_result` locks the exact uncertain attempt, rebuilds and + reauthorizes execute preflight, strictly validates the complete result + against the original context and hashes, and stores the accepted canonical + result using the existing POL-03A transition; or +- `record_invalid_result` locks the exact uncertain attempt, rebuilds and + reauthorizes execute preflight, maps only the existing allowlisted validation + failure codes, and records the existing terminal-invalid transition. + +No arbitrary provider error, text, trace, prompt, URL, path, credential, or +exception is durable. A transport timeout, connection loss, process crash, or +caller cancellation after the dispatch fence leaves the state uncertain; it +is not rewritten as invalid or retried. + +### D. Fresh final authorized persistence + +For `provider_result_accepted`: + +1. Open a new root transaction, lock the exact attempt and request operation, + and reload current setup/source and append-only predecessor lineage. +2. Revalidate the stored canonical result from its original strict context and + recompute every full/component hash. +3. Build complete `ProjectGuideCompilationExecutePersistFacts` from durable + values, including its canonical resource-context digest. +4. Prepare and consume fresh execute authority for the fixed service. +5. Insert the immutable compilation and transition the attempt to + `compilation_persisted` through the existing repository. +6. Commit the allowed execute event, compilation, and attempt transition + atomically. + +Concurrent final callers may prepare independently, but only the winner's root +transaction commits. The loser rolls back its event and returns the winner's +bounded persisted receipt only after a fresh reload verifies exact identity. +Exact persisted replay creates no second event or mutation and returns no raw +result. Stale predecessor, changed authority, changed result/hash, or lineage +drift denies with no allowed event or product write. + +## Closed invariants + +### Authority and tenant isolation + +- Only the covered active Project Manager human may create the request receipt. +- Only active `workstream.project.setup` may fence execution, record a known + outcome, or persist; humans and all other services deny before mutation. +- Actor, identity link, project, guide, source snapshot, setup run/generation, + catalogues, agent/instruction versions, operation/request/idempotency IDs, + attempt, provider key, predecessor, and result/component hashes cannot be + substituted or copied across a call. +- Request and final allowed events are durable only with their protected POL + mutations. Preflight remains non-evidencing. + +### Idempotency, replay, and recovery + +- One setup generation has one attempt, one provider key, one request receipt, + at most one request allowed event, and at most one final allowed event. +- Exact request replay returns the original receipt; changed reuse conflicts. +- Once dispatch is fenced, no path in this chunk returns to reserved or emits a + second dispatch permit. +- Accepted-not-persisted recovery performs persistence only and never provider + dispatch. Persisted and invalid-terminal states are terminal. +- Unknown provider outcome remains bounded and unresolved until a proven + provider reconciliation capability exists. + +### External I/O, crash, and cancellation + +- POL-03B has no model/provider/network call, callback, Celery task, broker + publish, or outbox write. +- No database transaction, row lock, prepared AUTH handle, session, workspace, + or mutable ORM object crosses future provider/model I/O. +- The dispatch fence commits before a later caller may touch the provider. +- Cancellation before commit rolls back. Cancellation after a committed fence + leaves the durable state uncertain and cannot trigger cleanup redispatch. +- Process restart recovers solely from PostgreSQL state; no process-local + handle is needed. + +### Rollback and forbidden effects + +- Denial, validation failure, mismatch, stale state, known constraint conflict, + unknown database error, or injected failure before commit leaves no partial + request event/receipt/attempt or final event/compilation/transition. +- There is no outbox event because this hidden chunk dispatches nothing. + Request authority is not authorization to publish a message. +- The chunk creates no approval, effective policy, policy projection, guide + activation, setup success, Submission, Review, contribution, reputation, + compensation, or settlement truth. ## Allowed files -Compilation request/service/repository/fixed-service execution composition, AUTH resource -adapter consumption, focused tests, specifications, and WS-POL-003 docs. +```text +backend/app/modules/projects/guide_compilation/__init__.py +backend/app/modules/projects/guide_compilation/authorization.py # delete merged deny-only seam +backend/app/modules/projects/guide_compilation/contracts.py +backend/app/modules/projects/guide_compilation/models.py +backend/app/modules/projects/guide_compilation/repository.py +backend/app/modules/projects/guide_compilation/service.py +backend/app/modules/projects/guide_compilation/validation.py +backend/app/db/models.py # metadata discovery only, if required +backend/alembic/versions/0008_guide_compilation_authorized_persistence.py +backend/alembic/env.py # only exact 0007 -> 0008 current-head constant +backend/tests/projects/guide_compilation/test_authorized_request_service.py +backend/tests/projects/guide_compilation/test_authorized_execution_service.py +backend/tests/projects/guide_compilation/test_request_operation_postgresql.py +backend/tests/projects/guide_compilation/test_authorized_recovery_postgresql.py +backend/tests/projects/guide_compilation/test_authorized_concurrency_postgresql.py +backend/tests/projects/guide_compilation/test_durable_dispatch_handoff.py +backend/tests/projects/guide_compilation/test_migration_authorized_persistence.py +backend/tests/projects/guide_compilation/test_migration_contract.py # exact current-head expectation only +backend/tests/projects/guide_compilation/test_public_authorization.py # exact boundary assertions only +backend/tests/authorization/guide_compilation/test_migration_contract.py # exact current-head expectation only +backend/tests/test_alembic.py # exact 0008 topology/fingerprint only +backend/tests/conftest.py # generic DB fixture plumbing only, if required +backend/scripts/run_test_lanes.py # exact new-test lane registration only +backend/tests/test_ci_test_lanes.py # exact lane inventory assertion only +backend/scripts/behavior_ownership.py # exact new callable ownership only, if required +backend/tests/test_behavior_ownership.py # exact ownership assertion only, if required +backend/scripts/test_structure_boundary.py # exact new-file scope only, if required +backend/tests/architecture/test_test_structure_boundary.py +.ci/behavior-ownership/partition.v1.json # exact new callable ownership only, if required +.ci/behavior-ownership/auth/** # exact authority behavior atoms only, if required +.ci/behavior-ownership/lifecycle/** # exact lifecycle behavior atoms only, if required +.github/workflows/backend.yml # exact materially-changed coverage gate only +.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/** +.agent-loop/CURRENT_STATE.md # exact 03B on-merge completion and next eligible dependency only +docs/operations_backend_testing.md # lane command only, if registration changes it +``` + +No file is implicitly allowed by a directory wildcard other than the initiative +documents and the two already-governed behavior-ownership assertion directories. +If implementation needs another production, test, schema, CI, generated, or +documentation file, stop and amend/re-review this contract first. + +### Atomic on-merge state synchronization + +The branch completion wording describes the state that will become true only +if this branch is merged. It must not say or imply that protected main already +contains the unmerged implementation. + +Before publication, exactly four projections must be synchronized in one +commit: + +1. this contract gains one `## Merge state` section containing exactly one + ``- Outcome on merge: `complete` `` declaration; +2. the sole `WS-POL-003-03B` row in `CHUNK_MAP.md` states `complete` or + `merged` without review-only or temporal wording; +3. `STATUS.md` contains the exact `WS-POL-003-03B` identifier and states the + same complete or merged outcome; and +4. `.agent-loop/CURRENT_STATE.md` contains the exact `WS-POL-003-03B` + identifier, states the same complete or merged outcome, and identifies only + the next dependency that becomes eligible from that outcome. + +The `CURRENT_STATE.md` edit may change only those exact POL-03B completion and +next-eligible-dependency facts. No unrelated initiative, capability, calendar, +priority, or status wording is admitted. The four projections must pass the +repository's atomic chunk-state gate together; none may land alone. + +## Prohibited changes + +- Provider/model calls, prompts, agent-adapter changes, new provider + idempotency claims, retries, polling, or fake reconciliation. +- Existing Celery task modules, Celery configuration, broker dispatch, outbox + creation/publication, route, API router, or live composition changes. +- A public Projects port, application composition root, public endpoint, live + setup cutover, or another execution path; these belong to POL-04A/04B. +- Policy projection or publication, sufficiency approval, pre/post approval, + effective policy, guide activation, setup-ledger success, or checker work. +- AUTH public fact/Protocol changes, AUTH catalogue/policy/evaluator/runtime/ + service-matrix changes, broad compile authority, admin bypass, service + inheritance, dynamic service authority, or another authorization adapter. +- A generic operation, workflow, idempotency, provider, outbox, or transaction + framework; compatibility aliases; dual writes; mutable-current flags; or a + second repository path. +- Imports from AUTH private modules or a new private cross-module dependency. +- Serialized or durable prepared AUTH handles, sessions, transactions, + workspaces, authorization contexts, ORM objects, or provider credentials. +- Raw guide duplication, raw provider output outside the existing bounded + canonical result, reasoning/traces, secret-bearing errors, paths, URLs, or + unbounded strings in rows, logs, receipts, or exceptions. +- ART, TASK, Submission, Review, revision, contribution, compensation, payment, + reputation, CP05, 02H hardening, or later POL behavior. +- Dependency additions, generated spreadsheet changes, threshold weakening, + skip/xfail/pass-with-no-tests paths, mocks as sole lifecycle evidence, or + changes to the current seven semantic-lane topology. +- Any `backend/alembic/env.py` change except replacing the single existing + `_CURRENT_HEAD_REVISION` value with + `0008_guide_compilation_authorized_persistence`. Baseline recognition, + unsupported-revision rejection, recreation guidance, transaction handling, + online/offline behavior, and all other migration policy remain byte-for-byte + unchanged. + +## Requirement -> risk -> test -> evidence matrix + +| Requirement | Primary risk | Required discriminating test | Canonical observable evidence | +|---|---|---|---| +| Atomic PM request custody | Event commits without operation/attempt, or reverse | Real-PostgreSQL injected failure before each flush/commit boundary | Either all three exact rows exist and cross-reference, or none exist; audit count is exact | +| Concurrent request idempotency | Two prepared handles commit two events or attempts | Two independent sessions race identical operation/request/key; repeat with changed facts | One attempt, one provider key, one operation receipt, one allowed event; changed replay has zero new rows | +| Audit binding | Borrowed/cross-tenant allowed event satisfies custody | Direct SQL attempts every actor/link/project/resource/action/digest substitution | Postgres rejects each substitution at the owning constraint/trigger; original rows remain unchanged | +| SQL/Python digest parity | SQL trigger hashes a different preimage and accepts or rejects the wrong receipt | Golden values plus per-field mutation for both exact SQL functions, including null/non-null predecessor | SQL and public Python helper digests are byte-for-byte equal; every mutation changes both identically | +| Exact replay | Retried request mutates state or leaks raw result | Replay every recovery state and mutate one fact at a time | Exact bounded receipt/classification only; zero new audit/outbox/product rows; mutations conflict | +| Execute authority before dispatch | Human/wrong service or stale lineage obtains a dispatch fence | Production AUTH adapter with real DB actors/grants/services; all-pairs substitutions | Only fixed service commits reserved -> uncertain; denials leave reserved and create no event/outbox | +| Committed pre-I/O fence | Provider could start while tx/lock/handle is live | Instrument session state and forbidden provider sentinel; subprocess exits immediately after receipt | Uncertain row is visible to a second process before any provider call; no active tx/handle/callback crosses boundary | +| No unsafe provider replay | Crash/cancel causes duplicate one-shot call | Restart from uncertain and invoke every recovery entry point | `provider_outcome_unresolved`, original key, zero provider calls and zero new dispatch permits | +| Known result acceptance | Raw/changed/unsafe output enters custody | Unit/property validation plus real-Postgres accepted and invalid transitions | Accepted canonical bytes and hashes match recomputation, or bounded terminal code; no raw error material | +| Fresh final authority | Revoked/stale service persists accepted output | Revoke each actor/link/service/action/matrix element after acceptance, before persist | No final event, compilation, or attempt transition | +| Atomic final persistence | Event, compilation, or terminal state commits alone | Inject failures after prepare, consume, insert, and transition; include cancellation | Exactly one complete triple or no effect; accepted result remains recoverable after rollback | +| Concurrent append-only finalization | Two roots/children or two final events commit | Independent-session race for root and expected predecessor | One immutable compilation and event; loser rolls back; stale fork rejected | +| Crash recovery | Restart repeats provider work or loses accepted result | Separate-process real-Postgres cases for reserved, uncertain, accepted, persisted, invalid | Closed classification and exact row counts; accepted recovery performs persistence only | +| Outbox/side-effect absence | Hidden request accidentally dispatches or projects | Snapshot outbox and all named later-product tables before/after success, denial, replay, crash | Zero outbox/broker/provider/policy/approval/setup/checker/contribution deltas | +| Database immutability | ORM bypass changes or deletes governed evidence | Direct SQL update/delete/truncate and non-empty downgrade probes | Postgres rejection with unchanged rows; downgrade refuses governed custody | +| Alembic current-head parity | Alembic environment, topology, PROJECTS custody, or AUTH custody retains 0007 as current after migration 0008 | Upgrade from baseline/current to 0008, run Alembic again at 0008, seed an unsupported revision, and run exact head assertions in the topology, PROJECTS, and AUTH migration contracts | `env.py`, the sole Alembic graph head, PROJECTS current-schema proof, and AUTH current-schema proof all report exact 0008; the second 0008 run is a no-op, unsupported revision retains the existing recreation failure, and only the current-head constant changed in env.py | +| Atomic on-merge state synchronization | One projection lands alone, contradicts another, or falsely claims protected main already contains the branch | Apply the four bounded completion projections in one commit, then run the canonical chunk-state gate from the fixed base | Contract, CHUNK_MAP, STATUS, and CURRENT_STATE agree on exact `WS-POL-003-03B` completion; only CURRENT_STATE's exact next eligible dependency is additionally changed | +| Module boundary | POL reaches AUTH private code or delivery reaches POL private code | Syntax-aware AUTH boundary and module reachability checks | Zero new private edges; no task/route/composition consumer | +| Test trust | Happy-path tests pass without exercising failure | Test-of-test mutations for dropped lock, skipped AUTH consume, removed rollback, redispatch from uncertain, weakened trigger | Each named test fails for its seeded defect and passes only after restoration | + +Every material assertion must name its exact SQL row/count, state transition, +decision-event ID, constraint/trigger, callback count, or absent downstream row. +Log text, mock invocation alone, test names, and narrative inspection are not +acceptance evidence. + +## Required proof suite + +### Unit, property, and contract proof + +- Fact-to-attempt and attempt-to-fact reconstruction, digest determinism, UUID + and bounded token/hash validation, exact mismatch classification, terminal + failure allowlist, result/component hash recomputation, and closed recovery + classification. +- Property tests mutate every bound field independently and prove a single + mutation denies rather than normalizes or aliases. +- Public AUTH Protocol conformance uses the merged production adapter; no mock + handle may stand in for final authority proof. + +### Real PostgreSQL proof + +- Migration upgrade, guarded downgrade, re-upgrade, sole-head topology, ORM + parity, exact constraints/triggers, direct SQL attack, insert-only custody, + transaction rollback, concurrent request, concurrent final persistence, + stale predecessor, and accepted-not-persisted recovery. +- Alembic preflight recognizes the unchanged baseline and exact 0008 current + head, a second run at 0008 is a no-op, and every unsupported revision still + raises the existing recreation guidance. A source assertion proves the + current-head constant is the only changed `env.py` line. +- The Alembic topology test and both existing domain migration-contract tests + must independently assert the exact + `0008_guide_compilation_authorized_persistence` head. The suite fails if the + environment preflight, graph topology, PROJECTS schema contract, or AUTH + schema contract disagrees with any other surface. +- Use the repository's digest-pinned PostgreSQL harness and independent + sessions. Do not add a second container abstraction or replace database + behavior with SQLite/mocks. +- The request and final happy paths must use real actor, link, grant/service, + AUTH adapter, audit, POL repository, and transaction composition. + +### Durable handoff/provider-boundary proof + +- A subprocess commits the dispatch fence and exits before any provider call; + a fresh process/session must observe uncertain and refuse redispatch. +- A cancellation at each await before commit proves rollback; cancellation + after the dispatch receipt proves the committed uncertain state survives. +- An instrumented provider sentinel must remain at zero calls throughout this + chunk. A test that merely omits a provider dependency is insufficient: the + coordinator surface and import graph must make provider invocation + unreachable. +- Actual provider invocation, Celery redelivery, and reconcile-by-key proof are + mandatory in POL-04A if that chunk introduces those effects; they are not + faked here. + +### Authorization and negative proof + +- PM request vs fixed-service execute separation; wrong actor kind, identity + link, grant, service identity, service matrix row, action, permission, tenant, + resource, generation, request/key, attempt, predecessor, and hashes. +- Revocation between request and dispatch, and between acceptance and final + persistence. +- Copied/replayed/wrong-session/wrong-transaction prepared handles through the + existing AUTH suite plus composed rollback proof here. +- No raw data, secret, reasoning, provider exception, or unbounded failure + enters audit, operation, attempt, compilation, receipt, or log output. + +### Coverage and semantic-lane custody + +- Register every new test module exactly once in the existing + `project_lifecycle` semantic lane and update its exact inventory assertion. +- Preserve the seven-lane fan-in, missing/duplicate-node failures, no-skip + policy, and all existing timeout/failure behavior. +- Repository-wide hosted coverage remains at least 78 percent. +- Each materially changed subsystem/file group is at least 90 percent: + `app/modules/projects/guide_compilation/*`, the exact new service/repository/ + model files, and any changed AUTH or owner-composition surface. No gate may be + weakened or satisfied by excluding changed files. + +## Exact verification commands + +Implementation must run these from repository root unless a command changes +directory explicitly: + +```bash +git diff --check origin/main +python3 scripts/check_markdown_links.py +python3 scripts/check_stale_workstream_wording.py +python3 scripts/check_chunk_state_sync.py \ + --base-ref c716fa424c1a86bda9e0f85c77c307fa07172bca + +cd backend +uv run ruff check app/modules/projects/guide_compilation app/db/models.py \ + scripts/run_test_lanes.py tests/projects/guide_compilation \ + tests/test_alembic.py tests/test_ci_test_lanes.py +uv run pytest -q tests/projects/guide_compilation \ + tests/architecture/test_authorization_boundary.py +uv run pytest -q tests/projects/guide_compilation \ + -p pytest_cov.plugin --cov=app.modules.projects.guide_compilation \ + --cov-branch --cov-report=term-missing --cov-fail-under=90 +uv run pytest -q tests/test_alembic.py tests/test_ci_test_lanes.py \ + tests/test_behavior_ownership.py \ + tests/architecture/test_test_structure_boundary.py +uv run python -m scripts.authorization_boundary validate \ + --ledger ../.agent-loop/initiatives/WS-AUTH-003-module-boundary-recovery/IMPORT_LEDGER.md +uv run python -m scripts.test_structure_boundary validate \ + --policy ../.agent-loop/initiatives/WS-AUTH-003-module-boundary-recovery/TEST_STRUCTURE_POLICY.md \ + --ledger ../.agent-loop/initiatives/WS-AUTH-003-module-boundary-recovery/TEST_STRUCTURE_DEBT.json +uv run python -m scripts.behavior_ownership validate + +lane_run_dir="$(mktemp -d)" +uv run python -m scripts.run_test_lanes \ + --metadata-dir "$lane_run_dir/metadata" \ + --summary-json "$lane_run_dir/summary.json" \ + --lane project_lifecycle +cd .. +``` + +Run the exact `project_lifecycle` lane through the repository lane runner after +registration. Full seven-lane semantic tests, combined coverage with +`coverage report --precision=2 --fail-under=78`, every accumulated subsystem +gate, Agent Gates, and review-evidence gates run on the exact pushed Phase 3 +head in GitHub Actions. A local focused pass does not replace hosted proof. + +## Phase 3 implementation evidence + +- The hidden coordinator now owns authorized request custody, the committed + dispatch fence, known-result custody, and accepted-result persistence without + a provider, background task, route, outbox, or public Projects API. +- Migration 0008 installs the immutable request-operation receipt, exact SQL + digest reconstruction, authorization-event custody trigger, and guarded + downgrade while preserving one Alembic head. +- Fifty-five guide-compilation tests pass against an isolated real PostgreSQL + database. They cover exact replay, concurrency, process restart, cancellation, + digest parity, immutable custody, guarded migration, and authorization-bound + request and persistence paths. +- Branch-aware coverage is 96.47 percent for the package. The materially changed + files are independently above the required floor: repository 94.31 percent, + service 97 percent, model 100 percent, contracts 98 percent, and validation + 93 percent. +- Authorization-boundary, test-structure, behavior-ownership, Alembic topology, + lane-inventory, and the exact 4,103-node `project_lifecycle` semantic lane pass + locally. Hosted seven-lane and combined-coverage gates remain verification + work, not claimed proof. + +## Stop and rollback conditions + +Stop implementation and amend/re-review this contract if: + +- the base, prerequisite behavior, sole migration head, frozen AUTH Protocol, + provider port, or semantic-lane topology changes; +- the 0008 implementation requires any Alembic environment change beyond the + exact current-head constant/parity update or weakens unsupported-revision + rejection and recreation policy; +- safe implementation requires a provider call, background task, route, public Projects + API, application composition root, outbox, policy projection, live setup + behavior, generic operation abstraction, dependency, or file outside scope; +- the current provider must be retried/reconciled from uncertain without a + proven same-key observation contract; +- exact request audit binding cannot be enforced by PostgreSQL without an AUTH + schema or public-contract change, or the exact SQL/Python digest parity proof + fails on any bound field; +- any transaction/lock/handle must cross external I/O, any handle/context must + be serialized, or rollback cannot remove the matching allowed event; +- a test can pass without its canonical effect/forbidden-effect assertion, a + required real-PostgreSQL boundary is mocked, or the 78/90 coverage floors + require weakening. + +Before merge, rollback is branch deletion. After migration is deployed, +downgrade is allowed only when the request-operation table is empty and no +request or execute compilation evidence/compilation custody would be orphaned. +Otherwise recovery is forward-only or requires a separately reviewed +retention/destructive-cleanup plan. -## Not allowed +## Required preimplementation reviewers -Policy projection writes, approval, live setup-service cutover, checker -execution, broad authority, handles in Celery, or transactions across I/O. +All nine repository reviewer definitions must independently inspect the exact +final contract head using `python3 scripts/review_target.py` at start and end, +replay prior findings, supply atomized traceability and a discriminating +test-of-the-test/residual-escape probe, and finish `PASS`: -## Acceptance +- architecture: owner/consumer/public-port matrix, transaction placement, + hidden composition, and absence of a competing protocol; +- reuse/dedup: reuse of AUTH-12I/POL-03A and rejection of generic operation, + provider, transaction, or outbox abstractions; +- security/authorization: actor/action/resource/state/failure/side-effect + substitutions, audit custody, replay, rollback, and data safety; +- QA: split acceptance atoms, real PostgreSQL concurrency/crash/recovery, and + forbidden-effect assertions; +- test delta: one primary invariant per test, test-of-test discrimination, + structure limits, and false-green resistance; +- senior engineering: implementability, failure taxonomy, session lifecycle, + cancellation, error classification, and simplicity; +- CI integrity: exact lane inventory, seven-lane fan-in, thresholds, commands, + and no weakened gate; +- product/operations: bounded recovery, operator-visible uncertainty, and no + false setup/approval/economic truth; and +- docs: current-main accuracy, terminology, links, commands, and clear deferred + ownership. -- PM request records only authorized dispatch/recovery custody and identifiers. -- `workstream.project.setup` independently authenticates as the fixed service and performs exact - pre-I/O admission, committed attempt reservation, same-key provider recovery, - and fresh final result-bound PREP before immutable persistence. -- Compilation/result/evidence commit atomically in the final transaction; - already-issued provider I/O is represented by durable recovery state. -- Replay/copy/stale/session/transaction/service/resource mismatches fail closed. +Any Critical, High, unresolved Medium, narrative-only proof row, dirty/moving +target, or non-PASS verdict blocks Phase 3. Every valid finding must be resolved +in this contract and all reviewers rerun against the unchanged final contract +head. -## Verification and review +## Human review focus -PREP all-pairs, provider uncertainty/recovery, crash, replay, rollback, and 90% -changed-subsystem coverage; all L1 tracks. Human focus: authorization around, -never across, external I/O. +- Is the conservative pre-I/O uncertain fence acceptable until the provider + port can prove reconciliation, including the possibility of a stranded call + that never started? +- Does one concurrent request/finalization commit exactly one matching allowed + event and product mutation while every loser rolls back? +- Is 03B still a hidden persistence boundary with zero provider, background task, + outbox, policy, approval, live setup, or economic behavior? +- Are all external-I/O and public delivery concerns left explicitly to POL-04A + rather than implied by this contract? diff --git a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/reviews/WS-POL-003-03B-implementation-review-evidence.md b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/reviews/WS-POL-003-03B-implementation-review-evidence.md new file mode 100644 index 000000000..00b452c17 --- /dev/null +++ b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/reviews/WS-POL-003-03B-implementation-review-evidence.md @@ -0,0 +1,284 @@ +# WS-POL-003-03B Implementation Review Evidence + +Date: 2026-08-21. Risk: L1. Outcome: PASS. + +## Review target and boundary + +- Authoritative base: `c716fa424c1a86bda9e0f85c77c307fa07172bca`. +- Phase 3 implementation head: `3403860ac4022fb9305fdea96ce62f4ec8289dbe`. +- Phase 4 remediation head: `fd4364a76a45610750d1132c9e1912d14e8741a8`. +- Final planning head: `400d55f004cc906bec6ae1c8e1cad5f3adfd031c`. +- Atomic completion-projection and Docker-test head: + `5062b64d02a0898fb38fd55fbdf08f147f143593`. +- Phase 5 external-review remediation code head: + `c9709e976e45d1b0ab588731c3b470440969d733`. +- Scope remains the hidden POL-03B persistence boundary. It adds no provider + call, worker dispatch, outbox message, route, public Projects API, setup + cutover, approval, policy projection, review, contribution, or economic + effect. + +Evidence below is labelled as **executed**, **inspected**, **hosted-only**, or +**deferred**. A test name or narrative claim is not treated as proof by itself. + +## Findings closed during Phase 4 + +| Finding | Resolution | Proof status | +|---|---|---| +| Seven stale human/worker authorization phrases | Replaced with the exact principal, actor-kind, identity-link, grant, fixed-service, action, permission, resource, and state language enforced by AUTH | Executed stale-authorization scanner PASS | +| Two pre-existing repository test files were touched outside the contract | Restored `test_repository_attempts.py` and `test_repository_persistence.py` byte-for-byte to the base; new proof lives only in admitted files | Executed `git diff --quiet` against both files | +| Digest proof could miss individual inputs | Real PostgreSQL now recomputes the SQL and Python facts digest while mutating all 24 request fields, then mutates all six authority inputs independently; each result remains byte-identical across implementations and differs from the base | Executed in `test_sql_and_python_request_digests_are_byte_identical` | +| A trigger could accept stale digest evidence even if helpers agree | Direct SQL changes the stored facts digest and authority digest separately; both trigger paths reject. Null and non-null predecessor inputs are also covered. | Executed real-PostgreSQL trigger tests | +| Request rollback did not directly assert the consumed AUTH event disappeared | An injected request-operation insert failure now proves attempt, request receipt, and matching allowed audit-event counts remain `(0, 0, 0)` | Executed in `test_request_failure_rolls_back_attempt_and_authority_event` | +| Exact-file coverage could be hidden by package averaging | Added meaningful missing-row, replay, immutable-custody, recovery, and error-path tests until each named changed surface exceeded 90 percent | Executed branch coverage: repository 94.31% at Phase 3 and 96.24% in the final semantic aggregate | +| Completion projections were not atomic | Contract, CHUNK_MAP, STATUS, and CURRENT_STATE were changed in one commit and distinguish the on-merge outcome from protected-main truth | Executed chunk-state gate plus 18 regression tests | +| An uncertain replay returned the original permit with no stateless dispatch distinction | `CompilationDispatchReceipt` now carries an explicit one-shot `dispatch_permitted` signal: the first committed fence returns `true`, while every uncertain replay returns `false` without touching AUTH or a provider | Executed real-PostgreSQL restart test plus a killed permit-replay mutant | +| Execution trusted setup lineage captured only at request time | Every dispatch, result-recording, and persistence transaction now locks and revalidates the exact draft guide, setup run, source snapshot, setup generation, and latest generation | Executed real-PostgreSQL failed, blocked, and superseded-lineage tests plus a killed guard-removal mutant | +| Completion wording still described a pending review outcome | The chunk contract and initiative status now describe POL-03B as complete while keeping merge and protected-main truth explicitly human-controlled | Executed stale-wording and atomic chunk-state checks | + +## Requirement, risk, test, and evidence + +| Requirement | Main escape risk | Discriminating proof | Evidence custody | +|---|---|---|---| +| Atomic authorized request | AUTH event, attempt, or receipt commits alone | Inject failure after AUTH consumption and query all three row counts | Executed, real PostgreSQL: `(0, 0, 0)` after rollback | +| Exact idempotency | Concurrent roots create duplicate request or final custody | Two independent sessions race request and finalization | Executed, real PostgreSQL: one attempt/event/receipt and one compilation/final event | +| SQL/Python digest parity | Trigger and application hash different preimages | Mutate 24 facts fields and six authority inputs one at a time | Executed, real PostgreSQL; all mutations discriminated | +| Immutable audit binding | Borrowed event or changed actor/action/resource satisfies custody | Direct SQL substitutions plus immutable update/delete/truncate probes | Executed, real PostgreSQL; owning trigger/constraint rejects | +| Committed pre-I/O fence | External work starts before durable uncertain state | Fresh subprocess reads the committed uncertain row and key; provider sentinel remains zero | Executed; second process observed exact row before any provider call | +| No redispatch from uncertainty | Restart duplicates a one-shot provider call | Fresh session invokes recovery with a fail-on-access authority/provider sentinel | Executed; returns `provider_outcome_unresolved`, sentinel calls `0` | +| Atomic final persistence | Final AUTH event, compilation, or attempt transition commits alone | Inject failure after final authority consumption and assert rollback, then retry | Executed, real PostgreSQL; no partial rows and accepted result remains recoverable | +| Closed recovery/status mapping | Unknown or repeated state is misreported | Exercise reserved, uncertain, accepted, persisted, invalid, repeated, and illegal transitions | Executed; only the closed classifications are returned | +| No later-product effects | Hidden persistence accidentally dispatches or projects | Import/call sentinel and before/after downstream-table counts | Executed and inspected; zero provider/outbox/setup/policy/approval/checker/contribution effects | +| Migration custody | 0008 drifts from the sole head or can remove governed evidence | Baseline/current/no-op/unsupported-head tests; update/delete/truncate and guarded downgrade | Executed in digest-pinned PostgreSQL 16 | +| Test sensitivity | Correct-looking tests pass after a key guard is removed | Six temporary seeded defects, restored before commit | Executed; all six mutants killed and clean code passed afterward | + +## Seeded-fault sensitivity + +Each mutation was applied temporarily to one bounded working copy, its exact +discriminating test was required to fail, the mutation was restored, and the +same test was required to pass. No mutant was committed. + +| Fault family | Seeded defect | Required observation | Result | +|---|---|---|---| +| Authorization | Skip or bypass final AUTH consumption | Final-authorization integration test fails before persistence | Killed, restored, PASS | +| Digest | Remove/change a request digest input | SQL/Python per-field parity or custody trigger test fails | Killed, restored, PASS | +| Locking/idempotency | Remove the locking/replay convergence guard | Independent-session race produces a failing cardinality assertion | Killed, restored, PASS | +| Rollback atomicity | Allow a request event to survive operation failure | Exact `(attempt, operation, event)` count assertion fails | Killed, restored, PASS | +| Status mapping | Map an uncertain/recovery state to the wrong public classification | Closed recovery matrix test fails | Killed, restored, PASS | +| No redispatch | Permit uncertain recovery to touch authority/provider | Zero-call sentinel raises immediately | Killed, restored, PASS | +| Dispatch permit replay | Return `dispatch_permitted=true` after the attempt is already uncertain | Restart test fails on the explicit one-shot permit assertion | Killed, restored, PASS | +| Current setup lineage | Remove the locked current-lineage recheck from execution | Failed, blocked, and newer-generation PostgreSQL tests all fail | Killed, restored, PASS | + +## Focused and structural verification + +Executed before the completion projection; production code did not change +afterward: + +- Focused guide-compilation/AUTH suite: 114 passed against real PostgreSQL. +- Migration, authorization-boundary, lane-inventory, behavior-ownership, and + test-structure suite: 184 passed. +- Package branch coverage: 96.47 percent. +- Named changed surfaces: contracts 97.94 percent, models 100 percent, + repository 94.31 percent, service 97.44 percent, and validation 92.59 percent. +- Ruff, migration head parity, Markdown links, stale Workstream wording, stale + authorization wording, stale artifact contracts, and diff integrity passed. + +At draft evidence head `2002dfcd4e2576de40632bc70f32d6ec7c70d85d`, +the same Docker/Linux focused suite passed 114 tests at 96.47 percent branch +coverage. The isolated migration/structure suite passed all 184 tests. + +The exact completion-projection head also passed: + +```text +python3 scripts/check_chunk_state_sync.py \ + --base-ref c716fa424c1a86bda9e0f85c77c307fa07172bca +python3 -m unittest -v scripts.test_chunk_state_sync +``` + +The first command passed and all 18 regression tests passed. + +### Phase 5 remediation verification + +At exact remediation code head +`c9709e976e45d1b0ab588731c3b470440969d733`: + +- Focused guide-compilation/AUTH suite: 118 passed against real PostgreSQL. +- Package branch coverage: 96.27 percent. +- Named changed surfaces all remained above the 90 percent floor. +- The real-PostgreSQL remediation subset passed 13 tests before the full + focused run. +- The migration, authorization-boundary, lane-inventory, behavior-ownership, + and test-structure suite passed all 184 tests. +- Both new mutants were killed, restored, and followed by passing clean-code + reruns: the replay-permit test passed once, and all four setup-lineage cases + passed. +- Docker build, Ruff, Markdown links, stale Workstream wording, stale + authorization wording, stale artifact contracts, stale review contracts, + and atomic chunk-state synchronization passed. + +## Phase 4 canonical Docker and semantic-lane proof + +The canonical run used native Linux, Python 3.12.13, and backend image manifest +`sha256:6abac3af7fdb493c334738a4adc183c2c81489637df9da5634f79661ab71133f`. +Only Git 2.39.5 was added to the ephemeral test container so the repository's +head-binding runner could execute. The source mount was read-only and bound to +head `5062b64d02a0898fb38fd55fbdf08f147f143593`. + +Services were real and digest pinned: + +- PostgreSQL 16: + `sha256:33f923b05f64ca54ac4401c01126a6b92afe839a0aa0a52bc5aeb5cc958e5f20`. +- Redis 7: + `sha256:b2b95679e3b46fb51864949ed25ea976fc3a6bcc00a40a1bc00d568cb2822e50`. +- MinIO: + `sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e`. + +Each canonical lane command ran exactly once, sequentially, in a fresh evidence +root. The independent merger and validator accepted the common manifest and +exact custody of all 4,107 nodes. + +| Lane | Collected | Completed | Skipped | Deselected | Exit | Seconds | +|---|---:|---:|---:|---:|---:|---:| +| shared_foundations_a | 1,540 | 1,540 | 0 | 0 | 0 | 187.027 | +| shared_foundations_b | 1,503 | 1,503 | 0 | 0 | 0 | 193.995 | +| schema_contracts_a | 73 | 73 | 0 | 0 | 0 | 27.728 | +| schema_contracts_b | 3 | 3 | 0 | 0 | 0 | 9.492 | +| schema_contracts_c | 7 | 7 | 0 | 0 | 0 | 16.654 | +| project_lifecycle | 526 | 526 | 0 | 0 | 0 | 308.267 | +| task_lifecycle | 455 | 455 | 0 | 0 | 0 | 262.958 | + +There were no duplicate completions, unexplained skips, deselections, +timeouts, cancellations, interruptions, or retries. Each isolation receipt +reported database and MinIO cleanup complete. + +Combined semantic coverage passed every enforced local floor: + +- Repository: 91.16 percent, floor 78 percent. +- Artifact foundation: 90.14 percent, floor 90 percent. +- Artifact module: 90.11 percent, floor 90 percent. +- Cancellation and file locks: 95.56 percent, floor 90 percent. +- Artifact interfaces: 99.69 percent, floor 90 percent. +- External-service interface: 100 percent, floor 90 percent. +- Final guide-compilation package: contracts 98.90 percent, models 100 percent, + repository 96.24 percent, service 98.46 percent, validation 95.24 percent. + +## Phase 5 canonical Docker and semantic-lane proof + +The remediation code head +`c9709e976e45d1b0ab588731c3b470440969d733` ran in a fresh, read-only Linux +source mount with Python 3.12.13 and tester image +`sha256:35878a30fdb8c0e0270dc857e6e06b95e8e7223bea0cca0785734398f44f8581`. +The same real PostgreSQL, Redis, and MinIO services were used. Lanes ran once, +sequentially, because concurrent local lane processes share infrastructure and +are not a supported local isolation mode. + +The independent merger and validator accepted one common manifest and exact +custody of all 4,111 nodes. + +| Lane | Collected | Completed | Skipped | Deselected | Exit | Seconds | +|---|---:|---:|---:|---:|---:|---:| +| shared_foundations_a | 1,539 | 1,539 | 0 | 0 | 0 | 184.172 | +| shared_foundations_b | 1,504 | 1,504 | 0 | 0 | 0 | 189.246 | +| schema_contracts_a | 73 | 73 | 0 | 0 | 0 | 28.645 | +| schema_contracts_b | 3 | 3 | 0 | 0 | 0 | 9.691 | +| schema_contracts_c | 7 | 7 | 0 | 0 | 0 | 14.984 | +| project_lifecycle | 530 | 530 | 0 | 0 | 0 | 279.915 | +| task_lifecycle | 455 | 455 | 0 | 0 | 0 | 231.770 | + +There were no duplicate completions, skips, deselections, interruptions, +timeouts, cancellations, or retries. Aggregate runner time was 938.423 +seconds, and each isolation receipt reported database and MinIO cleanup +complete. + +The independently combined Phase 5 coverage passed every required floor: + +- Repository: 91.18 percent, floor 78 percent. +- Guide-compilation package: 97.78 percent, floor 90 percent. +- `contracts.py`: 98.91 percent. +- `models.py`: 100 percent. +- `repository.py`: 95.98 percent. +- `service.py`: 98.47 percent. +- `validation.py`: 95.24 percent. + +### Invalid exploratory attempts + +Four earlier attempts are recorded only as failed operational evidence and +are not counted as product passes: + +1. An operator launched local lane runners in parallel against one shared + PostgreSQL/MinIO namespace. The shared-foundation bucket collided and an + isolation self-test observed another runner's database cleanup. This was + operator-induced parallel isolation interference. +2. An all-lane local invocation also starts lanes concurrently and reproduced + the same unsupported shared-MinIO collision. It was stopped and discarded. +3. A host macOS lane completed custody but failed Linux `/proc`-dependent + artifact tests. The contract requires Docker/Linux for this platform. The + first Docker preflight then rejected non-loopback service names before test + execution, as designed; the final container namespace used loopback for + both providers. +4. A later standalone migration/structure command omitted its runner-owned + target database and produced 174 passes plus 10 fixture setup errors. It was + not product evidence. The canonical isolated runner then supplied the owned + target and passed all 184 tests. + +None of these attempts was retried into or merged with the canonical green +bundle. Exact residual databases, roles, and buckets were ownership-checked +and removed before the fresh Linux run. + +## Phase 4 final reviewer verdicts + +All nine tracks reviewed clean draft evidence head +`2002dfcd4e2576de40632bc70f32d6ec7c70d85d`. Both reviewer groups ran +`python3 scripts/review_target.py` at start and end; base, head, merge-base, and +clean-worktree receipts matched, and `final_ready` was true. No finding +remained. After these receipts were written, all reviewers must rerun read-only +against the resulting unchanged final head. The durable file records the +verdicts without inventing a self-referential Git receipt. + +| Track | Verdict | Final focus | +|---|---|---| +| Architecture | PASS | One PROJECTS-owned hidden coordinator; exact root transactions; no public/provider/worker/outbox path | +| Reuse/dedup | PASS | Reuses AUTH-12I and POL-03A; the retired deny seam is deleted; no generic framework added | +| Security/authorization | PASS | Exact principal/action/resource/state binding, replay/revocation denial, SQL audit custody, and rollback | +| QA | PASS | Real PostgreSQL, independent-session races, recovery, exact row counts, and six fault probes | +| Test delta | PASS | Per-field digest mutations, meaningful assertions, stable inventory, and restored out-of-scope files | +| Senior engineering | PASS | Bounded failure taxonomy, short transactions, durable uncertainty, cancellation, and simple ownership | +| CI integrity | PASS | Exact 4,107-node custody, all floors, pinned Linux services, and invalid attempts excluded | +| Product/operations | PASS | Operator-visible unresolved state and no false provider/setup/approval/economic truth | +| Documentation | PASS | Accurate protected-main timing, terminology, links, commands, and deferred POL-04A ownership | + +## Phase 5 external-review remediation + +GitHub review identified three valid P1 gaps at published head +`6034e337acc15eaf46347c552c12fb9848854d01`: the dispatch receipt did not +distinguish a first permit from uncertain replay, execution did not revalidate +current setup lineage, and status wording retained a transient review state. +All three were corrected at code head +`c9709e976e45d1b0ab588731c3b470440969d733` and received the regression, +fault-sensitivity, full-lane, and coverage proof recorded above. + +The first exact-code-head re-review passed architecture, reuse/dedup, senior +engineering, documentation, and security/authorization. QA, test-delta, CI, +and product/operations correctly withheld final PASS because this durable file +still contained only the Phase 4 proof. This update closes that evidence gap. +After it is committed, all nine tracks must rerun read-only against the new +clean evidence head. Their immutable start/end receipts provide finality +without requiring this file to claim its own not-yet-created commit hash. + +## Residual and deferred proof + +- **Hosted-only:** GitHub Actions, branch protection, required checks, and the + exact remote SHA are recorded by the PR rather than predicted by this local + file. Local Docker proof uses the same lane and evidence validators but does + not claim hosted status. +- **Deferred to POL-04A:** actual provider execution, Celery delivery/redelivery, + reconcile-by-key observation, and private setup-service consumption. +- **Deferred to POL-04B and later:** public/live setup cutover, approvals, + policy projection, review, contribution, reputation, compensation, and + settlement behavior. +- The conservative uncertain state can strand work when it is unknown whether + a provider call began. It deliberately refuses redispatch until POL-04A + proves a same-key observation contract. + +Phase 5 publication is authorized only after every final reviewer returns PASS, +the final head is clean, and all affected deterministic gates still pass. This +document does not authorize push, pull-request creation, merge, or POL-04A. diff --git a/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/reviews/WS-POL-003-03B-preimplementation-review-evidence.md b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/reviews/WS-POL-003-03B-preimplementation-review-evidence.md new file mode 100644 index 000000000..365919dfa --- /dev/null +++ b/.agent-loop/initiatives/WS-POL-003-unified-project-guide-compilation/reviews/WS-POL-003-03B-preimplementation-review-evidence.md @@ -0,0 +1,214 @@ +# WS-POL-003-03B Preimplementation Review Evidence + +Date: 2026-08-21. Risk: L1. Runtime implementation: not started. + +## Review target + +- Authoritative base: `c716fa424c1a86bda9e0f85c77c307fa07172bca`. +- First contract head: `4a749b46aadb1207b367ebaf175b6945f3054cc8`. +- Remediated contract head: `f6d3fd265c08693d0a747df080f4240469ea9832`. +- Both reviewer groups ran `python3 scripts/review_target.py` at the start and + end of each review. The recorded base/head/merge-base matched, and the + worktree was clean. +- Reviewer receipts are read-only session evidence. This durable file records + their findings, dispositions, commands, and verdicts without fabricating a + self-referential Git receipt. + +## Findings and resolution + +| Finding | Severity | Resolution | Remediated result | +|---|---|---|---| +| PostgreSQL could not prove the request digest preimage from the original draft | High | The contract now requires two exact PROJECTS-owned SQL functions using built-in PostgreSQL SHA-256, byte-for-byte parity with the public Python helpers, null/non-null predecessor cases, and mutation of every bound input. The custody trigger compares both the facts and authority digests. | Closed | +| The obsolete POL-03A deny-only authorization seam could remain as a second adapter | Medium | The contract now requires deletion of the temporary seam and a syntax-aware test proving no production or test import remains. No alias or fallback is allowed. | Closed | +| The semantic-lane instruction lacked a runnable exact command | Medium | The command block now creates a task-local temporary directory and supplies the required metadata, summary, and `project_lifecycle` arguments. | Closed | +| Initiative decisions overstated provider recovery | Low | `DECISIONS.md` now records `provider_outcome_unresolved`, no redispatch, and a separate proof requirement before future same-key retrieval/reuse. | Closed | + +## Final reviewer verdicts on the remediated contract + +| Track | Verdict | Primary evidence focus | +|---|---|---| +| Architecture | PASS | Owner/consumer/public-port matrix, exact SQL digest custody, hidden composition, transaction and external-I/O boundaries | +| Reuse/dedup | PASS | Reuse of AUTH-12I and POL-03A, retired deny seam, no generic operation/provider/outbox framework | +| Security/authorization | PASS | Actor/action/resource/state substitutions, SQL/Python digest parity, audit/event rollback, replay, and data safety | +| QA | PASS | Atomic effects, real-PostgreSQL concurrency, crash/recovery, exact row counts, and forbidden-effect absence | +| Test delta | PASS | Per-field and seeded-defect probes, syntax-aware seam removal, one primary invariant, no mock-only lifecycle evidence | +| Senior engineering | PASS | Feasible PostgreSQL 16 design, failure taxonomy, short root transactions, conservative provider uncertainty, runnable commands | +| CI integrity | PASS | Exact project-lifecycle registration, seven-lane fan-in, 78/90 coverage floors, and no weakened gate | +| Product/operations | PASS | Operator-visible unresolved state and no false setup, approval, guide, review, or economic truth | +| Documentation | PASS | Current-main decisions, exact command truth, terminology, Markdown links, and deferred POL-04A ownership | + +Each track supplied atomized requirement/owner/proof traceability, distinguished +executed from inspected evidence, replayed its prior findings, and stated a +residual escape hypothesis with a discriminating test-of-the-test probe. No +Critical, High, Medium, Low, or Informational finding remained after the +remediated review. + +## Executed evidence + +```text +python3 scripts/review_target.py --base c716fa424c1a86bda9e0f85c77c307fa07172bca --head +git diff --check c716fa424c1a86bda9e0f85c77c307fa07172bca.. +python3 scripts/check_markdown_links.py +python3 scripts/check_stale_workstream_wording.py +``` + +Reviewers also executed bounded static probes for AUTH/POL ownership, current +deny-seam references, provider/worker/route reachability, the lane-runner CLI, +seven-lane inventory, and coverage gates. They inspected the merged AUTH public +facts/adapter, POL-03A repository/model/validator, audit schema, current +migration head, initiative state, and pinned PostgreSQL 16 runtime. + +No application runtime, migration, test, CI workflow, dependency, worker, +provider, route, generated artifact, push, pull request, or merge was created +during this planning review. + +## Repair 1: Alembic head-parity scope + +The first Phase 3 drift check stopped cleanly before edits. The implementation +contract allocated migration 0008, while `backend/alembic/env.py` recognized +only 0007 and was absent from the allowed files. Leaving it unchanged would +make a later Alembic run reject a database already migrated to 0008. + +The contract amendment permits only replacement of the existing +`_CURRENT_HEAD_REVISION` value from 0007 to the exact 0008 revision. It also +requires proof that baseline and exact current-head recognition still work, a +second 0008 run is a no-op, unsupported revisions retain the existing +recreation failure, and every other `env.py` line remains unchanged. Dynamic +head discovery and any migration-policy change remain prohibited. + +Repair 1 exact-head reviewer verdicts: + +| Track | Verdict | Repair-specific conclusion | +|---|---|---| +| Architecture | PASS | One existing static head constant remains the sole mechanism; no dynamic discovery or second head path | +| Reuse/dedup | PASS | Reuses the current Alembic preflight and existing migration test owner | +| Security/authorization | PASS | No AUTH, transaction, provider, route, worker, or product authority boundary changes | +| QA | PASS | Baseline, 0008 current head, second-run no-op, unsupported revision, and source-only-change proofs are discriminating | +| Test delta | PASS | Exact test scope adds no skip, weakening, mock-only proof, or lane-topology change | +| Senior engineering | PASS | One-line runtime parity change is implementable and preserves recreation policy | +| CI integrity | PASS | Existing Alembic/lane/coverage gates remain authoritative and unchanged | +| Product/operations | PASS | Planning-only repair creates no live, provider, approval, setup, or economic truth | +| Documentation | PASS | Status and evidence accurately describe the stopped attempt and bounded amendment | + +Both reviewer groups ran the repository target-integrity command at start and +end against planning head `2937c566c711a9395f483fc8f70b2ce9bfb5da24`; +base/head/merge-base matched and the worktree remained clean. Each track +reported no findings and supplied a residual escape hypothesis with a +discriminating probe. + +No runtime, migration, test, CI, dependency, or Alembic environment file was +changed by this planning repair. + +## Repair 2: Complete hard-coded-head scan + +Phase 3 Retry 1 stopped cleanly before edits when the two existing domain +migration-contract tests were found to assert 0007 as the current schema head +while remaining outside the allowed list. Repair 2 permits only replacement of +those two current-head expectations with exact 0008. It does not authorize test +refactoring, broader migration cleanup, or any other change in those files. + +The complete tracked-repository scan used exact-literal and generic +head-variable searches across executable checks, fixtures, scripts, workflows, +and documentation. Every exact 0007 match is classified below. + +| Classification | Exact surfaces | Disposition | +|---|---|---| +| Needs current-head parity update; already admitted | `backend/alembic/env.py`; `backend/tests/test_alembic.py` | Replace only the current-head value with exact 0008 and retain existing policy/topology behavior | +| Needs current-head parity update; admitted by Repair 2 | `backend/tests/projects/guide_compilation/test_migration_contract.py`; `backend/tests/authorization/guide_compilation/test_migration_contract.py` | Replace only each current-schema expected revision with exact 0008 | +| Intentionally historical migration identity | `backend/alembic/versions/0007_contribution_policy_publication_custody.py` | Keep the immutable revision identifier unchanged; migration 0008 points back to it | +| Intentionally historical planning and handoff record | `WS-ARCH-001-CP04B-con-policy-publication-behavior.md` (two matches); `WS-CON-001/AUTHORIZATION_HANDOFF.md` | Keep as the reviewed CP04B delivery record and pre-0008 handoff snapshot | +| Intentional POL-03B starting-point and repair history | This chunk contract (current starting head plus 0007-to-0008 bounds); this review evidence | Keep as provenance explaining the required transition | + +Generic searches found no additional executable sole-head literal. The +isolated-test runner discovers the sole Alembic head from the migration graph; +coverage/evidence validators compare caller-supplied or runner-observed values; +workflow and script calls to `alembic upgrade head` follow the graph; and older +initiative documents name their then-current revisions as historical evidence. +None requires a parity edit. No additional production, test, fixture, script, +workflow, or documentation surface is admitted. + +The amended proof now fails if any of these four current-head authorities +disagrees: `backend/alembic/env.py`, the sole Alembic graph asserted through +`backend/tests/test_alembic.py`, the PROJECTS migration contract, or the AUTH +migration contract. + +Repair 2 exact-head reviewer verdicts on planning head +`58a788dbe5aa83f4d7961705919db50cfc0fcc98`: + +| Track | Verdict | Repair-specific conclusion | +|---|---|---| +| Architecture | PASS | The four current-head authorities must agree on exact 0008; immutable historical revision identities remain unchanged | +| Reuse/dedup | PASS | Existing preflight, topology, and domain migration-contract owners are reused; no second head mechanism or abstraction is added | +| Security/authorization | PASS | AUTH behavior and its public Protocol remain frozen; only its existing current-schema test expectation is admitted | +| QA | PASS | Exact and generic scans classify every executable 0007 parity surface; disagreement among any of the four authorities must fail | +| Test delta | PASS | Only the two named existing domain tests are newly admitted, solely for exact current-head expectations, with no self-derived oracle | +| Senior engineering | PASS | The bounded changes are implementable and preserve sole-head topology, unsupported-revision rejection, and source-only env.py custody | +| CI integrity | PASS | Existing lane ownership, inventory, seven-lane fan-in, and coverage gates remain unchanged and authoritative | +| Product/operations | PASS | The repair creates no live behavior, dispatch, setup, approval, policy, contribution, or economic truth | +| Documentation | PASS | Status, scan classification, implementation state, and stop/re-review boundary are accurate | + +Both reviewer groups ran the repository target-integrity command at start and +end; exact base, head, and merge-base matched and the worktree remained clean. +Each track replayed prior valid closures and supplied atomized +requirement/owner/proof traceability plus a residual escape hypothesis with a +discriminating test-of-the-test probe. No finding remained. + +No runtime, migration, test, fixture, script, workflow, CI, dependency, or +application file was changed by Repair 2. + +## Repair 3: Atomic chunk-state publication scope + +Phase 4 stopped cleanly after the canonical chunk-state gate found that a +completed implementation branch must update four state projections atomically. +The contract, initiative `CHUNK_MAP.md`, and initiative `STATUS.md` already fall +under the initiative wildcard. Root `.agent-loop/CURRENT_STATE.md` did not. + +Repair 3 admits only the root current-state edit needed to state the exact +`WS-POL-003-03B` on-merge completion outcome and the next dependency made +eligible by that outcome. It also requires the future publication commit to +synchronize exactly four projections: + +- one unique contract merge-state declaration with outcome `complete`; +- the sole `WS-POL-003-03B` CHUNK_MAP row with a complete or merged outcome; +- STATUS with the exact chunk identifier and the same outcome; and +- CURRENT_STATE with the exact chunk identifier, the same outcome, and only + the next eligible dependency. + +This wording describes the intended state if the branch merges. It does not +claim that protected main already contains the unmerged implementation. No +unrelated initiative, capability, calendar, priority, or status edit is +allowed. The canonical `scripts/check_chunk_state_sync.py` command must pass +after all four projections are applied together. + +No completion projection, runtime, migration, test, CI, dependency, +application, or other current-state change was made by this planning repair. +Phase 4 remains paused until all nine reviewer tracks pass at the exact amended +contract head. + +Repair 3 first exact-head review verdicts on planning head +`9af056c4ed83581ca5a55fa97f15ed352cc4eb9b`: + +| Track | Verdict | Repair-specific conclusion | +|---|---|---| +| Architecture | PASS | One canonical gate owns the four-projection atomic rule; the root projection scope is exact and bounded | +| Reuse/dedup | PASS | The existing chunk-state gate is reused; no second checklist, script, or projection mechanism is introduced | +| Security/authorization | PASS | The amendment changes no authority or runtime behavior and permits no unrelated current-state capability claim | +| QA | PASS | Missing, contradictory, temporal, or partial completion projections are rejected by the canonical gate | +| Test delta | PASS | Existing chunk-state regressions discriminate missing, duplicate, temporal, review-only, and mismatched outcomes | +| Senior engineering | PASS | The rule is narrow, implementable in one publication commit, and stops unrelated state cleanup | +| CI integrity | PASS | The exact canonical command and fixed base are recorded without changing or weakening CI | +| Product/operations | PASS | Pre-merge intent is distinct from protected-main truth; no setup, approval, policy, or economic state is invented | +| Documentation | PASS | Contract and evidence state the same four projections, completion semantics, and deferred application step | + +Both reviewer groups ran `scripts/review_target.py` at the start and end. The +exact base, head, and merge-base matched, the worktree remained clean, and no +finding remained. Each track supplied atomized traceability and a residual +probe covering partial projection, temporal wording, changed base, unrelated +CURRENT_STATE edits, or false downstream product truth. + +## Phase 3 gate + +Phase 3 is authorized only if all exact-final-head reviewer receipts remain +PASS and the contract's base, prerequisites, sole migration head, frozen AUTH +Protocol, provider limitation, and allowed scope still match. Any drift invokes +the contract's stop-and-re-review rule. diff --git a/.ci/behavior-ownership/lifecycle/project-guide-compilation-authorization.json b/.ci/behavior-ownership/lifecycle/project-guide-compilation-authorization.json deleted file mode 100644 index 40995c01a..000000000 --- a/.ci/behavior-ownership/lifecycle/project-guide-compilation-authorization.json +++ /dev/null @@ -1,21 +0,0 @@ -{ - "behavior_id": "lifecycle.project_guide_compilation.authorization", - "boundaries": [], - "callables": [ - "app.modules.projects.guide_compilation.authorization.DenyProjectGuideCompilationAuthorization._deny", - "app.modules.projects.guide_compilation.authorization.DenyProjectGuideCompilationAuthorization.authorize_execute_preflight", - "app.modules.projects.guide_compilation.authorization.DenyProjectGuideCompilationAuthorization.consume_execute_persist", - "app.modules.projects.guide_compilation.authorization.DenyProjectGuideCompilationAuthorization.consume_request", - "app.modules.projects.guide_compilation.authorization.DenyProjectGuideCompilationAuthorization.prepare_execute_persist", - "app.modules.projects.guide_compilation.authorization.DenyProjectGuideCompilationAuthorization.prepare_request" - ], - "group": "lifecycle", - "outcomes": ["denial"], - "reviewed_by": ["WS-POL-003-03A required reviewers"], - "schema": "workstream.behavior-ownership.v1", - "status": "reviewed", - "target": "backend/app/modules/projects/guide_compilation/authorization.py", - "tests": [ - "backend/tests/projects/guide_compilation/test_public_authorization.py::test_hidden_authorization_denies_before_touching_product_state" - ] -} diff --git a/.ci/behavior-ownership/lifecycle/project-guide-compilation-repository.json b/.ci/behavior-ownership/lifecycle/project-guide-compilation-repository.json index 9babb1d0c..67d603787 100644 --- a/.ci/behavior-ownership/lifecycle/project-guide-compilation-repository.json +++ b/.ci/behavior-ownership/lifecycle/project-guide-compilation-repository.json @@ -9,18 +9,26 @@ "app.modules.projects.guide_compilation.repository.GuideCompilationRepository._required_attempt", "app.modules.projects.guide_compilation.repository.GuideCompilationRepository._transition", "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.accept_result", + "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.attempt", + "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.current_compilation", + "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.insert_request_operation", "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.mark_invalid_terminal", "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.mark_provider_uncertain", + "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.matching_request_operation", "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.persist_accepted", + "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.persisted_compilation", "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.recovery_classification", + "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.request_operation_for_attempt", "app.modules.projects.guide_compilation.repository.GuideCompilationRepository.reserve_attempt", "app.modules.projects.guide_compilation.repository._compilation_identity_values", "app.modules.projects.guide_compilation.repository._identity_values", - "app.modules.projects.guide_compilation.repository._matches" + "app.modules.projects.guide_compilation.repository._matches", + "app.modules.projects.guide_compilation.repository._persistence_error", + "app.modules.projects.guide_compilation.repository._request_matches" ], "group": "lifecycle", "outcomes": ["persisted_state", "idempotent_replay", "recovery_outcome", "mapped_error"], - "reviewed_by": ["WS-POL-003-03A required reviewers"], + "reviewed_by": ["WS-POL-003-03B required reviewers"], "schema": "workstream.behavior-ownership.v1", "status": "reviewed", "target": "backend/app/modules/projects/guide_compilation/repository.py", @@ -30,6 +38,8 @@ "backend/tests/projects/guide_compilation/test_repository_attempts.py::test_uncertain_to_invalid_terminal_preserves_one_attempt", "backend/tests/projects/guide_compilation/test_repository_persistence.py::test_accepted_crash_recovery_persists_exactly_once", "backend/tests/projects/guide_compilation/test_repository_persistence.py::test_stale_predecessor_fails_closed", - "backend/tests/projects/guide_compilation/test_repository_persistence.py::test_compilation_update_is_rejected" + "backend/tests/projects/guide_compilation/test_repository_persistence.py::test_compilation_update_is_rejected", + "backend/tests/projects/guide_compilation/test_request_operation_postgresql.py::test_request_operation_rejects_every_change", + "backend/tests/projects/guide_compilation/test_authorized_concurrency_postgresql.py::test_concurrent_identical_requests_commit_one_attempt_event_and_receipt" ] } diff --git a/.ci/behavior-ownership/lifecycle/project-guide-compilation-service.json b/.ci/behavior-ownership/lifecycle/project-guide-compilation-service.json new file mode 100644 index 000000000..6179c8ce3 --- /dev/null +++ b/.ci/behavior-ownership/lifecycle/project-guide-compilation-service.json @@ -0,0 +1,48 @@ +{ + "behavior_id": "lifecycle.project_guide_compilation.service", + "boundaries": ["postgresql", "lock", "trigger", "concurrency"], + "callables": [ + "app.modules.projects.guide_compilation.service.GuideCompilationService.__init__", + "app.modules.projects.guide_compilation.service.GuideCompilationService._recover_request", + "app.modules.projects.guide_compilation.service.GuideCompilationService._require_fresh_session", + "app.modules.projects.guide_compilation.service.GuideCompilationService.authorize_request", + "app.modules.projects.guide_compilation.service.GuideCompilationService.fence_dispatch", + "app.modules.projects.guide_compilation.service.GuideCompilationService.persist_accepted", + "app.modules.projects.guide_compilation.service.GuideCompilationService.record_accepted_result", + "app.modules.projects.guide_compilation.service.GuideCompilationService.record_invalid_result", + "app.modules.projects.guide_compilation.service._dispatch_receipt", + "app.modules.projects.guide_compilation.service._locked_exact", + "app.modules.projects.guide_compilation.service._outcome_receipt", + "app.modules.projects.guide_compilation.service._persist_facts", + "app.modules.projects.guide_compilation.service._persisted_receipt", + "app.modules.projects.guide_compilation.service._preflight_facts", + "app.modules.projects.guide_compilation.service._request_facts", + "app.modules.projects.guide_compilation.service._request_receipt", + "app.modules.projects.guide_compilation.service._require_identity_matches" + ], + "group": "lifecycle", + "outcomes": [ + "return", + "persisted_state", + "emitted_fact", + "denial", + "mapped_error", + "idempotent_replay", + "recovery_outcome" + ], + "reviewed_by": ["WS-POL-003-03B required reviewers"], + "schema": "workstream.behavior-ownership.v1", + "status": "reviewed", + "target": "backend/app/modules/projects/guide_compilation/service.py", + "tests": [ + "backend/tests/projects/guide_compilation/test_authorized_request_service.py::test_authorized_request_commits_one_bound_receipt_and_exact_replay", + "backend/tests/projects/guide_compilation/test_authorized_execution_service.py::test_authorized_execution_fences_accepts_and_persists_atomically", + "backend/tests/projects/guide_compilation/test_authorized_recovery_postgresql.py::test_uncertain_restart_returns_unresolved_without_redispatch", + "backend/tests/projects/guide_compilation/test_authorized_recovery_postgresql.py::test_changed_request_replay_fails_without_new_authority_event", + "backend/tests/projects/guide_compilation/test_authorized_concurrency_postgresql.py::test_concurrent_identical_requests_commit_one_attempt_event_and_receipt", + "backend/tests/projects/guide_compilation/test_authorized_concurrency_postgresql.py::test_concurrent_finalization_commits_one_compilation_and_event", + "backend/tests/projects/guide_compilation/test_durable_dispatch_handoff.py::test_fence_is_visible_to_a_fresh_process_before_any_provider_call", + "backend/tests/projects/guide_compilation/test_durable_dispatch_handoff.py::test_cancellation_before_fence_commit_leaves_attempt_reserved", + "backend/tests/projects/guide_compilation/test_durable_dispatch_handoff.py::test_coordinator_cannot_import_or_call_a_provider" + ] +} diff --git a/.ci/behavior-ownership/partition.v1.json b/.ci/behavior-ownership/partition.v1.json index 9e67dee5f..147b13315 100644 --- a/.ci/behavior-ownership/partition.v1.json +++ b/.ci/behavior-ownership/partition.v1.json @@ -688,10 +688,6 @@ "group": "lifecycle", "target": "backend/app/modules/projects/create_service.py" }, - { - "group": "lifecycle", - "target": "backend/app/modules/projects/guide_compilation/authorization.py" - }, { "group": "lifecycle", "target": "backend/app/modules/projects/guide_compilation/contracts.py" @@ -704,6 +700,10 @@ "group": "lifecycle", "target": "backend/app/modules/projects/guide_compilation/repository.py" }, + { + "group": "lifecycle", + "target": "backend/app/modules/projects/guide_compilation/service.py" + }, { "group": "lifecycle", "target": "backend/app/modules/projects/guide_compilation/validation.py" @@ -949,7 +949,7 @@ "target": "backend/scripts/week2_api_e2e.py" } ], - "authority_digest": "495ba32b1f2baa63622cd0bd48bc4ac3fa8e70c3208d05231673a451aa72f34c", + "authority_digest": "3feaa14f51b9d5fc1820104ea1463cd33949bed3d2da1e96fd8a9a96a73316b1", "protected_base_commit": "7676ce4347db0c9694962a9b587a20765e16eac6", "schema": "workstream.behavior-ownership-partition.v1" } diff --git a/backend/alembic/env.py b/backend/alembic/env.py index 309eb6439..845eaea81 100644 --- a/backend/alembic/env.py +++ b/backend/alembic/env.py @@ -21,7 +21,7 @@ target_metadata = Base.metadata _BASELINE_REVISION = "0001_v01_baseline" -_CURRENT_HEAD_REVISION = "0007_contribution_policy_publication_custody" +_CURRENT_HEAD_REVISION = "0008_guide_compilation_authorized_persistence" _RECREATE_GUIDANCE = ( "Workstream v0.1 requires a fresh database; recreate this database before " "running the 0001_v01_baseline migration" diff --git a/backend/alembic/versions/0008_guide_compilation_authorized_persistence.py b/backend/alembic/versions/0008_guide_compilation_authorized_persistence.py new file mode 100644 index 000000000..9830e9af4 --- /dev/null +++ b/backend/alembic/versions/0008_guide_compilation_authorized_persistence.py @@ -0,0 +1,318 @@ +"""Install authorized project-guide compilation request custody.""" + +from alembic import op +import sqlalchemy as sa + +revision = "0008_guide_compilation_authorized_persistence" +down_revision = "0007_contribution_policy_publication_custody" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.create_unique_constraint( + "uq_compilation_attempt_exact_request", + "project_guide_compilation_attempts", + [ + "id", + "project_id", + "guide_id", + "source_snapshot_id", + "setup_run_id", + "setup_generation", + ], + ) + op.create_table( + "project_guide_compilation_request_operations", + sa.Column("operation_id", sa.Uuid(), primary_key=True), + sa.Column("request_id", sa.Uuid(), nullable=False), + sa.Column("idempotency_key", sa.Uuid(), nullable=False), + sa.Column("actor_profile_id", sa.String(36), nullable=False), + sa.Column("identity_link_id", sa.String(36), nullable=False), + sa.Column("project_id", sa.String(36), nullable=False), + sa.Column("guide_id", sa.String(36), nullable=False), + sa.Column("source_snapshot_id", sa.String(36), nullable=False), + sa.Column("setup_run_id", sa.String(36), nullable=False), + sa.Column("setup_generation", sa.BigInteger(), nullable=False), + sa.Column("expected_predecessor_compilation_id", sa.Uuid()), + sa.Column("request_facts_digest", sa.String(71), nullable=False), + sa.Column("attempt_id", sa.Uuid(), nullable=False), + sa.Column("authorization_decision_event_id", sa.String(36), nullable=False), + sa.Column( + "created_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.ForeignKeyConstraint( + ["actor_profile_id"], + ["actor_profiles.id"], + name="fk_compilation_request_actor", + ), + sa.ForeignKeyConstraint( + ["identity_link_id", "actor_profile_id"], + ["actor_identity_links.id", "actor_identity_links.actor_profile_id"], + name="fk_compilation_request_actor_link", + ), + sa.ForeignKeyConstraint( + ["project_id"], ["projects.id"], name="fk_compilation_request_project" + ), + sa.ForeignKeyConstraint( + ["guide_id"], ["project_guides.id"], name="fk_compilation_request_guide" + ), + sa.ForeignKeyConstraint( + ["source_snapshot_id", "project_id", "guide_id"], + [ + "guide_source_snapshots.id", + "guide_source_snapshots.project_id", + "guide_source_snapshots.guide_id", + ], + name="fk_compilation_request_snapshot", + ), + sa.ForeignKeyConstraint( + [ + "setup_run_id", + "project_id", + "guide_id", + "source_snapshot_id", + "setup_generation", + ], + [ + "project_setup_runs.id", + "project_setup_runs.project_id", + "project_setup_runs.guide_id", + "project_setup_runs.source_snapshot_id", + "project_setup_runs.setup_generation", + ], + name="fk_compilation_request_setup", + ), + sa.ForeignKeyConstraint( + [ + "attempt_id", + "project_id", + "guide_id", + "source_snapshot_id", + "setup_run_id", + "setup_generation", + ], + [ + "project_guide_compilation_attempts.id", + "project_guide_compilation_attempts.project_id", + "project_guide_compilation_attempts.guide_id", + "project_guide_compilation_attempts.source_snapshot_id", + "project_guide_compilation_attempts.setup_run_id", + "project_guide_compilation_attempts.setup_generation", + ], + name="fk_compilation_request_exact_attempt", + ), + sa.ForeignKeyConstraint( + ["expected_predecessor_compilation_id", "project_id", "guide_id"], + [ + "project_guide_compilations.id", + "project_guide_compilations.project_id", + "project_guide_compilations.guide_id", + ], + name="fk_compilation_request_predecessor", + ), + sa.ForeignKeyConstraint( + ["authorization_decision_event_id"], + ["audit_events.id"], + name="fk_compilation_request_authorization_event", + ), + sa.UniqueConstraint( + "actor_profile_id", + "request_id", + name="uq_compilation_request_actor_request", + ), + sa.UniqueConstraint( + "actor_profile_id", + "idempotency_key", + name="uq_compilation_request_actor_key", + ), + sa.UniqueConstraint("attempt_id", name="uq_compilation_request_attempt"), + sa.UniqueConstraint( + "authorization_decision_event_id", + name="uq_compilation_request_authorization_event", + ), + sa.CheckConstraint( + "setup_generation > 0", name="ck_compilation_request_generation" + ), + sa.CheckConstraint( + "request_facts_digest ~ '^sha256:[0-9a-f]{64}$'", + name="ck_compilation_request_facts_digest", + ), + ) + _install_digest_functions() + _install_custody_guards() + + +def _install_digest_functions() -> None: + op.execute( + r""" + create function project_guide_compilation_request_facts_digest( + op project_guide_compilation_request_operations, + attempt project_guide_compilation_attempts + ) returns text immutable strict language sql as $$ + select 'sha256:' || encode(sha256(convert_to( + '{"domain":"workstream.project_guide_compilation.facts.v1","facts":{' || + '"agent_identity":' || to_json(attempt.agent_identity)::text || ',' || + '"agent_version":' || to_json(attempt.agent_version)::text || ',' || + '"canonical_input_hash":' || to_json(attempt.canonical_input_hash)::text || ',' || + '"expected_predecessor_compilation_id":' || + coalesce(to_json(op.expected_predecessor_compilation_id::text)::text, 'null') || ',' || + '"guide_id":' || to_json(op.guide_id)::text || ',' || + '"guide_material_hash":' || to_json(attempt.guide_material_hash)::text || ',' || + '"guide_version":' || to_json(attempt.guide_version)::text || ',' || + '"idempotency_key":' || to_json(op.idempotency_key::text)::text || ',' || + '"instruction_version":' || to_json(attempt.instruction_version)::text || ',' || + '"operation_id":' || to_json(op.operation_id::text)::text || ',' || + '"post_catalogue_id":' || to_json(attempt.post_catalogue_id)::text || ',' || + '"post_catalogue_manifest_hash":' || to_json(attempt.post_catalogue_manifest_hash)::text || ',' || + '"post_catalogue_schema_version":' || to_json(attempt.post_catalogue_schema_version)::text || ',' || + '"post_catalogue_version":' || to_json(attempt.post_catalogue_version)::text || ',' || + '"pre_catalogue_id":' || to_json(attempt.pre_catalogue_id)::text || ',' || + '"pre_catalogue_manifest_hash":' || to_json(attempt.pre_catalogue_manifest_hash)::text || ',' || + '"pre_catalogue_schema_version":' || to_json(attempt.pre_catalogue_schema_version)::text || ',' || + '"pre_catalogue_version":' || to_json(attempt.pre_catalogue_version)::text || ',' || + '"project_id":' || to_json(op.project_id)::text || ',' || + '"request_id":' || to_json(op.request_id::text)::text || ',' || + '"setup_generation":' || op.setup_generation::text || ',' || + '"setup_run_id":' || to_json(op.setup_run_id)::text || ',' || + '"source_snapshot_hash":' || to_json(attempt.source_snapshot_hash)::text || ',' || + '"source_snapshot_id":' || to_json(op.source_snapshot_id)::text || '}}', + 'UTF8')), 'hex') + $$ + """ + ) + op.execute( + r""" + create function project_guide_compilation_request_authority_digest( + op project_guide_compilation_request_operations, + grant_row admin_role_grants + ) returns text immutable strict language sql as $$ + select 'sha256:' || encode(sha256(convert_to( + '{"action_id":"project.guide_compilation.request",' || + '"actor_profile_id":' || to_json(op.actor_profile_id)::text || ',' || + '"identity_link_id":' || to_json(op.identity_link_id)::text || ',' || + '"permission_id":"project.guide_compilation.request",' || + '"project_manager_grant_id":' || to_json(grant_row.id::text)::text || ',' || + '"request_facts_digest":' || to_json(op.request_facts_digest)::text || ',' || + '"resource_id":' || to_json(op.operation_id::text)::text || ',' || + '"resource_type":"project_guide_compilation_request",' || + '"scope_project_id":' || to_json(op.project_id)::text || '}', + 'UTF8')), 'hex') + $$ + """ + ) + + +def _install_custody_guards() -> None: + op.execute( + """ + create function guard_project_guide_compilation_request_operation() + returns trigger language plpgsql as $$ + declare + attempt project_guide_compilation_attempts%rowtype; + event audit_events%rowtype; + grant_row admin_role_grants%rowtype; + begin + select * into attempt from project_guide_compilation_attempts + where id=new.attempt_id; + select * into event from audit_events + where id=new.authorization_decision_event_id; + select * into grant_row from admin_role_grants + where id=event.matched_grant_id::uuid; + if attempt.id is null or event.id is null or grant_row.id is null then + raise exception 'guide compilation request references are invalid' + using errcode='23514'; + end if; + if new.request_facts_digest is distinct from + project_guide_compilation_request_facts_digest(new, attempt) then + raise exception 'guide compilation request facts digest is invalid' + using errcode='23514'; + end if; + if event.event_domain is distinct from 'authority' + or event.event_type is distinct from 'SensitiveAuthorizationAllowed' + or event.action_id is distinct from 'project.guide_compilation.request' + or event.permission_id is distinct from 'project.guide_compilation.request' + or event.resource_type is distinct from 'project_guide_compilation_request' + or event.resource_id is distinct from new.operation_id::text + or event.project_id is distinct from new.project_id + or event.actor_id is distinct from new.actor_profile_id + or event.actor_ref_kind is distinct from 'actor_profile' + or event.after_facts->>'allowed' is distinct from 'true' then + raise exception 'guide compilation request audit event is invalid' + using errcode='23514'; + end if; + if grant_row.target_actor_profile_id is distinct from new.actor_profile_id + or grant_row.role is distinct from 'project_manager' + or grant_row.status is distinct from 'active' + or grant_row.scope_type is distinct from 'project' + or grant_row.scope_project_id is distinct from new.project_id then + raise exception 'guide compilation request grant is invalid' + using errcode='23514'; + end if; + if event.after_facts->>'resource_context_digest' is distinct from + project_guide_compilation_request_authority_digest(new, grant_row) then + raise exception 'guide compilation request authority digest is invalid' + using errcode='23514'; + end if; + return new; + end; + $$ + """ + ) + op.execute( + """ + create function reject_project_guide_compilation_request_change() + returns trigger language plpgsql as $$ + begin + raise exception 'guide compilation request custody is immutable' + using errcode='55000'; + end; + $$ + """ + ) + op.execute( + "create trigger guide_compilation_request_insert_guard before insert on " + "project_guide_compilation_request_operations for each row execute function " + "guard_project_guide_compilation_request_operation()" + ) + op.execute( + "create trigger guide_compilation_request_change_guard before update or delete on " + "project_guide_compilation_request_operations for each row execute function " + "reject_project_guide_compilation_request_change()" + ) + op.execute( + "create trigger guide_compilation_request_truncate_guard before truncate on " + "project_guide_compilation_request_operations execute function " + "reject_project_guide_compilation_request_change()" + ) + + +def downgrade() -> None: + connection = op.get_bind() + protected = connection.execute( + sa.text( + "select exists(select 1 from project_guide_compilation_request_operations) " + "or exists(select 1 from project_guide_compilation_attempts) " + "or exists(select 1 from project_guide_compilations) " + "or exists(select 1 from audit_events where action_id in " + "('project.guide_compilation.request','project.guide_compilation.execute'))" + ) + ).scalar_one() + if protected: + raise RuntimeError("guide compilation custody is non-empty; downgrade refused") + op.execute("drop trigger guide_compilation_request_insert_guard on project_guide_compilation_request_operations") + op.execute("drop trigger guide_compilation_request_change_guard on project_guide_compilation_request_operations") + op.execute("drop trigger guide_compilation_request_truncate_guard on project_guide_compilation_request_operations") + op.execute("drop function project_guide_compilation_request_authority_digest") + op.execute("drop function project_guide_compilation_request_facts_digest") + op.execute("drop function reject_project_guide_compilation_request_change") + op.execute("drop function guard_project_guide_compilation_request_operation") + op.drop_table("project_guide_compilation_request_operations") + op.drop_constraint( + "uq_compilation_attempt_exact_request", + "project_guide_compilation_attempts", + type_="unique", + ) diff --git a/backend/app/modules/projects/guide_compilation/authorization.py b/backend/app/modules/projects/guide_compilation/authorization.py deleted file mode 100644 index e801b42e5..000000000 --- a/backend/app/modules/projects/guide_compilation/authorization.py +++ /dev/null @@ -1,71 +0,0 @@ -"""Deny-only composition seam for inactive guide compilation authority.""" - -from __future__ import annotations - -from typing import Never - -from app.modules.authorization.api import ( - ActorIdentityFacts, - AuthorizationUnavailable, - ProjectGuideCompilationExecutePersistFacts, - ProjectGuideCompilationExecutePreflightFacts, - ProjectGuideCompilationRequestFacts, -) - - -class DenyProjectGuideCompilationAuthorization: - """Keep every compilation boundary unavailable until AUTH-12I.""" - - @staticmethod - def _deny() -> Never: - """Raise the stable unavailable-authority denial.""" - raise AuthorizationUnavailable("project guide compilation authority is unavailable") - - async def prepare_request( - self, *, actor: ActorIdentityFacts, facts: ProjectGuideCompilationRequestFacts - ) -> Never: - """Deny request preparation before AUTH activates the action.""" - del actor, facts - return self._deny() - - async def consume_request( - self, - *, - handle: object, - actor: ActorIdentityFacts, - facts: ProjectGuideCompilationRequestFacts, - ) -> Never: - """Deny request consumption without inspecting an alleged handle.""" - del handle, actor, facts - return self._deny() - - async def authorize_execute_preflight( - self, - *, - actor: ActorIdentityFacts, - facts: ProjectGuideCompilationExecutePreflightFacts, - ) -> Never: - """Deny fixed-service execution preflight while unavailable.""" - del actor, facts - return self._deny() - - async def prepare_execute_persist( - self, - *, - actor: ActorIdentityFacts, - facts: ProjectGuideCompilationExecutePersistFacts, - ) -> Never: - """Deny accepted-result persistence preparation while unavailable.""" - del actor, facts - return self._deny() - - async def consume_execute_persist( - self, - *, - handle: object, - actor: ActorIdentityFacts, - facts: ProjectGuideCompilationExecutePersistFacts, - ) -> Never: - """Deny persistence consumption without touching product state.""" - del handle, actor, facts - return self._deny() diff --git a/backend/app/modules/projects/guide_compilation/contracts.py b/backend/app/modules/projects/guide_compilation/contracts.py index c1220aac2..44fe9bdd3 100644 --- a/backend/app/modules/projects/guide_compilation/contracts.py +++ b/backend/app/modules/projects/guide_compilation/contracts.py @@ -32,12 +32,39 @@ class CompilationRecoveryClassification(StrEnum): """Bounded hidden recovery outcomes safe for operator inspection.""" RESERVED = "compilation_reserved" - PROVIDER_UNCERTAIN = "compilation_provider_uncertain" + PROVIDER_UNCERTAIN = "provider_outcome_unresolved" ACCEPTED_NOT_PERSISTED = "provider_result_accepted_not_persisted" PERSISTED = "compilation_persisted" INVALID_TERMINAL = "compilation_invalid_terminal" +class CompilationRequestReceipt(BaseModel): + """Bounded durable request result safe to return across transactions.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + operation_id: UUID + attempt_id: UUID + provider_idempotency_key: UUID + classification: CompilationRecoveryClassification + + +class CompilationDispatchReceipt(CompilationRequestReceipt): + """Committed permission boundary preceding possible provider I/O.""" + + dispatch_permitted: bool + + +class CompilationOutcomeReceipt(CompilationRequestReceipt): + """Bounded receipt for a known provider outcome.""" + + +class CompilationPersistenceReceipt(CompilationRequestReceipt): + """Bounded receipt for an immutable persisted compilation.""" + + compilation_id: UUID + + class CompilationAttemptIdentity(BaseModel): """Complete immutable identity of one logical provider attempt.""" diff --git a/backend/app/modules/projects/guide_compilation/models.py b/backend/app/modules/projects/guide_compilation/models.py index 44588a97a..5123ee42b 100644 --- a/backend/app/modules/projects/guide_compilation/models.py +++ b/backend/app/modules/projects/guide_compilation/models.py @@ -89,6 +89,15 @@ class ProjectGuideCompilationAttempt(Base): UniqueConstraint( "provider_idempotency_key", name="uq_compilation_attempt_provider_key" ), + UniqueConstraint( + "id", + "project_id", + "guide_id", + "source_snapshot_id", + "setup_run_id", + "setup_generation", + name="uq_compilation_attempt_exact_request", + ), CheckConstraint( "status in ('compilation_reserved','compilation_provider_uncertain'," "'provider_result_accepted','compilation_invalid_terminal'," @@ -184,6 +193,115 @@ class ProjectGuideCompilationAttempt(Base): persisted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) +class ProjectGuideCompilationRequestOperation(Base): + """Immutable authorized request receipt bound to one exact attempt.""" + + __tablename__ = "project_guide_compilation_request_operations" + __table_args__ = ( + ForeignKeyConstraint( + ["identity_link_id", "actor_profile_id"], + ["actor_identity_links.id", "actor_identity_links.actor_profile_id"], + name="fk_compilation_request_actor_link", + ), + ForeignKeyConstraint( + ["source_snapshot_id", "project_id", "guide_id"], + [ + "guide_source_snapshots.id", + "guide_source_snapshots.project_id", + "guide_source_snapshots.guide_id", + ], + name="fk_compilation_request_snapshot", + ), + ForeignKeyConstraint( + [ + "setup_run_id", + "project_id", + "guide_id", + "source_snapshot_id", + "setup_generation", + ], + [ + "project_setup_runs.id", + "project_setup_runs.project_id", + "project_setup_runs.guide_id", + "project_setup_runs.source_snapshot_id", + "project_setup_runs.setup_generation", + ], + name="fk_compilation_request_setup", + ), + ForeignKeyConstraint( + [ + "attempt_id", + "project_id", + "guide_id", + "source_snapshot_id", + "setup_run_id", + "setup_generation", + ], + [ + "project_guide_compilation_attempts.id", + "project_guide_compilation_attempts.project_id", + "project_guide_compilation_attempts.guide_id", + "project_guide_compilation_attempts.source_snapshot_id", + "project_guide_compilation_attempts.setup_run_id", + "project_guide_compilation_attempts.setup_generation", + ], + name="fk_compilation_request_exact_attempt", + ), + ForeignKeyConstraint( + ["expected_predecessor_compilation_id", "project_id", "guide_id"], + [ + "project_guide_compilations.id", + "project_guide_compilations.project_id", + "project_guide_compilations.guide_id", + ], + name="fk_compilation_request_predecessor", + ), + UniqueConstraint( + "actor_profile_id", + "request_id", + name="uq_compilation_request_actor_request", + ), + UniqueConstraint( + "actor_profile_id", + "idempotency_key", + name="uq_compilation_request_actor_key", + ), + UniqueConstraint("attempt_id", name="uq_compilation_request_attempt"), + UniqueConstraint( + "authorization_decision_event_id", + name="uq_compilation_request_authorization_event", + ), + CheckConstraint( + "setup_generation > 0", name="ck_compilation_request_generation" + ), + CheckConstraint( + "request_facts_digest " + _HASH_CHECK, + name="ck_compilation_request_facts_digest", + ), + ) + + operation_id: Mapped[UUID] = mapped_column(Uuid(), primary_key=True) + request_id: Mapped[UUID] = mapped_column(Uuid()) + idempotency_key: Mapped[UUID] = mapped_column(Uuid()) + actor_profile_id: Mapped[str] = mapped_column(ForeignKey("actor_profiles.id")) + identity_link_id: Mapped[str] = mapped_column(String(36)) + project_id: Mapped[str] = mapped_column(ForeignKey("projects.id")) + guide_id: Mapped[str] = mapped_column(ForeignKey("project_guides.id")) + source_snapshot_id: Mapped[str] = mapped_column(String(36)) + setup_run_id: Mapped[str] = mapped_column(String(36)) + setup_generation: Mapped[int] = mapped_column(BigInteger) + expected_predecessor_compilation_id: Mapped[UUID | None] = mapped_column(Uuid()) + request_facts_digest: Mapped[str] = mapped_column(String(71)) + attempt_id: Mapped[UUID] = mapped_column(Uuid()) + authorization_decision_event_id: Mapped[str] = mapped_column( + ForeignKey("audit_events.id") + ) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), server_default=func.now() + ) + + class ProjectGuideCompilation(Base): """Immutable accepted compilation in one append-only guide lineage.""" diff --git a/backend/app/modules/projects/guide_compilation/repository.py b/backend/app/modules/projects/guide_compilation/repository.py index dc49309c7..b85e605b9 100644 --- a/backend/app/modules/projects/guide_compilation/repository.py +++ b/backend/app/modules/projects/guide_compilation/repository.py @@ -6,7 +6,7 @@ from typing import Literal from uuid import UUID, uuid4 -from sqlalchemy import exists, select, update +from sqlalchemy import exists, or_, select, update from sqlalchemy.dialects.postgresql import insert from sqlalchemy.exc import DBAPIError from sqlalchemy.ext.asyncio import AsyncSession @@ -19,7 +19,10 @@ from app.modules.authorization.api import ( ActorIdentityFacts, ProjectGuideCompilationExecutePersistFacts, + ProjectGuideCompilationRequestFacts, + project_guide_compilation_facts_digest, ) +from app.modules.projects.models import GuideSourceSnapshot, ProjectGuide, ProjectSetupRun from .contracts import ( CompilationAttemptIdentity, @@ -27,7 +30,11 @@ accepted_compilation_result, validate_accepted_compilation_result, ) -from .models import ProjectGuideCompilation, ProjectGuideCompilationAttempt +from .models import ( + ProjectGuideCompilation, + ProjectGuideCompilationAttempt, + ProjectGuideCompilationRequestOperation, +) from .validation import ( accepted_from_attempt, identity_from_attempt, @@ -54,6 +61,25 @@ class GuideCompilationStorageError(GuideCompilationIntegrityError): "uq_project_guide_compilation_root", } ) +_REQUEST_CONSTRAINTS = frozenset( + { + "pk_project_guide_compilation_request_operations", + "uq_compilation_request_actor_request", + "uq_compilation_request_actor_key", + "uq_compilation_request_attempt", + "uq_compilation_request_authorization_event", + } +) +_BLOCKED_SETUP_STATUSES = frozenset( + { + "enqueue_failed", + "enqueue_identity_mismatch", + "sufficiency_blocked", + "post_submit_setup_blocked", + "setup_blocked", + "failed", + } +) def _persistence_error(exc: DBAPIError) -> GuideCompilationIntegrityError: @@ -74,6 +100,172 @@ def __init__(self, session: AsyncSession) -> None: """Bind repository operations to the caller-owned transaction.""" self._session = session + async def matching_request_operation( + self, + *, + actor: ActorIdentityFacts, + facts: ProjectGuideCompilationRequestFacts, + lock: bool = False, + ) -> ProjectGuideCompilationRequestOperation | None: + """Load one operation touching any replay identity and require exactness.""" + statement = select(ProjectGuideCompilationRequestOperation).where( + or_( + ProjectGuideCompilationRequestOperation.operation_id == facts.operation_id, + ( + ProjectGuideCompilationRequestOperation.actor_profile_id + == str(actor.actor_profile_id) + ) + & (ProjectGuideCompilationRequestOperation.request_id == facts.request_id), + ( + ProjectGuideCompilationRequestOperation.actor_profile_id + == str(actor.actor_profile_id) + ) + & ( + ProjectGuideCompilationRequestOperation.idempotency_key + == facts.idempotency_key + ), + ) + ) + if lock: + statement = statement.with_for_update() + rows = list((await self._session.scalars(statement)).all()) + if not rows: + return None + if len(rows) != 1 or not _request_matches(rows[0], actor, facts): + raise GuideCompilationIntegrityError("compilation request replay mismatch") + return rows[0] + + async def request_operation_for_attempt( + self, attempt_id: UUID, *, lock: bool + ) -> ProjectGuideCompilationRequestOperation: + """Load the exact immutable request custody for an attempt.""" + statement = select(ProjectGuideCompilationRequestOperation).where( + ProjectGuideCompilationRequestOperation.attempt_id == attempt_id + ) + if lock: + statement = statement.with_for_update() + operation = await self._session.scalar(statement) + if operation is None: + raise GuideCompilationIntegrityError("compilation request custody is missing") + return operation + + async def insert_request_operation( + self, + *, + actor: ActorIdentityFacts, + facts: ProjectGuideCompilationRequestFacts, + attempt: ProjectGuideCompilationAttempt, + authorization_decision_event_id: UUID, + ) -> ProjectGuideCompilationRequestOperation: + """Insert one authorized operation receipt bound to exact custody.""" + operation = ProjectGuideCompilationRequestOperation( + operation_id=facts.operation_id, + request_id=facts.request_id, + idempotency_key=facts.idempotency_key, + actor_profile_id=str(actor.actor_profile_id), + identity_link_id=str(actor.identity_link_id), + project_id=str(facts.project_id), + guide_id=str(facts.guide_id), + source_snapshot_id=str(facts.source_snapshot_id), + setup_run_id=str(facts.setup_run_id), + setup_generation=facts.setup_generation, + expected_predecessor_compilation_id=facts.expected_predecessor_compilation_id, + request_facts_digest=project_guide_compilation_facts_digest(facts), + attempt_id=attempt.id, + authorization_decision_event_id=str(authorization_decision_event_id), + ) + self._session.add(operation) + try: + await self._session.flush() + except DBAPIError as exc: + if integrity_constraint_name(exc) in _REQUEST_CONSTRAINTS: # type: ignore[arg-type] + raise GuideCompilationConcurrencyError( + "concurrent compilation request won; reload its exact receipt" + ) from exc + raise GuideCompilationStorageError( + "compilation request custody failed before commit" + ) from exc + return operation + + async def attempt( + self, attempt_id: UUID, *, lock: bool + ) -> ProjectGuideCompilationAttempt: + """Load an attempt, optionally holding its row for a root transaction.""" + return ( + await self._lock_attempt(attempt_id) + if lock + else await self._required_attempt(attempt_id) + ) + + async def require_current_setup_lineage( + self, attempt: ProjectGuideCompilationAttempt + ) -> None: + """Lock and require the attempt's exact active, latest setup lineage.""" + guide = await self._session.scalar( + select(ProjectGuide) + .where( + ProjectGuide.id == attempt.guide_id, + ProjectGuide.project_id == attempt.project_id, + ) + .with_for_update() + ) + if guide is None or guide.version != attempt.guide_version or guide.status != "draft": + raise GuideCompilationIntegrityError("compilation guide lineage is stale") + + setup = await self._session.scalar( + select(ProjectSetupRun) + .where(ProjectSetupRun.id == attempt.setup_run_id) + .with_for_update() + ) + snapshot = await self._session.scalar( + select(GuideSourceSnapshot).where( + GuideSourceSnapshot.id == attempt.source_snapshot_id + ) + ) + if ( + setup is None + or snapshot is None + or setup.project_id != attempt.project_id + or setup.guide_id != attempt.guide_id + or setup.guide_version != attempt.guide_version + or setup.source_snapshot_id != attempt.source_snapshot_id + or setup.source_snapshot_hash != attempt.source_snapshot_hash + or setup.setup_generation != attempt.setup_generation + or snapshot.project_id != attempt.project_id + or snapshot.guide_id != attempt.guide_id + or snapshot.guide_version != attempt.guide_version + or snapshot.bundle_hash != attempt.source_snapshot_hash + or setup.status in _BLOCKED_SETUP_STATUSES + ): + raise GuideCompilationIntegrityError("compilation setup lineage is stale") + + latest_generation = await self._session.scalar( + select(ProjectSetupRun.setup_generation) + .where( + ProjectSetupRun.project_id == attempt.project_id, + ProjectSetupRun.guide_id == attempt.guide_id, + ) + .order_by(ProjectSetupRun.setup_generation.desc()) + .limit(1) + ) + if latest_generation != attempt.setup_generation: + raise GuideCompilationIntegrityError("compilation setup generation is stale") + + async def current_compilation( + self, project_id: UUID, guide_id: UUID, *, lock: bool + ) -> ProjectGuideCompilation | None: + """Return the exact current lineage tip.""" + return await self._current(project_id, guide_id, lock=lock) + + async def persisted_compilation( + self, attempt_id: UUID + ) -> ProjectGuideCompilation: + """Return an attempt's required immutable compilation.""" + compilation = await self._compilation_for_attempt(attempt_id) + if compilation is None: + raise GuideCompilationIntegrityError("persisted compilation is missing") + return compilation + async def reserve_attempt( self, identity: CompilationAttemptIdentity ) -> tuple[Literal["claimed", "existing", "mismatch"], ProjectGuideCompilationAttempt]: @@ -194,6 +386,8 @@ async def recovery_classification( attempt = await self._required_attempt(attempt_id) if attempt.status == "provider_result_accepted": return CompilationRecoveryClassification.ACCEPTED_NOT_PERSISTED + if attempt.status == "compilation_provider_uncertain": + return CompilationRecoveryClassification.PROVIDER_UNCERTAIN return CompilationRecoveryClassification(attempt.status) async def persist_accepted( @@ -364,6 +558,30 @@ def _matches( ) +def _request_matches( + operation: ProjectGuideCompilationRequestOperation, + actor: ActorIdentityFacts, + facts: ProjectGuideCompilationRequestFacts, +) -> bool: + """Require every immutable replay selector and the complete facts digest.""" + return ( + operation.operation_id == facts.operation_id + and operation.request_id == facts.request_id + and operation.idempotency_key == facts.idempotency_key + and operation.actor_profile_id == str(actor.actor_profile_id) + and operation.identity_link_id == str(actor.identity_link_id) + and operation.project_id == str(facts.project_id) + and operation.guide_id == str(facts.guide_id) + and operation.source_snapshot_id == str(facts.source_snapshot_id) + and operation.setup_run_id == str(facts.setup_run_id) + and operation.setup_generation == facts.setup_generation + and operation.expected_predecessor_compilation_id + == facts.expected_predecessor_compilation_id + and operation.request_facts_digest + == project_guide_compilation_facts_digest(facts) + ) + + def _compilation_identity_values(identity: CompilationAttemptIdentity) -> dict[str, object]: """Map immutable compilation lineage fields from the attempt identity.""" return { diff --git a/backend/app/modules/projects/guide_compilation/service.py b/backend/app/modules/projects/guide_compilation/service.py new file mode 100644 index 000000000..25bc58bc7 --- /dev/null +++ b/backend/app/modules/projects/guide_compilation/service.py @@ -0,0 +1,366 @@ +"""Authorized short-transaction coordinator for hidden guide compilation.""" + +from __future__ import annotations + +from dataclasses import asdict, replace +from typing import Any +from sqlalchemy.ext.asyncio import AsyncSession + +from app.interfaces.project_agents import ( + ProjectGuideCompilationContext, + ProjectGuideCompilationResult, +) +from app.modules.authorization.api import ( + ActorIdentityFacts, + ProjectGuideCompilationAuthorizationPort, + ProjectGuideCompilationExecutePersistFacts, + ProjectGuideCompilationExecutePreflightFacts, + ProjectGuideCompilationRequestFacts, + project_guide_compilation_execute_resource_digest, +) + +from .contracts import ( + CompilationAttemptIdentity, + CompilationDispatchReceipt, + CompilationOutcomeReceipt, + CompilationPersistenceReceipt, + CompilationRecoveryClassification, + CompilationRequestReceipt, +) +from .models import ( + ProjectGuideCompilationAttempt, + ProjectGuideCompilationRequestOperation, +) +from .repository import ( + GuideCompilationConcurrencyError, + GuideCompilationIntegrityError, + GuideCompilationRepository, +) +from .validation import accepted_from_attempt, identity_from_attempt + + +class GuideCompilationService: + """Own every privileged POL-03B mutation in one fresh root transaction.""" + + def __init__( + self, + session: AsyncSession, + authorization: ProjectGuideCompilationAuthorizationPort[Any], + ) -> None: + self._session = session + self._authorization = authorization + + async def authorize_request( + self, + *, + actor: ActorIdentityFacts, + facts: ProjectGuideCompilationRequestFacts, + identity: CompilationAttemptIdentity, + ) -> CompilationRequestReceipt: + """Atomically persist a PM-authorized request or recover its receipt.""" + self._require_fresh_session() + _require_identity_matches(facts, identity) + try: + async with self._session.begin(): + repository = GuideCompilationRepository(self._session) + existing = await repository.matching_request_operation( + actor=actor, facts=facts, lock=True + ) + if existing is not None: + return await _request_receipt(repository, existing) + handle = await self._authorization.prepare_request( + actor=actor, facts=facts + ) + outcome, attempt = await repository.reserve_attempt(identity) + if outcome == "mismatch": + raise GuideCompilationIntegrityError( + "compilation attempt identity mismatch" + ) + if outcome == "existing": + raise GuideCompilationConcurrencyError( + "existing attempt has no authorized request custody" + ) + event_id = await self._authorization.consume_request( + handle=handle, actor=actor, facts=facts + ) + operation = await repository.insert_request_operation( + actor=actor, + facts=facts, + attempt=attempt, + authorization_decision_event_id=event_id, + ) + receipt = await _request_receipt(repository, operation) + return receipt + except GuideCompilationConcurrencyError: + return await self._recover_request(actor=actor, facts=facts) + + async def fence_dispatch( + self, + *, + actor: ActorIdentityFacts, + facts: ProjectGuideCompilationExecutePreflightFacts, + ) -> CompilationDispatchReceipt: + """Commit the conservative may-have-dispatched fence after AUTH preflight.""" + self._require_fresh_session() + async with self._session.begin(): + repository = GuideCompilationRepository(self._session) + operation, attempt = await _locked_exact(repository, facts) + if attempt.status == "compilation_provider_uncertain": + return _dispatch_receipt( + operation, attempt, dispatch_permitted=False + ) + if attempt.status != "compilation_reserved": + raise GuideCompilationIntegrityError("attempt cannot be dispatched") + await self._authorization.authorize_execute_preflight( + actor=actor, facts=facts + ) + attempt = await repository.mark_provider_uncertain(attempt.id) + receipt = _dispatch_receipt(operation, attempt, dispatch_permitted=True) + return receipt + + async def record_accepted_result( + self, + *, + actor: ActorIdentityFacts, + facts: ProjectGuideCompilationExecutePreflightFacts, + context: ProjectGuideCompilationContext, + result: ProjectGuideCompilationResult, + ) -> CompilationOutcomeReceipt: + """Record one known, strictly validated provider result.""" + self._require_fresh_session() + async with self._session.begin(): + repository = GuideCompilationRepository(self._session) + operation, attempt = await _locked_exact(repository, facts) + if attempt.status != "compilation_provider_uncertain": + raise GuideCompilationIntegrityError("provider outcome is not recordable") + await self._authorization.authorize_execute_preflight( + actor=actor, facts=facts + ) + attempt = await repository.accept_result( + attempt_id=attempt.id, context=context, result=result + ) + receipt = _outcome_receipt(operation, attempt) + return receipt + + async def record_invalid_result( + self, + *, + actor: ActorIdentityFacts, + facts: ProjectGuideCompilationExecutePreflightFacts, + failure_code: str, + ) -> CompilationOutcomeReceipt: + """Record one allowlisted terminal invalid-result code.""" + self._require_fresh_session() + async with self._session.begin(): + repository = GuideCompilationRepository(self._session) + operation, attempt = await _locked_exact(repository, facts) + if attempt.status != "compilation_provider_uncertain": + raise GuideCompilationIntegrityError("provider outcome is not recordable") + await self._authorization.authorize_execute_preflight( + actor=actor, facts=facts + ) + attempt = await repository.mark_invalid_terminal( + attempt_id=attempt.id, failure_code=failure_code + ) + receipt = _outcome_receipt(operation, attempt) + return receipt + + async def persist_accepted( + self, + *, + actor: ActorIdentityFacts, + facts: ProjectGuideCompilationExecutePreflightFacts, + context: ProjectGuideCompilationContext, + ) -> CompilationPersistenceReceipt: + """Consume fresh result-bound authority with immutable persistence.""" + self._require_fresh_session() + async with self._session.begin(): + repository = GuideCompilationRepository(self._session) + operation, attempt = await _locked_exact(repository, facts) + if attempt.status == "compilation_persisted": + return await _persisted_receipt(repository, operation, attempt) + if attempt.status != "provider_result_accepted": + raise GuideCompilationIntegrityError("attempt is not ready to persist") + persist_facts = _persist_facts(actor, facts, attempt) + handle = await self._authorization.prepare_execute_persist( + actor=actor, facts=persist_facts + ) + event_id = await self._authorization.consume_execute_persist( + handle=handle, actor=actor, facts=persist_facts + ) + compilation = await repository.persist_accepted( + attempt_id=attempt.id, + context=context, + expected_predecessor_id=operation.expected_predecessor_compilation_id, + actor=actor, + facts=persist_facts, + authorization_decision_event_id=event_id, + ) + receipt = CompilationPersistenceReceipt( + operation_id=operation.operation_id, + attempt_id=attempt.id, + provider_idempotency_key=attempt.provider_idempotency_key, + classification=CompilationRecoveryClassification.PERSISTED, + compilation_id=compilation.id, + ) + return receipt + + async def _recover_request( + self, + *, + actor: ActorIdentityFacts, + facts: ProjectGuideCompilationRequestFacts, + ) -> CompilationRequestReceipt: + self._require_fresh_session() + async with self._session.begin(): + repository = GuideCompilationRepository(self._session) + operation = await repository.matching_request_operation( + actor=actor, facts=facts, lock=False + ) + if operation is None: + raise GuideCompilationIntegrityError( + "concurrent request left no exact durable receipt" + ) + return await _request_receipt(repository, operation) + + def _require_fresh_session(self) -> None: + if self._session.in_transaction(): + raise GuideCompilationIntegrityError( + "guide compilation requires a fresh root transaction" + ) + + +async def _locked_exact( + repository: GuideCompilationRepository, + facts: ProjectGuideCompilationExecutePreflightFacts, +) -> tuple[ProjectGuideCompilationRequestOperation, ProjectGuideCompilationAttempt]: + attempt = await repository.attempt(facts.attempt_id, lock=True) + operation = await repository.request_operation_for_attempt(attempt.id, lock=True) + if _preflight_facts(operation, attempt) != facts: + raise GuideCompilationIntegrityError("compilation execute facts mismatch") + await repository.require_current_setup_lineage(attempt) + return operation, attempt + + +def _request_facts( + operation: ProjectGuideCompilationRequestOperation, + attempt: ProjectGuideCompilationAttempt, +) -> ProjectGuideCompilationRequestFacts: + identity = identity_from_attempt(attempt) + values = identity.model_dump() + return ProjectGuideCompilationRequestFacts( + **values, + operation_id=operation.operation_id, + request_id=operation.request_id, + idempotency_key=operation.idempotency_key, + expected_predecessor_compilation_id=operation.expected_predecessor_compilation_id, + ) + + +def _preflight_facts( + operation: ProjectGuideCompilationRequestOperation, + attempt: ProjectGuideCompilationAttempt, +) -> ProjectGuideCompilationExecutePreflightFacts: + request = _request_facts(operation, attempt) + return ProjectGuideCompilationExecutePreflightFacts( + **asdict(request), + attempt_id=attempt.id, + provider_idempotency_key=attempt.provider_idempotency_key, + ) + + +def _persist_facts( + actor: ActorIdentityFacts, + preflight: ProjectGuideCompilationExecutePreflightFacts, + attempt: ProjectGuideCompilationAttempt, +) -> ProjectGuideCompilationExecutePersistFacts: + accepted = accepted_from_attempt(attempt) + hashes = accepted.component_hashes + facts = ProjectGuideCompilationExecutePersistFacts( + **asdict(preflight), + result_hash=accepted.result_hash, + sufficiency_component_hash=hashes.sufficiency_hash, + artifact_policy_component_hash=hashes.artifact_policy_hash, + requirement_inventory_component_hash=hashes.requirement_inventory_hash, + pre_submit_policy_component_hash=hashes.pre_submit_hash, + post_submit_policy_component_hash=hashes.post_submit_hash, + capability_suggestions_component_hash=hashes.capability_suggestions_hash, + setup_notes_component_hash=hashes.setup_notes_hash, + resource_context_digest="sha256:" + "0" * 64, + ) + return replace( + facts, + resource_context_digest=project_guide_compilation_execute_resource_digest( + actor, facts + ), + ) + + +def _require_identity_matches( + facts: ProjectGuideCompilationRequestFacts, + identity: CompilationAttemptIdentity, +) -> None: + identity_values = identity.model_dump() + fact_values = {name: getattr(facts, name) for name in identity_values} + if fact_values != identity_values: + raise GuideCompilationIntegrityError("request facts do not match attempt identity") + + +async def _request_receipt( + repository: GuideCompilationRepository, + operation: ProjectGuideCompilationRequestOperation, +) -> CompilationRequestReceipt: + attempt = await repository.attempt(operation.attempt_id, lock=False) + return CompilationRequestReceipt( + operation_id=operation.operation_id, + attempt_id=attempt.id, + provider_idempotency_key=attempt.provider_idempotency_key, + classification=await repository.recovery_classification(attempt.id), + ) + + +def _dispatch_receipt( + operation: ProjectGuideCompilationRequestOperation, + attempt: ProjectGuideCompilationAttempt, + *, + dispatch_permitted: bool, +) -> CompilationDispatchReceipt: + return CompilationDispatchReceipt( + operation_id=operation.operation_id, + attempt_id=attempt.id, + provider_idempotency_key=attempt.provider_idempotency_key, + classification=CompilationRecoveryClassification.PROVIDER_UNCERTAIN, + dispatch_permitted=dispatch_permitted, + ) + + +def _outcome_receipt( + operation: ProjectGuideCompilationRequestOperation, + attempt: ProjectGuideCompilationAttempt, +) -> CompilationOutcomeReceipt: + classification = ( + CompilationRecoveryClassification.ACCEPTED_NOT_PERSISTED + if attempt.status == "provider_result_accepted" + else CompilationRecoveryClassification.INVALID_TERMINAL + ) + return CompilationOutcomeReceipt( + operation_id=operation.operation_id, + attempt_id=attempt.id, + provider_idempotency_key=attempt.provider_idempotency_key, + classification=classification, + ) + + +async def _persisted_receipt( + repository: GuideCompilationRepository, + operation: ProjectGuideCompilationRequestOperation, + attempt: ProjectGuideCompilationAttempt, +) -> CompilationPersistenceReceipt: + compilation = await repository.persisted_compilation(attempt.id) + return CompilationPersistenceReceipt( + operation_id=operation.operation_id, + attempt_id=attempt.id, + provider_idempotency_key=attempt.provider_idempotency_key, + classification=CompilationRecoveryClassification.PERSISTED, + compilation_id=compilation.id, + ) diff --git a/backend/scripts/behavior_ownership.py b/backend/scripts/behavior_ownership.py index 27de9138a..d21ca532a 100644 --- a/backend/scripts/behavior_ownership.py +++ b/backend/scripts/behavior_ownership.py @@ -161,12 +161,15 @@ POL_03A_CALLABLE_TARGETS = frozenset( { "backend/app/modules/authorization/api/project_guide_compilation.py", - "backend/app/modules/projects/guide_compilation/authorization.py", "backend/app/modules/projects/guide_compilation/contracts.py", "backend/app/modules/projects/guide_compilation/repository.py", + "backend/app/modules/projects/guide_compilation/service.py", "backend/app/modules/projects/guide_compilation/validation.py", } ) +POL_03B_REMOVED_TARGETS = frozenset( + {"backend/app/modules/projects/guide_compilation/authorization.py"} +) POL_03A_DECLARATIVE_MODEL_TARGET = ( "backend/app/modules/projects/guide_compilation/models.py" ) @@ -334,7 +337,7 @@ def _validate_additive_partition_transition( ] if ( trusted_targets != sorted(trusted_targets) - or removed - V01_BASELINE_REMOVED_TARGETS + or removed - (V01_BASELINE_REMOVED_TARGETS | POL_03B_REMOVED_TARGETS) or [current_by_target[item["target"]] for item in retained_trusted] != retained_trusted ): diff --git a/backend/scripts/run_test_lanes.py b/backend/scripts/run_test_lanes.py index 9aba2241b..e682fdfd6 100644 --- a/backend/scripts/run_test_lanes.py +++ b/backend/scripts/run_test_lanes.py @@ -213,10 +213,17 @@ class TestLane: TestLane( "project_lifecycle", ( + "tests/projects/guide_compilation/test_authorized_concurrency_postgresql.py", + "tests/projects/guide_compilation/test_authorized_execution_service.py", + "tests/projects/guide_compilation/test_authorized_recovery_postgresql.py", + "tests/projects/guide_compilation/test_authorized_request_service.py", "tests/projects/guide_compilation/test_contracts.py", "tests/projects/guide_compilation/test_database_guards.py", + "tests/projects/guide_compilation/test_durable_dispatch_handoff.py", + "tests/projects/guide_compilation/test_migration_authorized_persistence.py", "tests/projects/guide_compilation/test_migration_contract.py", "tests/projects/guide_compilation/test_public_authorization.py", + "tests/projects/guide_compilation/test_request_operation_postgresql.py", "tests/projects/guide_compilation/test_repository_attempts.py", "tests/projects/guide_compilation/test_repository_persistence.py", "tests/projects/test_locked_policy_context.py", diff --git a/backend/tests/authorization/guide_compilation/test_migration_contract.py b/backend/tests/authorization/guide_compilation/test_migration_contract.py index 17550d2a3..0df781d4f 100644 --- a/backend/tests/authorization/guide_compilation/test_migration_contract.py +++ b/backend/tests/authorization/guide_compilation/test_migration_contract.py @@ -60,7 +60,7 @@ def test_current_schema_preserves_exact_compilation_registries( isolated_database_env: str, ) -> None: assert asyncio.run(_registry_state(isolated_database_env)) == ( - "0007_contribution_policy_publication_custody", + "0008_guide_compilation_authorized_persistence", 1, 1, 1, diff --git a/backend/tests/conftest.py b/backend/tests/conftest.py index 118a66c31..a6315ec0c 100644 --- a/backend/tests/conftest.py +++ b/backend/tests/conftest.py @@ -21,7 +21,7 @@ from scripts.run_isolated_tests import LOOPBACK, NAME_RE, ROLE_RE DDL_LOCK_DIRECTORY = Path("/tmp") -EXPECTED_PUBLIC_SCHEMA_SHA256 = "ec3c257ae534e6371d6a9d9214106f07548f7b9bf156648052a667f216ddd2cd" +EXPECTED_PUBLIC_SCHEMA_SHA256 = "c1b9f2aacff92805665f75afc41852452b402bd7c9d92ba81324cb06f7986753" PROTECTED_TEST_TABLES = ( "actor_profile_migration_state", "alembic_version", @@ -88,6 +88,7 @@ "project_guides", "project_guide_compilation_attempts", "project_guide_compilations", + "project_guide_compilation_request_operations", "project_role_grants", "project_role_qualification_snapshots", "project_setup_runs", @@ -128,6 +129,7 @@ "project_create_idempotency_records", "project_guide_compilation_attempts", "project_guide_compilations", + "project_guide_compilation_request_operations", "project_role_grants", "project_role_qualification_snapshots", "review_admission_idempotency_records", diff --git a/backend/tests/projects/guide_compilation/test_authorized_concurrency_postgresql.py b/backend/tests/projects/guide_compilation/test_authorized_concurrency_postgresql.py new file mode 100644 index 000000000..da0343c89 --- /dev/null +++ b/backend/tests/projects/guide_compilation/test_authorized_concurrency_postgresql.py @@ -0,0 +1,102 @@ +"""Independent-session concurrency proof for request and final custody.""" + +from __future__ import annotations + +import asyncio + +import pytest +from sqlalchemy import text +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from app.modules.authorization.api import ActorIdentityFacts, ActorKind + +from .helpers import context, identity, result, seed_database, service_actor +from .test_authorized_execution_service import _execution_service, _preflight +from .test_authorized_request_service import _authorized_service, _request, _seed_human + + +@pytest.mark.asyncio +async def test_concurrent_identical_requests_commit_one_attempt_event_and_receipt( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, link, _grant = await _seed_human(clean_postgres_database, values) + actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + facts, attempt_identity = _request(values), identity(context(values)) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + + async def request_once(): + async with factory() as session: + return await _authorized_service(session, actor).authorize_request( + actor=actor, facts=facts, identity=attempt_identity + ) + + try: + first, second = await asyncio.gather(request_once(), request_once()) + assert first == second + async with factory() as session: + counts = ( + await session.execute( + text( + "select (select count(*) from project_guide_compilation_attempts)," + "(select count(*) from project_guide_compilation_request_operations)," + "(select count(*) from audit_events where action_id=" + "'project.guide_compilation.request')" + ) + ) + ).one() + await session.rollback() + assert counts == (1, 1, 1) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_concurrent_finalization_commits_one_compilation_and_event( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, link, _grant = await _seed_human(clean_postgres_database, values) + human_actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + service = service_actor(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + requested = await _authorized_service(session, human_actor).authorize_request( + actor=human_actor, + facts=_request(values), + identity=identity(context(values)), + ) + facts = _preflight(values, requested.attempt_id) + async with factory() as session: + execution = _execution_service(session, service) + await execution.fence_dispatch(actor=service, facts=facts) + async with factory() as session: + await _execution_service(session, service).record_accepted_result( + actor=service, facts=facts, context=context(values), result=result() + ) + + async def persist_once(): + async with factory() as session: + return await _execution_service(session, service).persist_accepted( + actor=service, facts=facts, context=context(values) + ) + + first, second = await asyncio.gather(persist_once(), persist_once()) + assert first == second + async with factory() as session: + counts = ( + await session.execute( + text( + "select (select count(*) from project_guide_compilations)," + "(select count(*) from audit_events where action_id=" + "'project.guide_compilation.execute')" + ) + ) + ).one() + await session.rollback() + assert counts == (1, 1) + finally: + await engine.dispose() diff --git a/backend/tests/projects/guide_compilation/test_authorized_execution_service.py b/backend/tests/projects/guide_compilation/test_authorized_execution_service.py new file mode 100644 index 000000000..44d78cc1a --- /dev/null +++ b/backend/tests/projects/guide_compilation/test_authorized_execution_service.py @@ -0,0 +1,486 @@ +"""Four-stage authorized execution custody through real PostgreSQL and AUTH.""" + +from __future__ import annotations + +from dataclasses import asdict, replace +from uuid import UUID, uuid4 + +import pytest +from sqlalchemy import text +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine + +from app.modules.authorization.api import ( + ActorIdentityFacts, + ActorKind as PublicActorKind, + ProjectGuideCompilationExecutePreflightFacts, +) +from app.modules.authorization.guide_compilation import ( + ProjectGuideCompilationAuthorizationAdapter, +) +from app.modules.authorization.kernel import AuthorizationService +from app.modules.authorization.prepared import PreparedAuthorizationService +from app.modules.authorization.repository import AdminAuthorizationRepository +from app.modules.authorization.runtime import ( + ActorKind, + ActorStatus, + IdentityLinkStatus, + ServiceAuthorizationContext, +) +from app.modules.actors.service_identities import ServiceIdentity +from app.modules.projects.guide_compilation.contracts import ( + CompilationRecoveryClassification, +) +from app.modules.projects.guide_compilation.repository import ( + GuideCompilationIntegrityError, + GuideCompilationRepository, +) +from app.modules.projects.guide_compilation.service import GuideCompilationService + +from .helpers import context, identity, persistence_facts, result, seed_database, service_actor +from .test_authorized_request_service import _authorized_service, _request, _seed_human + + +def _execution_service( + session: AsyncSession, actor: ActorIdentityFacts +) -> GuideCompilationService: + context_value = ServiceAuthorizationContext( + actor_profile_id=actor.actor_profile_id, + actor_kind=ActorKind.SERVICE, + actor_status=ActorStatus.ACTIVE, + identity_link_id=actor.identity_link_id, + identity_link_status=IdentityLinkStatus.ACTIVE, + service_identity=ServiceIdentity.PROJECT_SETUP, + request_id=uuid4(), + correlation_id=uuid4(), + ) + repository = AdminAuthorizationRepository(session) + authorization = AuthorizationService( + session, context_value, admin_repository=repository + ) + prepared = PreparedAuthorizationService( + session, context_value, authorization, repository + ) + return GuideCompilationService( + session, + ProjectGuideCompilationAuthorizationAdapter(authorization, prepared), + ) + + +def _preflight( + values: dict[str, UUID], attempt_id: UUID +) -> ProjectGuideCompilationExecutePreflightFacts: + complete = persistence_facts(values, attempt_id, identity(context(values))) + names = ProjectGuideCompilationExecutePreflightFacts.__dataclass_fields__ + return ProjectGuideCompilationExecutePreflightFacts( + **{name: asdict(complete)[name] for name in names} + ) + + +@pytest.mark.asyncio +async def test_authorized_execution_fences_accepts_and_persists_atomically( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, human_link, _grant = await _seed_human(clean_postgres_database, values) + human_actor = ActorIdentityFacts(human, human_link, PublicActorKind.HUMAN) + service = service_actor(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + requested = await _authorized_service(session, human_actor).authorize_request( + actor=human_actor, + facts=_request(values), + identity=identity(context(values)), + ) + facts = _preflight(values, requested.attempt_id) + async with factory() as session: + fenced = await _execution_service(session, service).fence_dispatch( + actor=service, facts=facts + ) + assert fenced.classification is CompilationRecoveryClassification.PROVIDER_UNCERTAIN + assert fenced.dispatch_permitted is True + async with factory() as session: + accepted = await _execution_service(session, service).record_accepted_result( + actor=service, facts=facts, context=context(values), result=result() + ) + assert ( + accepted.classification + is CompilationRecoveryClassification.ACCEPTED_NOT_PERSISTED + ) + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="not recordable"): + await _execution_service(session, service).record_accepted_result( + actor=service, facts=facts, context=context(values), result=result() + ) + async with factory() as session: + persisted = await _execution_service(session, service).persist_accepted( + actor=service, facts=facts, context=context(values) + ) + assert persisted.classification is CompilationRecoveryClassification.PERSISTED + async with factory() as session: + replay = await _execution_service(session, service).persist_accepted( + actor=service, facts=facts, context=context(values) + ) + assert replay == persisted + async with factory() as session: + rows = ( + await session.execute( + text( + "select (select count(*) from project_guide_compilations)," + "(select count(*) from audit_events where action_id=" + "'project.guide_compilation.execute')," + "(select count(*) from outbox_events)," + "(select status from project_guide_compilation_attempts where id=:id)" + ), + {"id": requested.attempt_id}, + ) + ).one() + await session.rollback() + assert rows == (1, 1, 0, "compilation_persisted") + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_invalid_provider_result_becomes_one_bounded_terminal_outcome( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, human_link, _grant = await _seed_human(clean_postgres_database, values) + human_actor = ActorIdentityFacts(human, human_link, PublicActorKind.HUMAN) + service = service_actor(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + requested = await _authorized_service(session, human_actor).authorize_request( + actor=human_actor, + facts=_request(values), + identity=identity(context(values)), + ) + facts = _preflight(values, requested.attempt_id) + async with factory() as session: + await _execution_service(session, service).fence_dispatch( + actor=service, facts=facts + ) + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="failure code"): + await _execution_service(session, service).record_invalid_result( + actor=service, facts=facts, failure_code="provider_exception_detail" + ) + async with factory() as session: + terminal = await _execution_service(session, service).record_invalid_result( + actor=service, facts=facts, failure_code="schema_invalid" + ) + assert terminal.classification is CompilationRecoveryClassification.INVALID_TERMINAL + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="not recordable"): + await _execution_service(session, service).record_invalid_result( + actor=service, facts=facts, failure_code="schema_invalid" + ) + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="cannot be dispatched"): + await _execution_service(session, service).fence_dispatch( + actor=service, facts=facts + ) + async with factory() as session: + row = ( + await session.execute( + text( + "select status,failure_code,(select count(*) from audit_events " + "where action_id='project.guide_compilation.execute') " + "from project_guide_compilation_attempts where id=:id" + ), + {"id": requested.attempt_id}, + ) + ).one() + await session.rollback() + assert row == ("compilation_invalid_terminal", "schema_invalid", 0) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_execution_rejects_nonfresh_session_and_durable_fact_drift( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, human_link, _grant = await _seed_human(clean_postgres_database, values) + human_actor = ActorIdentityFacts(human, human_link, PublicActorKind.HUMAN) + service = service_actor(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + requested = await _authorized_service(session, human_actor).authorize_request( + actor=human_actor, + facts=_request(values), + identity=identity(context(values)), + ) + facts = _preflight(values, requested.attempt_id) + async with factory() as session, session.begin(): + with pytest.raises(GuideCompilationIntegrityError, match="fresh root"): + await _execution_service(session, service).fence_dispatch( + actor=service, facts=facts + ) + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="facts mismatch"): + await _execution_service(session, service).fence_dispatch( + actor=service, facts=replace(facts, request_id=uuid4()) + ) + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="not ready"): + await _execution_service(session, service).persist_accepted( + actor=service, facts=facts, context=context(values) + ) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + "stale_change, expected_message", + [ + ("failed", "setup lineage is stale"), + ("setup_blocked", "setup lineage is stale"), + ("new_generation", "setup generation is stale"), + ], +) +async def test_execution_rejects_stale_setup_lineage_before_authority_or_transition( + clean_postgres_database: str, + stale_change: str, + expected_message: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, human_link, _grant = await _seed_human(clean_postgres_database, values) + human_actor = ActorIdentityFacts(human, human_link, PublicActorKind.HUMAN) + service = service_actor(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + requested = await _authorized_service(session, human_actor).authorize_request( + actor=human_actor, + facts=_request(values), + identity=identity(context(values)), + ) + async with factory() as session, session.begin(): + if stale_change == "new_generation": + await session.execute( + text( + "insert into project_setup_runs(id,project_id,guide_id,guide_version," + "source_snapshot_id,source_snapshot_hash,setup_generation,status," + "current_step,created_by) values(:setup,:project,:guide,'v1',:snapshot," + ":hash,2,'queued','guide_material_verified','test')" + ), + { + "setup": str(values["setup_2"]), + "project": str(values["project"]), + "guide": str(values["guide"]), + "snapshot": str(values["snapshot"]), + "hash": identity(context(values)).source_snapshot_hash, + }, + ) + else: + await session.execute( + text("update project_setup_runs set status=:status where id=:setup"), + {"status": stale_change, "setup": str(values["setup_1"])}, + ) + facts = _preflight(values, requested.attempt_id) + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match=expected_message): + await _execution_service(session, service).fence_dispatch( + actor=service, facts=facts + ) + async with factory() as session: + row = ( + await session.execute( + text( + "select status,(select count(*) from audit_events where action_id=" + "'project.guide_compilation.execute') from " + "project_guide_compilation_attempts where id=:id" + ), + {"id": requested.attempt_id}, + ) + ).one() + await session.rollback() + assert row == ("compilation_reserved", 0) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_execution_rechecks_setup_lineage_for_outcome_and_persistence( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, human_link, _grant = await _seed_human(clean_postgres_database, values) + human_actor = ActorIdentityFacts(human, human_link, PublicActorKind.HUMAN) + service = service_actor(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + requested = await _authorized_service(session, human_actor).authorize_request( + actor=human_actor, + facts=_request(values), + identity=identity(context(values)), + ) + facts = _preflight(values, requested.attempt_id) + async with factory() as session: + await _execution_service(session, service).fence_dispatch( + actor=service, facts=facts + ) + async with factory() as session, session.begin(): + await session.execute( + text("update project_setup_runs set status='failed' where id=:setup"), + {"setup": str(values["setup_1"])}, + ) + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="setup lineage"): + await _execution_service(session, service).record_accepted_result( + actor=service, + facts=facts, + context=context(values), + result=result(), + ) + async with factory() as session, session.begin(): + await session.execute( + text("update project_setup_runs set status='queued' where id=:setup"), + {"setup": str(values["setup_1"])}, + ) + async with factory() as session: + await _execution_service(session, service).record_accepted_result( + actor=service, + facts=facts, + context=context(values), + result=result(), + ) + async with factory() as session, session.begin(): + await session.execute( + text( + "insert into project_setup_runs(id,project_id,guide_id,guide_version," + "source_snapshot_id,source_snapshot_hash,setup_generation,status," + "current_step,created_by) values(:setup,:project,:guide,'v1',:snapshot," + ":hash,2,'queued','guide_material_verified','test')" + ), + { + "setup": str(values["setup_2"]), + "project": str(values["project"]), + "guide": str(values["guide"]), + "snapshot": str(values["snapshot"]), + "hash": identity(context(values)).source_snapshot_hash, + }, + ) + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="generation is stale"): + await _execution_service(session, service).persist_accepted( + actor=service, + facts=facts, + context=context(values), + ) + async with factory() as session: + row = ( + await session.execute( + text( + "select status,(select count(*) from " + "project_guide_compilations),(select count(*) from audit_events " + "where action_id='project.guide_compilation.execute') from " + "project_guide_compilation_attempts where id=:id" + ), + {"id": requested.attempt_id}, + ) + ).one() + await session.rollback() + assert row == ("provider_result_accepted", 0, 0) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_accepted_result_replay_requires_exact_canonical_result( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + compilation_context = context(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session, session.begin(): + repository = GuideCompilationRepository(session) + _outcome, attempt = await repository.reserve_attempt( + identity(compilation_context) + ) + accepted = await repository.accept_result( + attempt_id=attempt.id, + context=compilation_context, + result=result(), + ) + async with factory() as session, session.begin(): + replay = await GuideCompilationRepository(session).accept_result( + attempt_id=accepted.id, + context=compilation_context, + result=result(), + ) + assert replay.status == "provider_result_accepted" + changed_finding = result().findings[0].model_copy( + update={"message": "A different valid finding."} + ) + changed = result().model_copy(update={"findings": (changed_finding,)}) + async with factory() as session, session.begin(): + with pytest.raises(GuideCompilationIntegrityError, match="result mismatch"): + await GuideCompilationRepository(session).accept_result( + attempt_id=accepted.id, + context=compilation_context, + result=changed, + ) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_accepted_result_rejects_context_from_another_generation( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database, generations=2) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session, session.begin(): + repository = GuideCompilationRepository(session) + _outcome, attempt = await repository.reserve_attempt(identity(context(values))) + with pytest.raises(GuideCompilationIntegrityError, match="result is invalid"): + await repository.accept_result( + attempt_id=attempt.id, + context=context(values, generation=2), + result=result(), + ) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_reserved_attempt_cannot_persist_without_accepted_custody( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + compilation_context = context(values) + attempt_identity = identity(compilation_context) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session, session.begin(): + repository = GuideCompilationRepository(session) + _outcome, attempt = await repository.reserve_attempt(attempt_identity) + with pytest.raises(GuideCompilationIntegrityError, match="not ready"): + await repository.persist_accepted( + attempt_id=attempt.id, + context=compilation_context, + expected_predecessor_id=None, + actor=service_actor(values), + facts=persistence_facts(values, attempt.id, attempt_identity), + authorization_decision_event_id=uuid4(), + ) + finally: + await engine.dispose() diff --git a/backend/tests/projects/guide_compilation/test_authorized_recovery_postgresql.py b/backend/tests/projects/guide_compilation/test_authorized_recovery_postgresql.py new file mode 100644 index 000000000..5a85d9884 --- /dev/null +++ b/backend/tests/projects/guide_compilation/test_authorized_recovery_postgresql.py @@ -0,0 +1,135 @@ +"""Durable recovery classifications never invent provider certainty.""" + +from __future__ import annotations + +from dataclasses import replace +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine +import pytest + +from app.modules.authorization.api import ActorIdentityFacts, ActorKind +from app.modules.projects.guide_compilation.contracts import ( + CompilationRecoveryClassification, +) +from app.modules.projects.guide_compilation.service import GuideCompilationService +from app.modules.projects.guide_compilation.repository import ( + GuideCompilationIntegrityError, + GuideCompilationRepository, +) + +from .helpers import context, identity, result, seed_database, service_actor +from .test_authorized_execution_service import _execution_service, _preflight +from .test_authorized_request_service import _authorized_service, _request, _seed_human + + +class _NoAuthorityOrProvider: + """Fail if uncertain recovery reaches AUTH or any external operation.""" + + calls = 0 + + def __getattr__(self, name: str): + type(self).calls += 1 + raise AssertionError(f"uncertain recovery reached {name}") + + +@pytest.mark.asyncio +async def test_uncertain_restart_returns_unresolved_without_redispatch( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, link, _grant = await _seed_human(clean_postgres_database, values) + human_actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + service = service_actor(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + requested = await _authorized_service(session, human_actor).authorize_request( + actor=human_actor, + facts=_request(values), + identity=identity(context(values)), + ) + facts = _preflight(values, requested.attempt_id) + async with factory() as session: + first = await _execution_service(session, service).fence_dispatch( + actor=service, facts=facts + ) + _NoAuthorityOrProvider.calls = 0 + async with factory() as restarted_session: + recovered = await GuideCompilationService( + restarted_session, _NoAuthorityOrProvider() # type: ignore[arg-type] + ).fence_dispatch(actor=service, facts=facts) + assert first.dispatch_permitted is True + assert recovered.dispatch_permitted is False + assert recovered.model_dump(exclude={"dispatch_permitted"}) == first.model_dump( + exclude={"dispatch_permitted"} + ) + assert recovered.classification is CompilationRecoveryClassification.PROVIDER_UNCERTAIN + assert _NoAuthorityOrProvider.calls == 0 + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_changed_request_replay_fails_without_new_authority_event( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, link, _grant = await _seed_human(clean_postgres_database, values) + actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + facts = _request(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + await _authorized_service(session, actor).authorize_request( + actor=actor, facts=facts, identity=identity(context(values)) + ) + changed = replace(facts, instruction_version="v2") + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="replay mismatch"): + await _authorized_service(session, actor).authorize_request( + actor=actor, + facts=changed, + identity=identity(context(values)).model_copy( + update={"instruction_version": "v2"} + ), + ) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_repeated_terminal_transitions_preserve_one_attempt( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session, session.begin(): + repository = GuideCompilationRepository(session) + _outcome, attempt = await repository.reserve_attempt(identity(context(values))) + uncertain = await repository.mark_provider_uncertain(attempt.id) + assert await repository.mark_provider_uncertain(attempt.id) == uncertain + assert await repository.recovery_classification(attempt.id) == ( + "provider_outcome_unresolved" + ) + async with factory() as session, session.begin(): + repository = GuideCompilationRepository(session) + terminal = await repository.mark_invalid_terminal( + attempt_id=attempt.id, failure_code="schema_invalid" + ) + assert ( + await repository.mark_invalid_terminal( + attempt_id=attempt.id, failure_code="schema_invalid" + ) + == terminal + ) + with pytest.raises(GuideCompilationIntegrityError, match="accepted transition"): + await repository.accept_result( + attempt_id=attempt.id, + context=context(values), + result=result(), + ) + finally: + await engine.dispose() diff --git a/backend/tests/projects/guide_compilation/test_authorized_request_service.py b/backend/tests/projects/guide_compilation/test_authorized_request_service.py new file mode 100644 index 000000000..78799140d --- /dev/null +++ b/backend/tests/projects/guide_compilation/test_authorized_request_service.py @@ -0,0 +1,188 @@ +"""Authorized request custody through the real AUTH adapter and PostgreSQL.""" + +from __future__ import annotations + +from dataclasses import asdict +from datetime import UTC, datetime +from uuid import UUID, uuid4 + +import pytest +from sqlalchemy import text +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine + +from app.modules.authorization.api import ( + ActorIdentityFacts, + ActorKind as PublicActorKind, + ProjectGuideCompilationRequestFacts, +) +from app.modules.authorization.guide_compilation import ( + ProjectGuideCompilationAuthorizationAdapter, +) +from app.modules.authorization.kernel import AuthorizationService +from app.modules.authorization.prepared import PreparedAuthorizationService +from app.modules.authorization.repository import AdminAuthorizationRepository +from app.modules.authorization.runtime import ( + ActorKind, + ActorStatus, + HumanAuthorizationContext, + IdentityLinkStatus, +) +from app.modules.projects.guide_compilation.contracts import ( + CompilationRecoveryClassification, +) +from app.modules.projects.guide_compilation.repository import ( + GuideCompilationIntegrityError, +) +from app.modules.projects.guide_compilation.service import GuideCompilationService + +from .helpers import context, identity, persistence_facts, seed_database + + +def _request(values: dict[str, UUID]) -> ProjectGuideCompilationRequestFacts: + attempt_identity = identity(context(values)) + persist = persistence_facts(values, uuid4(), attempt_identity) + names = ProjectGuideCompilationRequestFacts.__dataclass_fields__ + return ProjectGuideCompilationRequestFacts( + **{name: asdict(persist)[name] for name in names} + ) + + +async def _seed_human( + database_url: str, values: dict[str, UUID] +) -> tuple[UUID, UUID, UUID]: + human, link, grant = uuid4(), uuid4(), uuid4() + engine = create_async_engine(database_url) + try: + async with engine.begin() as connection: + await connection.execute( + text( + "insert into actor_profiles(id,actor_kind,status,provisioning_method," + "created_by) values(:human,'human','active','automatic_first_access','test')" + ), + {"human": str(human)}, + ) + await connection.execute( + text( + "insert into actor_identity_links(id,actor_profile_id,issuer,subject," + "subject_kind,status,linked_by,last_verified_at) values" + "(:link,:human,'https://identity.flowresearch.tech',:subject,'human'," + "'active','test',:now)" + ), + { + "link": str(link), + "human": str(human), + "subject": f"compilation-{human}", + "now": datetime.now(UTC), + }, + ) + await connection.execute(text("alter table admin_role_grants disable trigger user")) + await connection.execute( + text( + "insert into admin_role_grants(id,target_actor_profile_id,role,scope_type," + "scope_project_id,status,version,granted_by_system_principal,grant_reason) " + "values(:grant,:human,'project_manager','project',:project,'active',1," + "'workstream:system:bootstrap','authorized compilation test')" + ), + { + "grant": grant, + "human": str(human), + "project": str(values["project"]), + }, + ) + await connection.execute(text("alter table admin_role_grants enable trigger user")) + finally: + await engine.dispose() + return human, link, grant + + +@pytest.mark.asyncio +async def test_request_rejects_caller_identity_drift_before_authority( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, link, _grant = await _seed_human(clean_postgres_database, values) + actor = ActorIdentityFacts(human, link, PublicActorKind.HUMAN) + facts = _request(values) + drifted = identity(context(values)).model_copy(update={"guide_version": "guide.v2"}) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + with pytest.raises(GuideCompilationIntegrityError, match="do not match"): + await _authorized_service(session, actor).authorize_request( + actor=actor, facts=facts, identity=drifted + ) + counts = ( + await session.execute( + text( + "select (select count(*) from project_guide_compilation_attempts)," + "(select count(*) from audit_events where action_id=" + "'project.guide_compilation.request')" + ) + ) + ).one() + await session.rollback() + assert counts == (0, 0) + finally: + await engine.dispose() + + +def _authorized_service( + session: AsyncSession, actor: ActorIdentityFacts +) -> GuideCompilationService: + context_value = HumanAuthorizationContext( + actor_profile_id=actor.actor_profile_id, + actor_kind=ActorKind.HUMAN, + actor_status=ActorStatus.ACTIVE, + identity_link_id=actor.identity_link_id, + identity_link_status=IdentityLinkStatus.ACTIVE, + request_id=uuid4(), + correlation_id=uuid4(), + ) + repository = AdminAuthorizationRepository(session) + authorization = AuthorizationService( + session, context_value, admin_repository=repository + ) + prepared = PreparedAuthorizationService( + session, context_value, authorization, repository + ) + adapter = ProjectGuideCompilationAuthorizationAdapter(authorization, prepared) + return GuideCompilationService(session, adapter) + + +@pytest.mark.asyncio +async def test_authorized_request_commits_one_bound_receipt_and_exact_replay( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, link, _grant = await _seed_human(clean_postgres_database, values) + actor = ActorIdentityFacts(human, link, PublicActorKind.HUMAN) + facts = _request(values) + attempt_identity = identity(context(values)) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + receipt = await _authorized_service(session, actor).authorize_request( + actor=actor, facts=facts, identity=attempt_identity + ) + assert receipt.classification is CompilationRecoveryClassification.RESERVED + async with factory() as session: + replay = await _authorized_service(session, actor).authorize_request( + actor=actor, facts=facts, identity=attempt_identity + ) + counts = ( + await session.execute( + text( + "select (select count(*) from project_guide_compilation_attempts)," + "(select count(*) from project_guide_compilation_request_operations)," + "(select count(*) from audit_events where action_id=" + "'project.guide_compilation.request')" + ) + ) + ).one() + await session.rollback() + assert replay == receipt + assert counts == (1, 1, 1) + finally: + await engine.dispose() diff --git a/backend/tests/projects/guide_compilation/test_durable_dispatch_handoff.py b/backend/tests/projects/guide_compilation/test_durable_dispatch_handoff.py new file mode 100644 index 000000000..4a31e66f5 --- /dev/null +++ b/backend/tests/projects/guide_compilation/test_durable_dispatch_handoff.py @@ -0,0 +1,123 @@ +"""Committed dispatch handoff, cancellation, and provider-boundary proof.""" + +from __future__ import annotations + +import asyncio +import json +import os +from pathlib import Path +import sys + +import pytest +from sqlalchemy import text +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from app.modules.authorization.api import ActorIdentityFacts, ActorKind +from app.modules.projects.guide_compilation.service import GuideCompilationService + +from .helpers import context, identity, seed_database, service_actor +from .test_authorized_execution_service import _execution_service, _preflight +from .test_authorized_request_service import _authorized_service, _request, _seed_human + + +async def _requested(database_url: str): + values = await seed_database(database_url) + human, link, _grant = await _seed_human(database_url, values) + actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + engine = create_async_engine(database_url) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + receipt = await _authorized_service(session, actor).authorize_request( + actor=actor, + facts=_request(values), + identity=identity(context(values)), + ) + finally: + await engine.dispose() + return values, receipt + + +@pytest.mark.asyncio +async def test_fence_is_visible_to_a_fresh_process_before_any_provider_call( + clean_postgres_database: str, +) -> None: + values, requested = await _requested(clean_postgres_database) + service, facts = service_actor(values), _preflight(values, requested.attempt_id) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + receipt = await _execution_service(session, service).fence_dispatch( + actor=service, facts=facts + ) + script = ( + "import asyncio,asyncpg,json,os\n" + "async def main():\n" + " c=await asyncpg.connect(os.environ['WORKSTREAM_TEST_DATABASE_URL'].replace('+asyncpg',''));\n" + " r=await c.fetchrow('select status,provider_idempotency_key::text from " + "project_guide_compilation_attempts where id=$1::uuid',os.environ['ATTEMPT']);\n" + " await c.close();print(json.dumps(dict(r)))\n" + "asyncio.run(main())" + ) + process = await asyncio.create_subprocess_exec( + sys.executable, + "-c", + script, + env={ + **os.environ, + "WORKSTREAM_TEST_DATABASE_URL": clean_postgres_database, + "ATTEMPT": str(requested.attempt_id), + }, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + ) + stdout, stderr = await process.communicate() + assert process.returncode == 0, stderr.decode() + observed = json.loads(stdout) + assert observed == { + "status": "compilation_provider_uncertain", + "provider_idempotency_key": str(receipt.provider_idempotency_key), + } + finally: + await engine.dispose() + + +class _CancelledPreflight: + async def authorize_execute_preflight(self, **_kwargs): + raise asyncio.CancelledError + + +@pytest.mark.asyncio +async def test_cancellation_before_fence_commit_leaves_attempt_reserved( + clean_postgres_database: str, +) -> None: + values, requested = await _requested(clean_postgres_database) + service, facts = service_actor(values), _preflight(values, requested.attempt_id) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + with pytest.raises(asyncio.CancelledError): + await GuideCompilationService( + session, _CancelledPreflight() # type: ignore[arg-type] + ).fence_dispatch(actor=service, facts=facts) + async with factory() as session: + status = await session.scalar( + text("select status from project_guide_compilation_attempts where id=:id"), + {"id": requested.attempt_id}, + ) + await session.rollback() + assert status == "compilation_reserved" + finally: + await engine.dispose() + + +def test_coordinator_cannot_import_or_call_a_provider() -> None: + source = ( + Path(__file__).resolve().parents[3] + / "app/modules/projects/guide_compilation/service.py" + ).read_text(encoding="utf-8") + assert "compile_project_guide" not in source + assert "app.workers" not in source + assert "celery" not in source.lower() diff --git a/backend/tests/projects/guide_compilation/test_migration_authorized_persistence.py b/backend/tests/projects/guide_compilation/test_migration_authorized_persistence.py new file mode 100644 index 000000000..420930085 --- /dev/null +++ b/backend/tests/projects/guide_compilation/test_migration_authorized_persistence.py @@ -0,0 +1,98 @@ +"""Alembic 0008 topology, schema, and guarded downgrade proof.""" + +from __future__ import annotations + +import asyncio +from pathlib import Path + +from alembic import command +from alembic.config import Config +import asyncpg +import pytest +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from app.modules.projects.guide_compilation.repository import GuideCompilationRepository + +from .helpers import context, identity, seed_database + +pytestmark = pytest.mark.postgres_schema_contract +HEAD = "0008_guide_compilation_authorized_persistence" + + +def _config() -> Config: + return Config(Path(__file__).resolve().parents[3] / "alembic.ini") + + +async def _schema(database_url: str) -> tuple[str, int, int, int]: + connection = await asyncpg.connect(database_url.replace("+asyncpg", "")) + try: + return ( + await connection.fetchval("select version_num from alembic_version"), + await connection.fetchval( + "select count(*) from pg_proc where proname like " + "'project_guide_compilation_request_%_digest'" + ), + await connection.fetchval( + "select count(*) from pg_trigger where not tgisinternal and " + "tgrelid='project_guide_compilation_request_operations'::regclass" + ), + await connection.fetchval( + "select count(*) from pg_constraint where " + "conrelid='project_guide_compilation_request_operations'::regclass" + ), + ) + finally: + await connection.close() + + +async def _fresh_schema(database_url: str) -> None: + connection = await asyncpg.connect(database_url.replace("+asyncpg", "")) + try: + await connection.execute("drop schema public cascade; create schema public") + finally: + await connection.close() + + +async def _version(database_url: str) -> str: + connection = await asyncpg.connect(database_url.replace("+asyncpg", "")) + try: + return await connection.fetchval("select version_num from alembic_version") + finally: + await connection.close() + + +def test_0008_installs_exact_request_custody_and_round_trips_empty( + isolated_database_env: str, migration_lock +) -> None: + assert asyncio.run(_schema(isolated_database_env)) == (HEAD, 2, 3, 16) + with migration_lock(): + command.downgrade(_config(), "0007_contribution_policy_publication_custody") + assert asyncio.run(_version(isolated_database_env)) == ( + "0007_contribution_policy_publication_custody" + ) + asyncio.run(_fresh_schema(isolated_database_env)) + command.upgrade(_config(), HEAD) + command.upgrade(_config(), HEAD) + assert asyncio.run(_schema(isolated_database_env)) == (HEAD, 2, 3, 16) + + +async def _seed_attempt(database_url: str) -> None: + values = await seed_database(database_url) + engine = create_async_engine(database_url) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session, session.begin(): + await GuideCompilationRepository(session).reserve_attempt( + identity(context(values)) + ) + finally: + await engine.dispose() + + +def test_0008_refuses_downgrade_with_compilation_custody( + isolated_database_env: str, migration_lock +) -> None: + asyncio.run(_seed_attempt(isolated_database_env)) + with migration_lock(), pytest.raises(RuntimeError, match="custody is non-empty"): + command.downgrade(_config(), "0007_contribution_policy_publication_custody") + assert asyncio.run(_schema(isolated_database_env))[0] == HEAD diff --git a/backend/tests/projects/guide_compilation/test_migration_contract.py b/backend/tests/projects/guide_compilation/test_migration_contract.py index 378053f5f..0bfd58ac2 100644 --- a/backend/tests/projects/guide_compilation/test_migration_contract.py +++ b/backend/tests/projects/guide_compilation/test_migration_contract.py @@ -48,7 +48,7 @@ def test_current_schema_preserves_guide_compilation_schema( isolated_database_env: str, ) -> None: assert asyncio.run(_schema_state(isolated_database_env)) == ( - "0007_contribution_policy_publication_custody", + "0008_guide_compilation_authorized_persistence", True, 4, 1, diff --git a/backend/tests/projects/guide_compilation/test_public_authorization.py b/backend/tests/projects/guide_compilation/test_public_authorization.py index 56d626fcb..09d11fa2d 100644 --- a/backend/tests/projects/guide_compilation/test_public_authorization.py +++ b/backend/tests/projects/guide_compilation/test_public_authorization.py @@ -1,21 +1,17 @@ -"""Public fact integrity and inactive authorization behavior.""" +"""Public fact integrity and retired-seam boundary behavior.""" from __future__ import annotations +import ast from dataclasses import replace +from pathlib import Path from uuid import uuid4 import pytest -from app.modules.authorization.api import ( - AuthorizationUnavailable, - ProjectGuideCompilationRequestFacts, -) -from app.modules.projects.guide_compilation.authorization import ( - DenyProjectGuideCompilationAuthorization, -) +from app.modules.authorization.api import ProjectGuideCompilationRequestFacts -from .helpers import SHA256, context, identity, ids, persistence_facts, service_actor +from .helpers import SHA256, context, identity, ids, persistence_facts def _request_facts() -> ProjectGuideCompilationRequestFacts: @@ -60,24 +56,18 @@ def test_public_facts_reject_wrong_uuid_and_unbounded_token() -> None: replace(facts, source_snapshot_hash=SHA256.upper()) -@pytest.mark.asyncio -async def test_hidden_authorization_denies_before_touching_product_state() -> None: - """Every request and execute operation remains unavailable in 03A.""" - values = ids() - compilation_context = context(values) - attempt_identity = identity(compilation_context) - facts = persistence_facts(values, uuid4(), attempt_identity) - actor = service_actor(values) - denial = DenyProjectGuideCompilationAuthorization() - request_facts = _request_facts() - - with pytest.raises(AuthorizationUnavailable): - await denial.prepare_request(actor=actor, facts=request_facts) - with pytest.raises(AuthorizationUnavailable): - await denial.consume_request(handle=object(), actor=actor, facts=request_facts) - with pytest.raises(AuthorizationUnavailable): - await denial.authorize_execute_preflight(actor=actor, facts=facts) - with pytest.raises(AuthorizationUnavailable): - await denial.prepare_execute_persist(actor=actor, facts=facts) - with pytest.raises(AuthorizationUnavailable): - await denial.consume_execute_persist(handle=object(), actor=actor, facts=facts) +def test_retired_deny_seam_has_no_consumer_or_compatibility_module() -> None: + """The merged AUTH Protocol is the only compilation authority seam.""" + backend = Path(__file__).resolve().parents[3] + retired = "app.modules.projects.guide_compilation.authorization" + assert not (backend / "app/modules/projects/guide_compilation/authorization.py").exists() + for root in (backend / "app", backend / "tests"): + for path in root.rglob("*.py"): + if path == Path(__file__): + continue + tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path)) + for node in ast.walk(tree): + if isinstance(node, ast.Import): + assert all(alias.name != retired for alias in node.names), path + if isinstance(node, ast.ImportFrom): + assert node.module != retired, path diff --git a/backend/tests/projects/guide_compilation/test_request_operation_postgresql.py b/backend/tests/projects/guide_compilation/test_request_operation_postgresql.py new file mode 100644 index 000000000..151b4db1f --- /dev/null +++ b/backend/tests/projects/guide_compilation/test_request_operation_postgresql.py @@ -0,0 +1,446 @@ +"""PostgreSQL digest parity and insert-only request custody proof.""" + +from __future__ import annotations + +from dataclasses import replace +import json +from uuid import UUID, uuid4 + +import pytest +from sqlalchemy import text +from sqlalchemy.exc import DBAPIError +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from app.modules.authorization.api import ( + ActorIdentityFacts, + ActorKind, + project_guide_compilation_facts_digest, + project_guide_compilation_request_authority_digest, +) +from app.modules.projects.guide_compilation.repository import ( + GuideCompilationConcurrencyError, + GuideCompilationIntegrityError, + GuideCompilationRepository, + GuideCompilationStorageError, +) + +from .helpers import context, identity, seed_database +from .test_authorized_request_service import _authorized_service, _request, _seed_human + + +async def _create_request(database_url: str) -> tuple[dict[str, UUID], UUID, UUID, UUID]: + values = await seed_database(database_url) + human, link, grant = await _seed_human(database_url, values) + actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + engine = create_async_engine(database_url) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + await _authorized_service(session, actor).authorize_request( + actor=actor, + facts=_request(values), + identity=identity(context(values)), + ) + finally: + await engine.dispose() + return values, human, link, grant + + +@pytest.mark.asyncio +async def test_sql_and_python_request_digests_are_byte_identical( + clean_postgres_database: str, +) -> None: + values, human, link, grant = await _create_request(clean_postgres_database) + facts = _request(values) + expected_facts = project_guide_compilation_facts_digest(facts) + expected_authority = project_guide_compilation_request_authority_digest( + actor_profile_id=human, + identity_link_id=link, + grant_id=grant, + project_id=values["project"], + operation_id=values["operation"], + request_facts_digest=expected_facts, + ) + engine = create_async_engine(clean_postgres_database) + try: + async with engine.connect() as connection: + row = (await connection.execute(text(_DIGEST_QUERY))).one() + base_facts, base_authority = row + assert row == (expected_facts, expected_authority) + + operation_fields = { + "expected_predecessor_compilation_id", + "guide_id", + "idempotency_key", + "operation_id", + "project_id", + "request_id", + "setup_generation", + "setup_run_id", + "source_snapshot_id", + } + for field_name in facts.__dataclass_fields__: + original = getattr(facts, field_name) + mutated = _different_value(field_name, original) + sql_digest = await connection.scalar( + text(_MUTATED_FACTS_DIGEST_QUERY), + { + "operation_patch": json.dumps( + {field_name: _json_value(mutated)} + if field_name in operation_fields + else {} + ), + "attempt_patch": json.dumps( + {field_name: _json_value(mutated)} + if field_name not in operation_fields + else {} + ), + }, + ) + python_digest = project_guide_compilation_facts_digest( + replace(facts, **{field_name: mutated}) + ) + assert sql_digest == python_digest + assert sql_digest != base_facts + + authority_values = { + "actor_profile_id": human, + "identity_link_id": link, + "grant_id": grant, + "project_id": values["project"], + "operation_id": values["operation"], + "request_facts_digest": expected_facts, + } + operation_authority_fields = { + "actor_profile_id", + "identity_link_id", + "project_id", + "operation_id", + "request_facts_digest", + } + for field_name, original in authority_values.items(): + mutated = _different_value(field_name, original) + sql_digest = await connection.scalar( + text(_MUTATED_AUTHORITY_DIGEST_QUERY), + { + "operation_patch": json.dumps( + {field_name: _json_value(mutated)} + if field_name in operation_authority_fields + else {} + ), + "grant_patch": json.dumps( + {"id": str(mutated)} if field_name == "grant_id" else {} + ), + }, + ) + changed = authority_values | {field_name: mutated} + python_digest = project_guide_compilation_request_authority_digest( + **changed + ) + assert sql_digest == python_digest + assert sql_digest != base_authority + await connection.rollback() + finally: + await engine.dispose() + + +_DIGEST_QUERY = ( + "select project_guide_compilation_request_facts_digest(o,a)," + "project_guide_compilation_request_authority_digest(o,g) " + "from project_guide_compilation_request_operations o " + "join project_guide_compilation_attempts a on a.id=o.attempt_id " + "join audit_events e on e.id=o.authorization_decision_event_id " + "join admin_role_grants g on g.id=e.matched_grant_id::uuid" +) +_MUTATED_FACTS_DIGEST_QUERY = ( + "select project_guide_compilation_request_facts_digest(" + "jsonb_populate_record(o,cast(:operation_patch as jsonb))," + "jsonb_populate_record(a,cast(:attempt_patch as jsonb))) " + "from project_guide_compilation_request_operations o " + "join project_guide_compilation_attempts a on a.id=o.attempt_id" +) +_MUTATED_AUTHORITY_DIGEST_QUERY = ( + "select project_guide_compilation_request_authority_digest(" + "jsonb_populate_record(o,cast(:operation_patch as jsonb))," + "jsonb_populate_record(g,cast(:grant_patch as jsonb))) " + "from project_guide_compilation_request_operations o " + "join audit_events e on e.id=o.authorization_decision_event_id " + "join admin_role_grants g on g.id=e.matched_grant_id::uuid" +) + + +def _json_value(value: object) -> object: + return str(value) if isinstance(value, UUID) else value + + +def _different_value(field_name: str, value: object) -> object: + if field_name == "expected_predecessor_compilation_id" or isinstance(value, UUID): + return uuid4() + if field_name == "setup_generation": + return 2 + if field_name.endswith(("_hash", "_digest")): + return "sha256:" + "b" * 64 + return "mutated.v2" + + +@pytest.mark.parametrize( + ("mutation", "message"), + ( + ("facts", "request facts digest is invalid"), + ("authority", "request authority digest is invalid"), + ), +) +@pytest.mark.asyncio +async def test_request_insert_guard_rejects_stale_digest_evidence( + clean_postgres_database: str, mutation: str, message: str +) -> None: + await _create_request(clean_postgres_database) + engine = create_async_engine(clean_postgres_database) + try: + async with engine.connect() as connection: + transaction = await connection.begin() + await connection.execute( + text( + "create temporary table request_candidate on commit drop as " + "select * from project_guide_compilation_request_operations" + ) + ) + await connection.execute( + text( + "alter table project_guide_compilation_request_operations disable trigger " + "guide_compilation_request_change_guard" + ) + ) + await connection.execute( + text("delete from project_guide_compilation_request_operations") + ) + await connection.execute( + text( + "alter table project_guide_compilation_request_operations enable trigger " + "guide_compilation_request_change_guard" + ) + ) + if mutation == "authority": + await connection.execute( + text( + "alter table audit_events disable trigger " + "audit_events_reject_update_delete" + ) + ) + await connection.execute( + text( + "update audit_events set after_facts=jsonb_set(after_facts::jsonb," + "'{resource_context_digest}',to_jsonb(cast(:digest as text)))::json " + "where id=(select authorization_decision_event_id " + "from request_candidate)" + ), + {"digest": "sha256:" + "b" * 64}, + ) + await connection.execute( + text( + "alter table audit_events enable trigger " + "audit_events_reject_update_delete" + ) + ) + nested = await connection.begin_nested() + with pytest.raises(DBAPIError, match=message): + await connection.execute( + text( + "insert into project_guide_compilation_request_operations " + "select operation_id,request_id,idempotency_key,actor_profile_id," + "identity_link_id,project_id,guide_id,source_snapshot_id,setup_run_id," + "setup_generation,expected_predecessor_compilation_id," + "case when :mutation='facts' then :digest else request_facts_digest end," + "attempt_id,authorization_decision_event_id,created_at " + "from request_candidate" + ), + {"mutation": mutation, "digest": "sha256:" + "b" * 64}, + ) + await nested.rollback() + await transaction.rollback() + finally: + await engine.dispose() + + +@pytest.mark.parametrize( + "statement", + ( + "update project_guide_compilation_request_operations set setup_generation=2", + "delete from project_guide_compilation_request_operations", + "truncate table project_guide_compilation_request_operations", + ), +) +@pytest.mark.asyncio +async def test_request_operation_rejects_every_change( + clean_postgres_database: str, statement: str +) -> None: + await _create_request(clean_postgres_database) + engine = create_async_engine(clean_postgres_database) + try: + async with engine.begin() as connection: + with pytest.raises(DBAPIError, match="request custody is immutable"): + await connection.execute(text(statement)) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_required_repository_reads_fail_closed_without_durable_rows( + clean_postgres_database: str, +) -> None: + await seed_database(clean_postgres_database) + missing = uuid4() + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session, session.begin(): + repository = GuideCompilationRepository(session) + with pytest.raises(GuideCompilationIntegrityError, match="custody is missing"): + await repository.request_operation_for_attempt(missing, lock=False) + with pytest.raises(GuideCompilationIntegrityError, match="was not found"): + await repository.attempt(missing, lock=True) + with pytest.raises(GuideCompilationIntegrityError, match="disappeared"): + await repository.attempt(missing, lock=False) + with pytest.raises(GuideCompilationIntegrityError, match="is missing"): + await repository.persisted_compilation(missing) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_repository_read_views_return_exact_request_and_empty_lineage( + clean_postgres_database: str, +) -> None: + values, human, link, _grant = await _create_request(clean_postgres_database) + actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + facts = _request(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session, session.begin(): + repository = GuideCompilationRepository(session) + operation = await repository.matching_request_operation( + actor=actor, facts=facts, lock=False + ) + assert operation is not None + assert ( + await repository.request_operation_for_attempt( + operation.attempt_id, lock=False + ) + == operation + ) + assert ( + await repository.current_compilation( + values["project"], values["guide"], lock=False + ) + is None + ) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_request_failure_rolls_back_attempt_and_authority_event( + clean_postgres_database: str, monkeypatch: pytest.MonkeyPatch +) -> None: + """A failure after AUTH consumption leaves no partial request custody.""" + values = await seed_database(clean_postgres_database) + human, link, _grant = await _seed_human(clean_postgres_database, values) + actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + + async def fail_request_insert(*_args: object, **_kwargs: object) -> None: + raise GuideCompilationStorageError("injected request custody failure") + + monkeypatch.setattr( + GuideCompilationRepository, + "insert_request_operation", + fail_request_insert, + ) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + with pytest.raises(GuideCompilationStorageError, match="injected"): + await _authorized_service(session, actor).authorize_request( + actor=actor, + facts=_request(values), + identity=identity(context(values)), + ) + async with factory() as session: + counts = ( + await session.execute( + text( + "select (select count(*) from " + "project_guide_compilation_attempts)," + "(select count(*) from " + "project_guide_compilation_request_operations)," + "(select count(*) from audit_events where action_id=" + "'project.guide_compilation.request')" + ) + ) + ).one() + await session.rollback() + assert counts == (0, 0, 0) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_duplicate_request_insert_is_classified_as_concurrent_replay( + clean_postgres_database: str, +) -> None: + values, human, link, _grant = await _create_request(clean_postgres_database) + actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + facts = _request(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + with pytest.raises(GuideCompilationConcurrencyError, match="request won"): + async with session.begin(): + repository = GuideCompilationRepository(session) + attempt_id, event_id = ( + await session.execute( + text( + "select attempt_id,authorization_decision_event_id " + "from project_guide_compilation_request_operations" + ) + ) + ).one() + attempt = await repository.attempt(attempt_id, lock=True) + await repository.insert_request_operation( + actor=actor, + facts=facts, + attempt=attempt, + authorization_decision_event_id=UUID(event_id), + ) + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_unknown_request_custody_failure_is_not_reported_as_replay( + clean_postgres_database: str, +) -> None: + values = await seed_database(clean_postgres_database) + human, link, _grant = await _seed_human(clean_postgres_database, values) + actor = ActorIdentityFacts(human, link, ActorKind.HUMAN) + facts = _request(values) + engine = create_async_engine(clean_postgres_database) + factory = async_sessionmaker(engine, expire_on_commit=False) + try: + async with factory() as session: + with pytest.raises(GuideCompilationStorageError, match="before commit"): + async with session.begin(): + repository = GuideCompilationRepository(session) + _outcome, attempt = await repository.reserve_attempt( + identity(context(values)) + ) + await repository.insert_request_operation( + actor=actor, + facts=facts, + attempt=attempt, + authorization_decision_event_id=uuid4(), + ) + finally: + await engine.dispose() diff --git a/backend/tests/test_alembic.py b/backend/tests/test_alembic.py index 5f8c534ae..662422092 100644 --- a/backend/tests/test_alembic.py +++ b/backend/tests/test_alembic.py @@ -24,7 +24,7 @@ ) from scripts.schema_baseline_sql import split_sql_statements -HEAD_REVISION = "0007_contribution_policy_publication_custody" +HEAD_REVISION = "0008_guide_compilation_authorized_persistence" BASELINE_REVISION = "0001_v01_baseline" RECREATE_GUIDANCE = "Workstream v0.1 requires a fresh database; recreate this database" pytestmark = pytest.mark.postgres_schema_contract @@ -84,6 +84,7 @@ def test_v01_graph_has_one_root_and_head() -> None: assert [revision.revision for revision in revisions] == [ HEAD_REVISION, + "0007_contribution_policy_publication_custody", "0006_contribution_policy_operations", "0005_compensation_adapter_identity", "0004_compensation_adapter_binding_lifecycle", diff --git a/backend/tests/test_behavior_ownership.py b/backend/tests/test_behavior_ownership.py index 883364b1b..63b59d27c 100644 --- a/backend/tests/test_behavior_ownership.py +++ b/backend/tests/test_behavior_ownership.py @@ -479,11 +479,14 @@ def test_pol03a_targets_are_narrow_and_keep_declarative_model_unresolved() -> No """POL-03A admits only callable ownership while model metadata stays unresolved.""" assert ownership.POL_03A_CALLABLE_TARGETS == { "backend/app/modules/authorization/api/project_guide_compilation.py", - "backend/app/modules/projects/guide_compilation/authorization.py", "backend/app/modules/projects/guide_compilation/contracts.py", "backend/app/modules/projects/guide_compilation/repository.py", + "backend/app/modules/projects/guide_compilation/service.py", "backend/app/modules/projects/guide_compilation/validation.py", } + assert ownership.POL_03B_REMOVED_TARGETS == { + "backend/app/modules/projects/guide_compilation/authorization.py" + } assert ownership.POL_03A_DECLARATIVE_MODEL_TARGET not in ( ownership.POL_03A_CALLABLE_TARGETS ) diff --git a/backend/tests/test_ci_test_lanes.py b/backend/tests/test_ci_test_lanes.py index c1d06ef8d..f150e0e29 100644 --- a/backend/tests/test_ci_test_lanes.py +++ b/backend/tests/test_ci_test_lanes.py @@ -48,10 +48,17 @@ def test_measured_hotspots_have_explicit_semantic_owners() -> None: modules_by_lane = {lane.name: set(lane.modules) for lane in LANES} assert modules_by_lane["project_lifecycle"] == { + "tests/projects/guide_compilation/test_authorized_concurrency_postgresql.py", + "tests/projects/guide_compilation/test_authorized_execution_service.py", + "tests/projects/guide_compilation/test_authorized_recovery_postgresql.py", + "tests/projects/guide_compilation/test_authorized_request_service.py", "tests/projects/guide_compilation/test_contracts.py", "tests/projects/guide_compilation/test_database_guards.py", + "tests/projects/guide_compilation/test_durable_dispatch_handoff.py", + "tests/projects/guide_compilation/test_migration_authorized_persistence.py", "tests/projects/guide_compilation/test_migration_contract.py", "tests/projects/guide_compilation/test_public_authorization.py", + "tests/projects/guide_compilation/test_request_operation_postgresql.py", "tests/projects/guide_compilation/test_repository_attempts.py", "tests/projects/guide_compilation/test_repository_persistence.py", "tests/projects/test_locked_policy_context.py",