Skip to content

[LAB] Daily simulated-developer feedback for BogKit - #9

Closed
urcades wants to merge 24 commits into
mainfrom
ed/developer-simulation
Closed

[LAB] Daily simulated-developer feedback for BogKit#9
urcades wants to merge 24 commits into
mainfrom
ed/developer-simulation

Conversation

@urcades

@urcades urcades commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Note: directly inspired by: https://gist.github.com/VictorTaelin/d632f46aa55e561d3cd2c43c66f2813e

Purpose

This long-lived draft PR is the evidence archive and discussion thread for a daily simulated-developer feedback lab.

Every run gives two fresh developer personas realistic existing-software problems before they inspect BogKit. They begin with the public onboarding surface, build the smallest meaningful integration or no-fit reproducer, run it, and record the complete friction trail. A separate skeptical reviewer reproduces material claims and rejects unsupported or bloated recommendations.

Boundaries

  • Daily changes stay under developer-simulation/.
  • The automation never changes BogKit core or existing examples automatically.
  • A trial may conclude that BogKit is not a fit.
  • Generated data, credentials, databases, build output, and unjustified large dependencies are never archived.
  • Findings remain proposals until a maintainer explicitly approves separate implementation work.
  • This PR is intentionally long-lived and should remain draft rather than being merged.

Reporting

  • Each day gets one immutable PR comment containing the verified report.
  • One rolling dashboard comment tracks coverage, recurring evidence, confirmed defects, no-fit signals, and candidate improvements.
  • A suggestion becomes a candidate only after two independent trials or one serious reproducible correctness defect.

Inaugural run

The first two trials cover:

  1. A PostgreSQL purchase-approval audit timeline requiring one transaction and role-protected immutability.
  2. Offline warehouse scanner reconciliation under duplicate, reordered, interrupted, and conflicting uploads.

Both produced runnable archives. Neither justified adding BogKit to the specified production architecture. The repeated, evidence-backed improvement is a concise onboarding capability matrix describing embedded storage, transaction, single-writer, and component boundaries.

Validation

  • 15 inaugural trial tests passed.
  • Strict lint passed with warnings denied.
  • All archived Rust sources passed formatting checks.
  • Both demonstrations ran successfully.
  • The 20,000-operation synthetic threshold passed.
  • All 45 existing BogKit unit and documentation tests passed.
  • A skeptical review reproduced the major claims and removed overclaims and speculative API proposals.

See developer-simulation/README.md for the protocol and developer-simulation/reports/2026-07-28.md for the first full report.

urcades commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-07-28

The inaugural run tested two practical systems selected before the scenario designer saw BogKit. Neither justified adding BogKit to the stated production architecture. Both independently exposed the same onboarding gap around embedded storage, transaction boundaries, and single-writer deployment.

Trial 1 — Purchase-approval audit timeline

Persona: finance software developer maintaining a Rust/PostgreSQL service.
Outcome: no fit.
Archive: 2026-07-28--purchase-audit

The required state mutation and seven-year audit event must share one PostgreSQL transaction and role model. Fold owns a separate embedded transaction, so a sidecar creates a partial-commit window and making Fold authoritative violates a fixed requirement.

Evidence:

  • 8 tests passed, including real-Fold split-store and retraction reproductions.
  • Strict lint and source formatting passed.
  • The demo reproduced approved current state with only the older create event after a failure between stores.
  • The PostgreSQL-style local model cleared the 10,000-event threshold, but no real PostgreSQL durability, permissions, concurrency, or query plan was tested.

Friction and findings:

  • The public onboarding does not state the embedded/external transaction boundary early.
  • The starter example declares ESE and ANNy despite using only Fold, so the smallest example attempted an embedding-model download.
  • Intentional Bag retraction and ordering behavior are scenario mismatches, not Fold defects.

Decision audit: retain a conventional PostgreSQL audit table. Reject a Fold sidecar, a source-of-truth migration, and a redundant outbox-to-Fold copy. Do not infer an append-log terminal or PostgreSQL sink from this trial.

Trial 2 — Offline warehouse reconciliation

Persona: warehouse device developer with six months of Rust experience.
Outcome: useful local proof, no demonstrated BogKit advantage, no production fit.
Archive: 2026-07-28--offline-reconciliation

The proof safely stored immutable device operations, rejected divergent identity reuse, survived replay/interruption, and exposed conflicting locations. However, it scans and recomputes all stored operations rather than demonstrating incremental materialization, has no Fjall/SQLite/PostgreSQL control, and cannot replace the fixed horizontally deployed PostgreSQL service.

Evidence:

  • 7 tests passed, including 128 shuffled arrival orders, checkpointed reopen, and conflict visibility.
  • Strict lint and source formatting passed.
  • Three synthetic release observations processed 20,000 operations in 54, 57, and 76 ms; these show only that the proof clears five seconds.
  • The caught interruption prints a panic diagnostic before reopening and continuing; it is a harness caveat, not evidence for a recovery API.

Friction and findings:

  • Cross-device causality is missing from the scenario schema; device sequence and timestamps cannot establish whether moves are concurrent.
  • KeyedStream::upsert replacement is documented behavior. The existing transactional get-then-upsert path handled compare-and-reject safely.
  • The storage/concurrency mismatch again required reading beyond top-level onboarding.

Decision audit: keep the conservative reconciliation algorithm and tests, but implement production state in PostgreSQL. Do not add insert-if-absent, panic recovery, or a new storage backend based on this one proof.

Skeptical review and synthesis

The reviewer reproduced all 15 tests, both strict lint checks, both demos, and the release threshold. It removed overclaims and rejected four speculative API directions.

Promoted candidate: add a concise public capability matrix covering component purpose, embedded source-of-truth use, external database integration, transaction scope, and single-writer deployment. This is promoted because both independent trials encountered it.

Observation, not yet promoted: remove unused ESE and ANNy dependencies from the starter example.

Full report: developer-simulation/reports/2026-07-28.md

Validation: 15 trial tests and all 45 existing BogKit unit/documentation tests passed; no generated databases, build output, credentials, or binary fixtures were committed.

urcades commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

BogKit simulated-developer dashboard

This is the rolling index for the daily developer-simulation lab. Detailed daily comments are immutable; this dashboard is the only edited comment.

Current totals

  • Trials: 44
  • Adopted for production: 0
  • Useful local proof only: 4
  • No fit: 40
  • Blocked: 0
  • Latest reviewed branch commit: e239f2e

Coverage summary

  • Fold was considered in 44/44 trials and retained in 17 evidence prototypes.
  • ESE and ANNy were considered in 43/44 trials and retained together in 1 local proof.
  • The corpus spans external transaction boundaries, constrained and embedded systems, exact offline transformations, scheduling, security and release tooling, reconciliation, search, temporal repair, publication safety, hard-real-time constraints, crash-safe history, epoch-safe projections, financial calculation, safety-filtered technical retrieval, causal multi-writer collaboration, and cryptographic proof verification.
  • Storage, transaction, concurrency, operating-system, and external-authority boundaries recur independently in 38 trials.

Confirmed BogKit defects

  1. Caught Fold write-transaction panic poisons later writes — 2 independent sources. Drop the underlying transaction before resuming the user panic; add Stream and KeyedStream rollback-plus-recovery regressions.
  2. Persisted-value codec incompatibilities panic during reads — 1 serious source. Document Postcard-compatible value shapes and replace internal serialization/deserialization unwraps with recoverable typed errors where API compatibility permits.

Candidate improvements meeting the threshold

  1. Capability and operational-boundary matrix — 38 trials. State embedded authority, external transaction, writer topology, operating-system, durability, resource, and no-component boundaries concisely.
  2. Component-selective project setup — 16 trials. Let consumers start with only the relevant components and keep the smallest Fold path free of unused ESE/ANNy setup.
  3. Nameable pipeline and reader patterns — 7 trials. Document type aliases, function-pointer patterns, macros, and ordinary helper signatures before considering type erasure.
  4. Fold panic recovery — 2 independent serious reproducers. Make a caught user panic roll back without poisoning the next writer.
  5. Persisted-value error handling — 1 serious reproducer. Surface incompatible persisted shapes as explicit errors instead of read-time panics.
  6. Query-time filtered search — 2 independent sources. Document bounded post-filter loss, then evaluate a narrow allowed-key or query predicate without inferring a general search subsystem.
  7. Hermetic ESE model assets — 2 independent sources. Document model identity, language scope, cache, and first-build behavior; pin immutable assets and verify checksums.
  8. Fallible Fold storage and decode APIs — 2 independent sources. Add documented fallible variants for existing open, keyspace, read/decode, commit, and checkpoint boundaries while retaining convenience wrappers.

Latest run and positioning signals

  • August 18 tested exact laboratory-unit conversion admission and deterministic freight-capacity batch clearing. Both trials were no fit for their full briefs.
  • The skeptic found eight Important and two Minor prototype/report issues. All ten were fixed test-first; the normalized archive passed a separate final audit with zero discrepancies remaining.
  • Trial 1 still honestly fails its 256 MiB memory gate on this macOS host. Trial 2 clears the host-only threshold but lacks the declared Linux and Ruby comparison.
  • No new BogKit defect or candidate emerged; the existing confirmed-defect and candidate counts are unchanged.
  • Repeated no-fit results remain useful positioning evidence: exact immutable transforms stay application-owned when BogKit's embedded materialization does not remove meaningful work.
  • Findings remain proposals. Do not promote decimal, unit-conversion, market-clearing, external-sort, publisher, manifest, bounded-line, or generic use-none subsystems from these scenarios.

Daily report index

Date Immutable comment Branch report
2026-07-28 comment report
2026-07-29 comment report
2026-07-30 comment report
2026-07-31 comment report
2026-08-01 comment report
2026-08-02 comment report
2026-08-03 comment report
2026-08-04 comment report
2026-08-05 comment report
2026-08-06 comment report
2026-08-07 comment report
2026-08-08 comment report
2026-08-09 comment report
2026-08-10 comment report
2026-08-11 comment report
2026-08-12 comment report
2026-08-13 comment report
2026-08-14 comment report
2026-08-15 comment report
2026-08-16 comment report
2026-08-17 comment report
2026-08-18 comment report

urcades commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-07-29

Two blind trials tested underexplored operational limits: a strictly bounded edge-telemetry spool and raw-NOR firmware configuration storage. Both reached reviewed no-fit conclusions. The edge trial found useful Fold behavior before the hard disk-bound requirement failed; the firmware trial rejected BogKit before integration because its filesystem and allocation model does not match raw flash.

Trial 1 — Edge telemetry spool under disk pressure

Persona: site reliability engineer with intermediate Rust experience.
Outcome: no fit for the strict allocated-disk bound.
Archive: 2026-07-29--edge-spool-pressure

The supplied newline-file baseline modeled 1,000,000 events. Age-only deletion removed 6,894 critical events, and one modeled whole-file retry exposed up to 12,064 events to duplication. These are baseline defects, not BogKit defects.

The representative Fold candidate processed 20,000 events. It retained all 600 critical and 2,400 operational events, durably accounted for every debug drop, recovered exact retained IDs after two process-exit cases, and explained an interrupted upload as 25 possible duplicates with 7 actual duplicates observed by the mock collector.

The no-fit evidence is the quota prerequisite: a 1 MiB logical limit retained 1,048,512 logical bytes while the database allocated 3,305,472 bytes. This proves logical accounting is not a physical cap and the evaluated public interface has no documented hard allocated-byte guarantee. It does not predict exact allocation at 256 MiB or evaluate an external filesystem quota.

Evidence boundaries:

  • 10 tests passed; strict lint and targeted formatting passed.
  • The final release demo completed successfully.
  • The candidate stage demonstrated roughly 2,200–2,500 events/s with prototype checkpoint and verification work, below the 5,000-events/s burst. This is not a general Fold throughput claim.
  • Recovery time and 19,392–19,456 KiB sampled RSS are host observations; 50 ms sampling may miss short peaks.
  • Existing Ranked range scans were not evaluated, so the reviewer rejected the original range-scan API proposal and general linear-scan performance claim.

Decision audit: retain the priority and durable-intent ideas, but use a storage engine designed around a fixed ring or segment budget. Do not promote hard-quota controls, remove-by-key iteration changes, or new range primitives from this one trial.

Trial 2 — Power-fail-safe raw-NOR configuration journal

Persona: embedded controls developer with beginner-to-intermediate Rust experience.
Outcome: no fit.
Archive: 2026-07-29--flash-config-journal

The supplied in-place single-slot baseline loses its only valid configuration after a modeled interrupted erase. This is a baseline defect, not a BogKit defect.

BogKit was rejected because Fold opens a filesystem-backed Fjall database and does not expose raw read/program/erase operations, no_std, fixed whole-stack memory, wear accounting, or a 32-block recovery bound. The dependency-free reference journal keeps the active record untouched until a final one-way commit byte publishes its replacement.

Evidence:

  • 5 tests passed; strict lint and targeted formatting passed.
  • The emulator audited all 6,177 modeled write boundaries for one 2 KiB update and all 53,281 for one 24 KiB update. Each booted the complete old or complete new revision.
  • Deterministic corruption of revision 101 made the 128 KiB file-backed emulator recover complete revision 100.
  • A test of 10,000 fixed 2 KiB updates produced 312–313 erases per block, a 0.32% imbalance. Mixed-size wear was not tested.
  • The 1,024-byte value is an explicit-buffer design budget, not a measured whole-stack bound.
  • Host scan times remained below 50 ms but do not prove MCU timing or real NOR failure behavior.

Decision audit: the circular whole-record journal is runnable algorithm evidence, not a firmware implementation or production recommendation. Raw NOR support remains a one-off capability idea; the smallest BogKit improvement is an explicit firmware and no_std boundary.

Skeptical review and cross-run synthesis

The reviewer reproduced all 15 new tests, both decisive no-fit claims, both demonstrations, all 59,458 modeled flash boundaries, and the edge quota gap. It reclassified baseline failures, narrowed all host and emulator claims, rejected the edge range proposal, and accepted both prototypes for archival after those wording fixes.

No serious BogKit correctness defect was demonstrated.

Two candidates meet the independent-trial threshold:

  1. Public capability matrix — now reinforced by four trials. Cover component purpose, storage ownership, transaction scope, horizontal deployment, filesystem and std requirements, strict quotas, and raw-flash non-goals.
  2. Component-selective project scaffold — supported by three trials. Keep the smallest Fold introduction free of unused ESE and ANNy dependencies.

Hard quota controls, raw NOR support, remove-by-key iteration changes, and new range primitives remain below threshold.

Full report: developer-simulation/reports/2026-07-29.md

Validation: 15 new trial tests, all 30 nested lab tests, and all 45 BogKit unit/documentation tests passed. Changed paths are confined to developer-simulation/; no generated build output, databases, binary fixtures, credentials, or private keys were committed.

urcades commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-07-30

Two blind trials tested underexplored distributed-system boundaries: durable CI job leasing across three active coordinators, and explainable payment-velocity screening across four partitioned consumers. Both reached reviewed no-fit conclusions. The prototypes produced useful local evidence, but neither demonstrated the required replicated authority and stream-ownership model.

Trial 1 — Crash-safe CI job leasing

Persona: CI platform engineer with intermediate Rust experience.
Outcome: no fit.
Archive: 2026-07-30--ci-lease-coordinator

The periodic-JSON baseline acknowledged worker 7, restarted from the older snapshot, and gave worker 8 the same attempt-1 fence. The Fold reference then exercised the useful single-writer subset: atomic dependency readiness, coordinator-time leases, attempts, immutable results, and explicit rejection reasons.

Evidence:

  • 4 tests passed; strict lint and targeted formatting passed.
  • The final demo survived 100 child-process exits immediately after acknowledged mutations.
  • Duplicate and reordered messages were no-ops.
  • Heartbeat at the exact deadline and completion far after it were rejected without mutation; the job was reassigned as attempt 2 and attempt 1 could not overwrite the winner.
  • A second concurrent Fold opener was rejected. This is consistent with an embedded single-writer store, not a BogKit defect.
  • At 100,000 jobs, five local reopen-plus-lease observations were 0.162–0.170 seconds and the apparent persistent directory size was 16,309,086 bytes.
  • Three local 2,000-heartbeat passes measured 334,115–348,759 updates/s with 0.108–0.126 ms p99 assigned batch-commit latency. These are embedded-store upper bounds that exclude batch formation, networking, scheduling, object storage, and replica coordination.
  • A naïve 20,248,904-byte in-place JSON rewrite took 28–30 ms, but this is only a lower bound, not a crash-safe snapshot comparison.

Skeptical review found a real prototype defect before archival: a far-late heartbeat could revive an expired lease if the reaper had not run. The coordinator added the same deadline <= observed coordinator time rule to heartbeat and completion, added exact-deadline and far-past regression coverage, and reran every check.

Decision audit: retain Fold as evidence for a single-writer state machine, but reject it for the complete scenario because it supplies no consensus, leader fencing, replicated compare-and-swap, or safe three-active-replica authority. Do not infer a distributed coordination subsystem from this trial.

Trial 2 — Explainable payment-velocity screening

Persona: fraud infrastructure developer with strong event-processing experience and beginner Rust experience.
Outcome: no fit.
Archive: 2026-07-30--fraud-velocity-evaluation

The Redis-style baseline double-counted a retry, allowed arrival-time TTL expiry to disagree with a one-minute event-time relationship, and retained no exact contributor list. The dependency-free reference prototype modeled deterministic event-time decisions without claiming a BogKit integration.

Evidence:

  • 5 tests passed; strict lint and targeted formatting passed.
  • Ten deliveries became nine unique events with one inert duplicate.
  • One late event produced six linked corrections; latest indexed decisions matched a separate naïve scan.
  • Normal close/reopen replay produced 3,311 identical decision bytes. No crash or torn write was tested.
  • Fifteen alert outcomes were internally reconstructable from retained records; this is not an independent oracle for every historical correction revision.
  • Four account-owned events were scrubbed, and the final audit checked both account and card identifiers in retained rows and indexes while preserving shared device/IP counts.
  • Three sparse-key in-memory passes measured 951,640–1,080,871 deliveries/s, 0.002 ms p99, 134 corrections, 50,056 alert outcomes, and an identical digest. The required 30-minute, 20-million-event, real partition-transfer, and durable deletion-compaction gates were not run.

Fold does provide ranked event-time range scans. The no-fit conclusion is narrower: no inspected BogKit contract supplies the complete maintained semantics for multi-rule aggregation, linked revisions, two-ID deduplication, deletion, atomic durable-stream offsets, partition ownership, and state transfer across four consumers. Partial adoption would leave the consequential system boundary custom and unverified.

Decision audit: keep the runnable reference as algorithm evidence, but reject a small BogKit adoption for the stated production system. Do not infer a combined event-time correction operator, broker integration, or partition-transfer system from this trial.

Skeptical review and cross-run synthesis

The reviewer reproduced both baseline failures, both no-fit prerequisites, both demonstrations, the local measurements, the concurrent-open boundary, Fold's 18 unit and 9 documentation tests, and the artifact scans. It found and forced correction of the CI expiry bug; rejected the JSON rewrite as a correctness comparison; relabeled fraud replay as normal close/reopen; confirmed Fold's range primitives; strengthened the account/card deletion audit; and narrowed the sparse benchmark and explanation claims.

No serious BogKit correctness defect was demonstrated.

Three candidates meet the independent-trial threshold:

  1. Public capability matrix — now reinforced by six trials. Cover component purpose, storage ownership, transaction scope, process and horizontal deployment, filesystem and std requirements, strict quotas, raw-flash non-goals, and atomic broker-offset ownership.
  2. Component-selective project path — supported by four trials. Keep the smallest component introduction free of unrelated dependencies and setup where practical.
  3. Nameable pipeline and reader patterns — supported by two trials. Document type-alias, function-pointer, and ordinary helper-signature patterns; do not infer a new type-erasure subsystem.

Distributed consensus, broker-offset coordination, crash-recovery examples, and a combined event-time correction operator remain observations or positioning boundaries, not promoted core-feature proposals.

Full report: developer-simulation/reports/2026-07-30.md

Validation: 9 new trial tests, all 39 nested lab tests, and all 45 BogKit unit/documentation tests passed. Both demonstrations, targeted formatting, strict nested lint, ledger parsing, and diff checks passed. Changed paths are confined to developer-simulation/; no generated build output, databases, binary fixtures, credentials, private keys, or large files were committed.

urcades commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-07-31

Two blind trials tested underexplored ordinary-software boundaries: stable
home-care rescheduling after cancellation bursts, and schema evolution in an
existing SQLite wholesale catalog. Both reached reviewed no-fit conclusions
for their stated production boundaries. The scheduling prototype still showed
useful Fold behavior inside an isolated projection; the catalog baseline met
its compact requirements without a BogKit component.

Trial 1 — Stable home-care gap filling

  • Persona: scheduling-platform developer with intermediate Rust experience
  • Existing system: a single-node Rust service imports caregiver availability
    and visits into SQLite, then uses a full-rescan greedy scheduler
  • Problem: process cancellation bursts without destabilizing unaffected
    assignments, violating constraints, or losing deterministic explanations
  • Outcome: no fit for the SQLite-authoritative handoff as implemented
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: Fold in an isolated projection
  • Archive:
    runs/2026-07-31--homecare-gap-fill

Evidence

  • Commands:
    • cargo test -p homecare-gap-fill --all-targets --offline
    • cargo fmt --manifest-path runs/2026-07-31--homecare-gap-fill/Cargo.toml -- --check
    • cargo clippy -p homecare-gap-fill --all-targets --offline -- -D warnings
    • cargo run -p homecare-gap-fill --release --offline
  • Tests: 5 passed, including the reviewer-required regression for spurious
    travel-conflict explanations.
  • Demonstration:
    • The seeded representative run used 2,000 caregivers and 12,000 visits over
      14 days, exactly 10% of the requested full scale.
    • After 200 cancellation changes, the incremental path preserved 100% of
      unaffected assignments. The sampled full-rescan baseline preserved 98.294%.
    • Both paths filled 10,000 visits and 2,761 of 2,913 urgent visits.
    • Independent validation found zero assignment violations after the review
      fix and rejected an explicitly unfilled visit when a caregiver was actually
      eligible.
    • All measured runs produced deterministic digest 3a9719e0c8654377.
    • Normal reopen recovered in 47.200–50.022 ms across recorded runs. Recovery
      after a child committed, checkpointed, and then aborted took
      48.456–50.722 ms.
    • Across the author, reviewer, and post-fix runs, sampled baseline p95 was
      102.118–116.540 ms and incremental p95 was 5.244–5.517 ms.
  • Baseline comparison:
    • The baseline met fill and urgent-coverage parity but missed the 99.5%
      unaffected-assignment target.
    • The local-change algorithm met the stability target at representative
      scale. Fold atomically persisted the Fold-owned visits, outcomes, and
      counts, but did not provide the scheduling algorithm.
    • The required SQLite adapter, change feed, rebuild policy, idempotent handoff,
      and cross-store transaction were absent. The isolated Fold result therefore
      does not establish a safe production integration.

Friction trail

The simulator began with the root README and then read the starter,
time-series, chat, and search examples. It selected Fold only after the
full-rescan baseline missed the stability target. Scheduling constraints,
candidate selection, explanations, and the validator stayed in ordinary Rust;
ESE and ANNy were rejected as unrelated.

The first dependency resolution needed registry access. The first Fold-backed
store used anonymous closures whose composed reader type was awkward to name;
named functions made the store reusable. This is the third independent
occurrence of the existing pipeline/reader naming friction.

The skeptical reviewer then found that the independent validator returned
TRAVEL_CONFLICT whenever earlier filters left a caregiver, without checking
travel feasibility. A minimal one-caregiver reproducer was incorrectly
accepted. The coordinator added an independent travel check, made an eligible
unfilled visit an error, added a regression test, and reran the full trial.

Findings

  1. Prototype correctness defect, fixed — high severity, high confidence.
    The original validator accepted a spurious travel-conflict reason without
    proving travel blocked all remaining caregivers. Reproduction: the
    validator_rejects_spurious_travel_conflict test now captures the reviewer's
    reproducer. Smallest improvement: independently check travel and reject an
    unfilled outcome whenever any caregiver is fully eligible.
  2. Poor product fit — high severity, high confidence. Fold's isolated
    database does not share SQLite's authoritative transaction. Reproduction:
    inspect src/store.rs; there is no SQLite adapter or handoff. Smallest
    improvement: document source-of-truth and cross-store atomicity boundaries;
    keep this scenario no-fit until synchronization and rebuild behavior are
    proven.
  3. Performance evidence scope — medium severity, high confidence at
    representative scale.
    The local p95 ranges are repeatable but do not prove
    the 20,000-caregiver/120,000-visit, 1 GiB requirement. Reproduction: run the
    release demo. Smallest improvement: add memory-bounded, full-scale,
    multi-seed measurements before production use.
  4. Time-boundary documentation — medium severity, high confidence. The
    parser tests explicit-offset instant arithmetic and rejects offset-free
    strings; it does not apply IANA timezone rules or reject a nonexistent local
    wall time paired with an explicit offset. Reproduction: run the time test.
    Smallest improvement: put timezone-aware conversion and ambiguity policy at
    the unimplemented import boundary.
  5. API friction — low severity, high confidence. Named functions or type
    aliases were again needed for a reusable composed Fold store. Reproduction:
    replace the named pipeline functions with closures and compile. Smallest
    improvement: document ordinary type-alias and function-pointer patterns; do
    not infer a type-erasure subsystem.

Decision audit

The simulator retained the full rescan as a deterministic baseline, chose an
incremental local-change scheduler to protect published assignments, and kept a
separate validator rather than sharing scheduler eligibility code. Fold was
used only to measure atomic keyed persistence and maintained counts.

The reviewer rejected the initial “partial fit” label. SQLite must remain the
authority, and no synchronization or atomic publication boundary exists. The
final decision is no fit for the stated system, while retaining the isolated
projection as useful lower-bound evidence. Full scale, peak memory, real
SQLite integration, IANA-zone policy, mid-transaction crash injection, and
production distributions remain unresolved.

Trial 2 — Evolving industrial parts catalog

  • Persona: wholesale catalog backend developer with beginner-to-intermediate
    Rust experience
  • Existing system: an Axum and SQLite CRUD service with fixed product columns,
    handwritten SQL, and category conditionals
  • Problem: add structured category revisions, reliable filters, conditional
    patches, and resumable import without replacing the SQLite authority
  • Outcome: no fit for the authoritative catalog path
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-07-31--parts-catalog-evolution

Evidence

  • Commands:
    • cargo test -p parts-catalog-evolution --all-targets --offline
    • cargo fmt --manifest-path runs/2026-07-31--parts-catalog-evolution/Cargo.toml -- --check
    • cargo clippy -p parts-catalog-evolution --all-targets --offline -- -D warnings
    • cargo run -p parts-catalog-evolution --release --offline -- demo
    • cargo run -p parts-catalog-evolution --release --offline -- import-check 250000 2048
    • cargo run -p parts-catalog-evolution --release --offline -- burst 250000
    • cargo run -p parts-catalog-evolution --release --offline -- storage-check 250000 2048
  • Tests: 9 passed, including changed-source and conflicting-duplicate import
    regressions required by skeptical review.
  • Demonstration:
    • The unrelated legacy table survived catalog initialization.
    • A laptop revision-1 record remained readable after revision 2 activated,
      then migrated explicitly to revision 2.
    • Partial patching preserved omitted fields and rejected a stale version.
    • Three exact/range query shapes agreed with a separately written in-memory
      evaluator.
    • The post-fix full import stopped after 7,000 committed rows, reopened,
      validated its source fingerprint, and reached 250,000 total and distinct
      IDs in 8.623 seconds.
    • The post-fix burst measured 2.104 ms read p95 and 2.721 ms write p95. Across
      all recorded runs the ranges were 1.518–4.037 ms and 1.389–2.721 ms.
    • At 250,000 synthetic 2 KiB records, the indexed database used
      1,139,924,992 bytes versus 1,039,552,512 bytes for the baseline, or 1.097x.
  • Baseline comparison:
    • SQLite kept validation metadata, products, facets, conditional versions,
      and import checkpoints in one transactional file.
    • Fold would use a separate Fjall store and would not by itself supply runtime
      validation, SQLite transaction integration, or online category revisions.
    • A Fold sidecar's synchronization and storage cost were not measured; no
      conclusion is made about optional future analytics.

Friction trail

The simulator read the public README, examples, and relevant Fold operators
before selecting a component. The starter and search examples demonstrated
useful maintained views, but SQLite authority and evolving runtime data shapes
made those views an unnecessary second store for the compact exact/range slice.
ESE and ANNy were rejected because semantic search and recommendations were
explicit non-goals.

The sanitized copy did not have a cached maintained SQLite binding, so the
prototype used a narrow system-sqlite3 wrapper. This requires a linkable
system library and is not a production recommendation.

Skeptical review found that the original importer bound a job only to its total
count and used unchecked INSERT OR IGNORE. The reviewer resumed one job with
changed payload size and introduced a conflicting pre-existing ID; both paths
advanced and could report a mixed import as complete. The coordinator added a
generator/payload fingerprint, complete duplicate-content comparison, rollback
before checkpoint advancement, and two regression tests.

Findings

  1. Prototype correctness defect, fixed — high severity, high confidence.
    Count-only job identity and unchecked duplicate IDs could silently mix
    sources or accept conflicting content. Reproduction: run the two importer
    regression tests. Smallest improvement: persist a source fingerprint and
    verify complete duplicate content before advancing the checkpoint.
  2. Storage integration boundary — high severity, high confidence. Fold is
    not an extension of an existing SQLite transaction. Reproduction: public
    examples open a Fold-owned path. Smallest improvement: document source-of-
    truth ownership and a supported synchronization/rebuild pattern if one
    exists.
  3. Runtime schema mismatch — medium severity, high confidence. This compact
    prototype has three hardcoded Rust validators and one laptop revision; Fold
    pipelines are also concrete startup-time Rust types. Reproduction: inspect
    the validators and public pipeline construction. Smallest improvement:
    document runtime-defined schemas as a fit boundary rather than implying a
    new validation subsystem.
  4. Concurrency and HTTP evidence gap — medium severity, high confidence.
    One mutex serializes SQLite access, the route smoke test covers POST/GET, and
    no same-version race, second process, or real-service contract was tested.
    Reproduction: inspect app and the HTTP test. Smallest improvement: add
    route-level error coverage and a two-writer conflict test before claiming
    service compatibility.
  5. Performance and storage scope — medium severity, high confidence. The
    measured latency and 1.097x ratio cover in-process serialized requests and
    synthetic 2 KiB rows in three categories, not a 512 MiB VM, network traffic,
    mixed 2–20 KiB records, the real existing file, or a Fold sidecar.
    Reproduction: run the scale commands. Smallest improvement: repeat under the
    actual resource limit and data distribution.
  6. Prototype dependency risk — medium severity, high confidence. The direct
    SQLite wrapper is smaller and less mature than a maintained binding and
    requires system sqlite3. Reproduction: inspect src/sqlite.rs. Smallest
    improvement: use a maintained binding, migration tooling, pooled reads, and
    explicit error mapping for production.

Decision audit

The simulator rejected optional-column growth, unvalidated JSON, Fold as the
authority, a Fold sidecar for the compact exact/range slice, and ESE/ANNy. It
chose SQLite products, schema metadata, facet indexes, version checks, and
import checkpoints in one file.

The no-fit conclusion is limited to this authoritative catalog path. The real
HTTP contract, 30 runtime-defined categories, 512 MiB limit, mixed record sizes,
same-version races, crash or power-loss injection, scalar-facet policy for
deeper data, and production SQLite binding remain unresolved.

Skeptical review

  • Claims reproduced:
    • Both formatting checks, both strict lints, all original tests, and both
      release demonstrations.
    • Scheduling stability, fill parity, deterministic digest, and committed
      Fold-state recovery across repeated runs.
    • Catalog full-count import, 1.097x storage ratio, three query shapes, and
      repeated burst measurements.
  • Claims rejected or softened:
    • Scheduling “partial fit,” independent reason validation before its fix,
      IANA/DST language, full-scale performance, and cross-store atomicity.
    • Catalog crash wording, general CRUD compatibility, concurrent-writer
      evidence, general runtime schemas, real-data storage scope, and the
      unmeasured claim that a Fold sidecar would exceed the budget.
  • Quality fixes required and completed:
    • Independent travel feasibility plus a regression test.
    • Import source identity, duplicate-content checking, rollback, and two
      regressions.
    • Removal of the catalog's unused optional Fold dependency and relocation of
      its test-only HTTP body dependency.
    • Nested-workspace manifests and archive-only reproduction instructions.
  • Remaining uncertainty:
    • The full scheduling scale and memory limit; real SQLite/Fold handoff;
      timezone rules; uncommitted-transaction crash points.
    • The real catalog contract and data distribution; multi-process and
      same-version concurrency; runtime-defined validators; crash/power loss;
      production SQLite operations.

No BogKit correctness defect was demonstrated. Both serious findings were
prototype defects found by review and fixed before archival.

Cross-run synthesis

  • New evidence:
    • SQLite-to-Fold synchronization and source-of-truth ownership independently
      blocked both trials.
    • Runtime-defined schema validation and conditional update APIs remain
      one-trial observations.
  • Recurring evidence:
    • The public capability and operational-boundary matrix is now supported by
      eight independent trials.
    • Nameable pipeline and reader patterns are now supported by three
      independent trials.
    • The component-selective project path remains supported by four trials; the
      catalog's self-added optional Fold dependency was removed and is not counted.
  • Candidate improvements:
    1. Public capability matrix covering component purpose, source-of-truth
      storage, transaction and cross-store scope, process/horizontal deployment,
      operating-system requirements, strict limits, raw-flash non-goals, and
      broker ownership.
    2. Component-selective project path that avoids unrelated setup.
    3. Documentation for nameable composed pipeline and reader types.
  • Observations not yet promoted:
    • Runtime schema validation, conditional upsert, SQLite sidecar integration,
      pooled catalog concurrency, and scheduling-specific indexes.
  • No-fit or positioning signals:
    • BogKit's embedded store can provide useful isolated evidence without being
      safe for a system whose correctness boundary remains in SQLite. Public
      onboarding should make cross-store atomicity and rebuild ownership clear.

Validation

  • Trial-specific tests: 5 scheduling tests and 9 catalog tests passed.
  • Strict lint and formatting: both new crates passed targeted formatting and
    clippy with warnings denied. The complete nested workspace also passed strict
    clippy. A repository-wide format check still reports pre-existing differences
    in examples/search/src/main.rs; that file was not modified.
  • Runnable demonstrations: both demos and all three full-scale catalog commands
    passed after review fixes.
  • Nested lab workspace tests: all 53 tests passed.
  • BogKit root workspace tests: all 45 unit and documentation tests passed after
    allowing the ESE model download required by its build.
  • Archive and secret checks: git diff --check, coverage parsing, daily-marker
    uniqueness, changed-path scope, large-file, database, symlink, binary, and
    credential-pattern scans passed. Every intended changed path is under
    developer-simulation/; generated targets and databases are ignored.

urcades commented Aug 1, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-01

Two blind trials tested BogKit against substantially different one-shot local
software problems: deterministic offline feature-flag evaluation and preflight
validation of CNC job bundles. Both developers reached reviewed no-fit
conclusions. Direct, dependency-light implementations met the bounded local
criteria without Fold, ESE, or ANNy.

Skeptical review preserved both no-fit decisions but rejected broader claims
and found two high-severity CNC prototype defects before archival. The
coordinator bounded flag-snapshot reads, narrowed its memory and portability
claims, made CNC staging temporary and all-or-nothing for ordinary failures,
rechecked copied content, added regressions, and reran the evidence. No BogKit
correctness defect was demonstrated.

Trial 1 — Offline feature-flag parity

  • Persona: client-platform SDK developer, Rust beginner with production
    TypeScript experience
  • Existing system: a TypeScript kiosk SDK loads JSON snapshots, evaluates
    ordered targeting rules, and returns short decision explanations
  • Problem: preserve deterministic decisions and percentage assignments across
    reloads and restarts while rejecting malformed snapshots without replacing
    the last valid in-process configuration
  • Outcome: no fit for the bounded local evaluator
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-01--offline-flag-parity

Evidence

  • Commands:
    • cargo test -p offline-flag-parity --all-targets --offline
    • cargo fmt --manifest-path runs/2026-08-01--offline-flag-parity/Cargo.toml -- --check
    • cargo clippy -p offline-flag-parity --all-targets --offline -- -D warnings
    • cargo run -p offline-flag-parity --release --offline -- verify runs/2026-08-01--offline-flag-parity/fixtures
    • cargo run -p offline-flag-parity --release --offline -- demo runs/2026-08-01--offline-flag-parity/fixtures
    • node runs/2026-08-01--offline-flag-parity/baseline/reference-evaluator.mjs runs/2026-08-01--offline-flag-parity/fixtures
    • cargo run -p offline-flag-parity --release --offline -- generate-benchmark /private/tmp/offline-flag-parity-benchmark.json
    • cargo run -p offline-flag-parity --release --offline -- bench /private/tmp/offline-flag-parity-benchmark.json
  • Tests: 4 passed, including duplicate-key rejection, failed-reload retention,
    a fixed hash vector, and a bounded-file-read regression added after review.
  • Demonstration:
    • Rust and an independently written TypeScript-style JavaScript evaluator
      matched all 250 stored golden decisions and their explanations on the
      measured host.
    • Recursively reversing JSON object-key order left all 250 results unchanged;
      targeting rule arrays remained ordered and semantically significant.
    • All 12 malformed fixtures were rejected. A valid reload changed the active
      configuration, and the following invalid reload left that in-process
      configuration and its decision unchanged.
    • Ten fresh processes produced one fingerprint,
      99c04dd07bb96094.
    • Forty measured 20,000-evaluation batches across the developer and
      post-review release runs had a worst p95 of 1.333 microseconds, below the
      250-microsecond target.
    • The corrected 6.2 MB, 5,000-flag/50,000-rule benchmark had sampled
      same-size-reload RSS of 63,471,616–63,569,920 bytes in four fresh runs. It
      passed the 64 MiB criterion with only about 3.5 MB of closest observed
      headroom. This is measured prototype evidence, not a formal maximum.
  • Baseline comparison:
    • Parse and validate a complete candidate, then replace one immutable
      in-memory snapshot only after success.
    • Ordered local lookup, a specified integer hash, and structured explanations
      directly satisfy the compact evaluation path.
    • Fold can maintain durable incremental views, but it does not supply JSON
      admission or percentage semantics and would add unnecessary storage and
      lifecycle work to this bounded hot path.

Friction trail

The developer began with the root README and all four public examples, then
defined the immutable TypeScript-style baseline before inspecting Fold's
transaction and snapshot surface. The examples showed durable state,
transactions, aggregates, and search, but no need in this scenario justified a
BogKit component.

The first ordinary Cargo command attempted an index refresh in the
network-disabled copy; cached dependencies worked with --offline. A first
fixed-bucket assertion used an incorrect expected value and was corrected only
after an independent Node BigInt implementation matched Rust. The first large
benchmark retained a generic JSON tree and built fixtures in-process, reaching
146,587,648 bytes sampled RSS. Duplicate checking was made streaming and large
fixture generation moved into its own process before the passing measurements.

Skeptical review then found that fs::read could allocate an entire file before
the 48 MiB policy check and that a 6.2 MB workload did not justify the broad
memory wording. File reads now stop at 8 MiB plus one byte, a sparse oversized
file is a regression case, and every memory statement is limited to the
measured workload.

Findings

  1. Input-validation correctness — high severity, high confidence. Generic
    JSON-to-map parsing can accept duplicate object keys with last-value-wins
    behavior. Reproduction: run the duplicate-key unit test or verify
    fixtures/malformed/06-duplicate-flag-key.json. Smallest improvement: keep
    a streaming duplicate-key preflight or use a parser that rejects duplicates.
    This is a configuration-admission concern, not a BogKit defect.
  2. Prototype memory-bound defect, fixed — high severity, high confidence.
    The original whole-file read occurred before its size check and its 48 MiB
    cap exceeded the measured evidence. Reproduction: the
    file_reader_stops_at_the_snapshot_limit test uses an oversized sparse file.
    Smallest improvement: cap the read itself and state only the measured memory
    envelope.
  3. Poor product fit — informational severity, high confidence for this
    boundary.
    The direct immutable evaluator met the bounded need without
    durable incremental state, embeddings, or nearest-neighbor search.
    Reproduction: compare the baseline, manifest, and public examples. Smallest
    improvement: document component purposes and no-fit boundaries; do not infer
    a new subsystem.
  4. Performance and portability scope — medium severity, high confidence.
    The local latency and sampled RSS are repeatable, but the memory margin is
    narrow and one arm64 macOS host running Rust and Node is not a platform
    matrix. Reproduction: run the release benchmark and reference evaluator.
    Smallest improvement: test real snapshots and a platform CI matrix before
    production claims.
  5. Persistence boundary — medium severity, high confidence. Failed reloads
    preserve the active snapshot only inside the current process. Reproduction:
    inspect Evaluator::reload_file; no durable last-known-good publication is
    implemented. Smallest improvement: assign persisted delivery ownership
    before considering an embedded store.

Decision audit

The developer kept ordered rules, order-insensitive JSON objects, strict
candidate validation, a versionable FNV-1a 64 bucket contract, and explanations
on every decision. Rust's process-randomized map hash, rule sorting, partial
mutation, per-decision persistence, and all commercial flag formats were
rejected.

The no-fit conclusion is limited to the bounded offline evaluator. Actual kiosk
snapshots and coercion rules, concurrent reloads, long-duration allocation,
platform parity, fuzzing, and durable last-known-good delivery remain
unresolved.

Trial 2 — CNC job-bundle preflight

  • Persona: manufacturing software engineer with intermediate Rust experience
  • Existing system: a Python ZIP preflight checks for manifest.json and
    allowed filename extensions before a bundle reaches an on-premises
    controller
  • Problem: classify incomplete or inconsistent bundles, stream size and digest
    checks, produce deterministic diagnostics, and stage only a completely
    rechecked valid bundle
  • Outcome: no fit for the one-shot bundle gate
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-01--cnc-job-bundle-preflight

Evidence

  • Commands:
    • cargo test -p cnc-job-bundle-preflight --all-targets --offline
    • cargo fmt --manifest-path runs/2026-08-01--cnc-job-bundle-preflight/Cargo.toml -- --check
    • cargo clippy -p cnc-job-bundle-preflight --all-targets --offline -- -D warnings
    • cargo run -p cnc-job-bundle-preflight --release --offline -- generate-fixtures /private/tmp/cnc-preflight-fixtures --include-huge
    • cargo run -p cnc-job-bundle-preflight --release --offline -- demo /private/tmp/cnc-preflight-fixtures /private/tmp/cnc-preflight-staging
    • cargo run -p cnc-job-bundle-preflight --release --offline -- check /private/tmp/cnc-preflight-fixtures/thousand-files.zip --tools /private/tmp/cnc-preflight-fixtures/tools.json
    • /usr/bin/time -l target/release/cnc-job-bundle-preflight check /private/tmp/cnc-preflight-fixtures/oversized-2gib-sparse.zip --tools /private/tmp/cnc-preflight-fixtures/tools.json --staging /private/tmp/cnc-preflight-staging/oversized
  • Tests: 6 passed, including skeptical-review regressions for file/parent path
    collisions, incomplete-output cleanup, and content rechecking before final
    naming.
  • Demonstration:
    • The required valid, truncated, checksum-mismatched, undeclared, missing,
      case-colliding, absolute-path, parent-traversal, missing-tool, and
      multi-error fixtures were classified as expected. Only the valid bundle
      received a final ready directory.
    • Two complete multi-error outputs had identical SHA-256 digests and contained
      15 stable-sorted diagnostics.
    • A bundle with exactly 1,000 declared files passed with 1,001 archive members
      including its manifest and 172,118 streamed bytes.
    • The final archived-workspace 2 GiB sparse-member run streamed and hashed
      2,147,483,919 bytes including the manifest in 17.00 seconds. It reported 1,998,848 bytes
      maximum resident set, returned only the provisional archive_oversized
      decision, and created no staged output.
    • A selected staging root that was already a symbolic link failed closed and
      left its target empty.
  • Baseline comparison:
    • Python 3.14.6 rejected the physically truncated ZIP on open.
    • The filename-only baseline still marked checksum mismatch, undeclared and
      missing files, case collisions, unsafe names, a missing tool, and the
      multi-error bundle ready.
    • A direct streaming parser and manifest gate address this one-shot boundary;
      durable views, embeddings, and nearest-neighbor search do not.

Friction trail

The developer froze the runnable filename-only Python baseline before reading
BogKit implementation details. Public examples did not address archive parsing,
manifest validation, bounded streaming, or filesystem staging, so no BogKit
component was selected.

To remain offline, the prototype used Serde plus local classic stored-ZIP,
SHA-256, and CRC32 code. Formatting and strict lint exposed ordinary quality
issues before the first passing run. Python corrected the developer's predicted
truncated-file baseline result. Sandboxed memory tools did not provide usable
process accounting; a fresh read-only timed run supplied the accepted
measurement.

Skeptical review found two high-severity prototype defects: a late write failure
could leave incomplete output carrying the final ready name, and the content
copied during staging was not compared with the already validated first read.
Staging now writes to a temporary directory, cleans incomplete output, rechecks
byte count, CRC, and SHA-256, and renames only after all entries succeed. The
review also required early entry-count and member-name bounds.

Findings

  1. Baseline correctness defects — critical severity, high confidence. The
    filename-only baseline accepts unsafe names plus content, declaration, size,
    and tool mismatches. Reproduction: run baseline.py over the generated
    fixtures. Smallest improvement: validate all names and a bounded manifest,
    then stream declared content before readiness. These are baseline defects,
    not BogKit defects.
  2. Prototype correctness defect, fixed — high severity, high confidence. A
    late staging failure could leave incomplete output named ready.
    Reproduction: the late_copy_failure_leaves_no_ready_or_pending_directory
    regression. Smallest improvement: stage to a temporary directory, clean it
    on failure, and rename only after completion.
  3. Prototype correctness defect, fixed — high severity, high confidence.
    The original second read for staging was not rechecked against validated
    content. Reproduction: the
    copied_content_is_rechecked_before_ready_is_named regression. Smallest
    improvement: verify byte count, ZIP CRC, and declared SHA-256 on the copied
    stream before final naming.
  4. Compatibility limitation — medium severity, high confidence. The local
    parser intentionally rejects compression, ZIP64, data descriptors,
    encryption, and multi-disk input and has no independent corpus or fuzzing.
    Reproduction: inspect src/archive.rs. Smallest improvement: use a mature
    bounded streaming library before production; keep the archived subset
    explicit.
  5. Product-policy gap — medium severity, high confidence. The provisional
    1 GiB total-member limit conflicts with the brief's occasional 2 GiB jobs.
    Reproduction: the sparse fixture is rejected only as oversized. Smallest
    improvement: make the local disk and size policy an explicit reviewed
    configuration before deployment.
  6. Poor product fit — informational severity, high confidence. No BogKit
    component improves the one-shot trust and copy boundary demonstrated here.
    Reproduction: compare the baseline and public examples. Smallest improvement:
    retain a direct tool; do not infer archive APIs in BogKit.

Decision audit

The developer validated before staging, streamed members with bounded buffers,
required exact UTF-8 relative names, rejected case collisions, checked manifest
references and tool inventory, and used create-new destinations. Extraction
before validation, member-sized allocation, a shell extractor, bundle repair,
and forced BogKit adoption were rejected.

After review, a completed temporary directory is renamed to ready only after
copy rechecks. Existing destinations fail closed. Coordinated filesystem access,
Unicode normalization, mature ZIP interoperability, crash persistence, the
correct 2 GiB policy, and G-code semantics remain unresolved.

Skeptical review

  • Claims reproduced:
    • Both targeted formatting checks, all 10 corrected tests, strict lints, and
      release demonstrations.
    • Flag golden/reference parity on one host, object-order invariance,
      malformed-reload retention, restart fingerprints, repeated local latency,
      and the corrected bounded-file read.
    • CNC required classifications, stable diagnostics, 1,000-file boundary,
      complete 2 GiB sparse stream, bounded measured RSS, and symbolic-link-root
      failure.
  • Claims rejected or softened:
    • Cross-machine flag parity, a whole-input 64 MiB guarantee, persistent
      last-known-good recovery, and any formal maximum-memory claim.
    • CNC production readiness, broad ZIP compatibility, complete filesystem
      coordination, and the initial staging-safety claim before its fixes.
  • Quality fixes required and completed:
    • Bounded flag file reads and an oversized sparse-file regression.
    • Temporary CNC staging, incomplete-output cleanup, per-copy byte/CRC/SHA-256
      checks, final rename, path-type collision detection, and early metadata
      bounds, all with regressions.
  • Remaining uncertainty:
    • Real flag schema parity, platform matrix, concurrent reloads, persisted
      delivery ownership, fuzzing, and the narrow observed memory margin.
    • Mature ZIP interoperability, independent parser corpus/fuzzing, coordinated
      filesystem access, Unicode normalization, crash persistence, and the local
      size policy.

No BogKit correctness defect was demonstrated. The serious findings were
prototype defects fixed before archival.

Cross-run synthesis

  • New evidence:
    • Two independent developers rejected BogKit for compact one-shot local
      validation paths after comparing direct baselines with public components.
    • Immutable configuration admission and manufacturing bundle staging remain
      domain-specific observations, not proposed BogKit subsystems.
  • Recurring evidence:
    • The public capability and operational-boundary matrix is now supported by
      ten independent trials.
    • The component-selective project path remains supported by four trials.
    • Nameable pipeline and reader patterns remain supported by three trials.
  • Candidate improvements:
    1. Public capability matrix covering component purpose, source-of-truth
      storage, transaction and process scope, deployment, operating-system
      requirements, hard limits, and clear one-shot/no-component boundaries.
    2. Component-selective project path that avoids unrelated setup.
    3. Documentation for nameable composed pipeline and reader types.
  • Observations not yet promoted:
    • A configuration-snapshot example, explicit no-component workspace
      isolation, ZIP/bundle validation guidance, and persisted flag delivery are
      one-trial or lab-specific observations.
  • No-fit or positioning signals:
    • BogKit's durable incremental data tools need not be present in compact
      immutable evaluators or one-shot validation gates. Public onboarding should
      make component purpose clear enough that developers can make that decision
      without reading implementation details.

Validation

  • Trial-specific tests: 4 flag tests and 6 CNC tests passed, including all
    skeptical-review regressions.
  • Strict lint and formatting: both targeted formatting checks and strict lints
    passed; the full nested workspace passed strict Clippy with warnings denied.
  • Runnable demonstrations: flag verification, the flag demo, Node reference,
    restart fingerprints, repeated release benchmark, CNC normal corpus, stable
    diagnostics, 1,000-file boundary, and final 2 GiB stream all passed their
    expected exit and evidence checks.
  • Nested lab workspace tests: all 63 tests passed.
  • BogKit root workspace tests: all 45 unit and documentation tests passed after
    allowing ESE's required build-model download.
  • Archive and secret checks: git diff --check, coverage JSON parsing,
    marker uniqueness, changed-path scope, symlink, generated-target, database,
    archive, model, binary, credential-pattern, and file-size scans passed. The
    new archives are 360 KiB and 104 KiB, and every intended changed path is
    under developer-simulation/.

Full archived report: developer-simulation/reports/2026-08-01.md

urcades commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-02

Two blind trials tested safety boundaries around external authorities: ambiguous
carrier-label purchases whose workflow state remains in PostgreSQL, and bounded
garbage collection over externally published backup manifests. Both developers
reached reviewed no-fit conclusions. Direct prototypes met the bounded local
criteria without Fold, ESE, or ANNy, but neither constitutes a production
integration with the stated authoritative system.

Skeptical review preserved both no-fit decisions while finding one serious
correctness defect in each prototype. The carrier journal accepted an incomplete
tail but did not remove it before later appends. The backup publisher checked its
final name before, rather than while holding, the publication lock. Both defects
were reproduced, fixed with regressions, and fully revalidated before archival.
No BogKit correctness defect was demonstrated.

Trial 1 — Carrier-label ambiguity

  • Persona: fulfillment-platform engineer with intermediate Rust and strong
    TypeScript experience
  • Existing system: a TypeScript service purchases labels through carrier APIs,
    stores workflow state in PostgreSQL, retries jobs, and consumes callbacks
  • Problem: a request may time out after the carrier charged and created a label;
    a retry can buy a duplicate while callbacks and reconciliation race
  • Outcome: no fit for the PostgreSQL-authoritative reliability core
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-02--carrier-label-ambiguity

Evidence

  • Commands:
    • cargo test -p carrier-label-ambiguity --all-targets
    • cargo fmt --manifest-path runs/2026-08-02--carrier-label-ambiguity/Cargo.toml -- --check
    • cargo clippy -p carrier-label-ambiguity --all-targets -- -D warnings
    • cargo run -p carrier-label-ambiguity --release -- demo --dir /private/tmp/carrier-label-demo/run
    • cargo run -p carrier-label-ambiguity --release -- crash-demo --dir /private/tmp/carrier-label-crashes/run
    • cargo run -p carrier-label-ambiguity --release -- acceptance --dir /private/tmp/carrier-label-acceptance/run --shipments 20000 --seeds 30
    • runs/2026-08-02--carrier-label-ambiguity/scripts/measure-acceptance.sh target/release/carrier-label-ambiguity /private/tmp/carrier-label-measured/run /private/tmp/carrier-label-measured/output.log
  • Tests: 3 passed, including callback order/duplication, a three-seed
    end-to-end fixture, and the reviewer-required partial-tail → reopen → later
    commit → second-reopen regression.
  • Demonstration:
    • The small run completed 100 shipments with 10 ambiguous timeouts, 94 paid
      labels, 5 review outcomes, 591 replayed decisions, and four simulated
      restarts.
    • Four real child-process exits around intent, carrier creation, confirmation,
      and callback persistence recovered with one attempt and zero automatic
      repurchases.
    • The corrected 30-seed fixture completed 600,000 shipments, made exactly one
      simulated carrier purchase call per shipment, replayed 3,478,477 decisions,
      exposed 30,154 inconclusive outcomes as needs_review, and left no shipment
      nonterminal.
    • Maximum simulated convergence was 30 seconds, below the 60-second criterion.
    • Developer, reviewer, and coordinator runs took 2.795–3.498 seconds and
      sampled 11.48–18.20 MiB peak RSS on one arm64 macOS host, below 256 MiB.
  • Baseline comparison:
    • The supplied unsafe retry policy can purchase again after a timeout that
      already created a paid label.
    • The prototype persists one intent before the network call, never purchases
      again after that intent exists, and resolves missing evidence through carrier
      lookup, a matching callback, or explicit human review.
    • The local journal models an atomic PostgreSQL row-plus-history transaction;
      it is not a PostgreSQL, network, multi-worker, or carrier integration test.
    • Fold persists local incremental state but cannot make the carrier HTTP
      operation atomic with PostgreSQL and would introduce a second authority.

Friction trail

The developer began with the public root README and then read the starter,
time-series, chat, and search examples. Fold's transactional writes, consistent
reads, and maintained views were clear, while ESE and ANNy addressed unrelated
search work. The required correctness boundary remained between PostgreSQL and an
external carrier, so the developer selected no BogKit dependency.

The first implementation retained decoded history and repeatedly rescanned it.
The developer observed 293.66 MiB RSS, then moved verification to streaming replay
with compact per-shipment counters. That discarded implementation was unavailable
to the reviewer, so the historical number is not treated as independently
reproduced evidence.

Skeptical review then appended an incomplete final journal record, reopened
successfully, committed another decision, and reproduced a checksum failure on the
next reopen. The coordinator now truncates and syncs the recognized incomplete
tail before returning a journal that may append. Review also found that the memory
script could emit a false zero when process inspection was denied; it now fails
without at least one valid sample.

Findings

  1. Baseline duplicate-purchase risk — critical severity, high confidence.
    Automatic repurchase after an ambiguous timeout can create a second paid label.
    Reproduction: run the deterministic acceptance fixture and inspect ambiguous
    carrier-created outcomes. Smallest improvement: persist one attempt before the
    request and prohibit repurchase whenever an attempt exists.
  2. Unknown must be durable review state — critical severity, high confidence.
    The corrected fixture routed 30,154 inconclusive outcomes to review without a
    second purchase. Reproduction: run the 30-seed acceptance command. Smallest
    improvement: make ambiguity visible rather than interpreting missing evidence as
    permission to retry.
  3. Prototype partial-tail defect, fixed — high severity, high confidence. The
    original reopen ignored an incomplete tail but did not truncate it, so a later
    append poisoned the journal. Reproduction: run
    journal_repairs_a_partial_tail_before_later_commits. Smallest improvement:
    truncate and sync the recognized tail before later writes.
  4. Callback evidence scope — high severity, medium confidence. Matching
    duplicate/reordered callbacks are monotonic, but conflicting transactions or
    prices, callback authentication, callback-before-creation, and concurrent
    reconciliation are untested. Reproduction: inspect the reducer test. Smallest
    improvement: define and test the production conflict policy before integration.
  5. Poor product fit — high severity if forced, high confidence for this boundary.
    Fold does not own carrier/PostgreSQL atomicity, reconciliation, or review state.
    Reproduction: compare the public Fold ownership model with the scenario's two
    authorities. Smallest improvement: make authority and transaction boundaries
    prominent in public onboarding; do not infer a workflow subsystem.
  6. Durability and performance scope — medium severity, high confidence. The
    evidence covers sequential simulation and ordinary process exits after completed
    local sync calls on one host, not power loss, production latency, or concurrent
    workers. Reproduction: run the demos and measurement. Smallest improvement: add
    database-enforced attempt uniqueness plus PostgreSQL/network/concurrency and
    power-failure tests before production claims.

Decision audit

The developer chose intent-before-network, no automatic repurchase, authoritative
carrier evidence only when transaction and price agree, monotonic state changes,
and atomic state-plus-history commits in the journal model. Automatic retry,
absence-as-rejection, Fold as workflow authority, a Fold audit sidecar, retained
decoded history, refunds, and carrier selection were rejected.

The no-fit conclusion is limited to this PostgreSQL-authoritative reliability core.
Carrier lookup semantics, real HTTP/database shapes, database-enforced concurrency,
callback trust/conflicts, multi-worker races, kernel or power failure, and production
deployment remain unresolved.

Trial 2 — Snapshot garbage-collection safety

  • Persona: backup-tools maintainer, Rust beginner with production Python experience
  • Existing system: a self-hosted daemon stores content-addressed blobs on POSIX and
    publishes append-only JSONL snapshot manifests; a Python collector loads all
    referenced hashes before direct deletion
  • Problem: bound collector memory while preventing malformed, newly published, or
    crash-interrupted state from deleting live blobs
  • Outcome: no fit for the external manifest/publication/quarantine protocol
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-02--snapshot-gc-safety

Evidence

  • Commands:
    • cargo test -p snapshot-gc-safety --all-targets --offline
    • cargo fmt --manifest-path runs/2026-08-02--snapshot-gc-safety/Cargo.toml -- --check
    • cargo clippy -p snapshot-gc-safety --all-targets --offline -- -D warnings
    • cargo build -p snapshot-gc-safety --release --offline
    • python3 runs/2026-08-02--snapshot-gc-safety/acceptance.py target/release/snapshot-gc-safety
    • target/release/snapshot-gc-safety fixture /private/tmp/snapshot-gc-scale
    • python3 runs/2026-08-02--snapshot-gc-safety/measure.py --scratch /private/tmp/snapshot-gc-scale/.snapshot-gc -- target/release/snapshot-gc-safety plan /private/tmp/snapshot-gc-scale realistic
    • target/release/snapshot-gc-safety verify-plan /private/tmp/snapshot-gc-scale realistic
    • python3 runs/2026-08-02--snapshot-gc-safety/measure.py -- target/release/snapshot-gc-safety apply /private/tmp/snapshot-gc-scale realistic
    • python3 runs/2026-08-02--snapshot-gc-safety/measure.py -- target/release/snapshot-gc-safety resume /private/tmp/snapshot-gc-scale realistic
  • Tests: 3 Rust tests passed. The Python acceptance harness passed all 30 seeded
    oracle repositories, malformed guards, publication during planning, publication
    from quarantine, the reviewer-required forced same-name contention, every tested
    quarantine/finalization crash boundary, and sequential idempotence.
  • Demonstration:
    • The post-fix requested-scale plan read 1,000,000 references across 10,000
      manifests and 300,000 blob entries, then selected all 51,000 eligible
      unreachable blobs and zero referenced blobs.
    • Planning took 1.485 seconds wall time, sampled 6,160,384 bytes peak RSS, and
      used 38,376,656 bytes peak scratch against a 76,000,000-byte logical manifest
      corpus.
    • Apply quarantined all 51,000 candidates in 6.864 seconds. Resume removed all
      51,000 in 2.963 seconds, restored none, and ended complete.
    • A forced same-name race now yields exactly one successful publisher and one
      existing-name failure; the successful manifest remains intact.
  • Baseline comparison:
    • The frozen Python baseline directly deletes and has no quarantine, recovery,
      or publisher fence.
    • Its memory was highly variable: the developer observed 183,648,256 bytes,
      while review sampled 124,436,480–133,971,968 bytes. The largest reviewed run
      was only 245,760 bytes below 128 MiB, so the evidence supports inadequate
      headroom, not a stable threshold breach.
    • Fold would duplicate external manifest state and does not provide cooperative
      publisher fencing, recoverable quarantine, or bounded external sorting.

Friction trail

The developer read the public README and all four examples, then froze a runnable
Python baseline before inspecting Fold's public persistence model. Atomic local
transactions and snapshots were useful concepts, but no component owned the
external manifest format, publication lock, quarantine, or bounded set difference.
The prototype therefore used the standard library plus justified serde_json.

The first online dependency resolution failed without registry access; cached
dependencies worked offline. Strict lint corrected two file-open and extension
checks. The developer moved unnecessary per-object directory syncs to phase commits
and reran the crash matrix.

Skeptical review forced two cooperative publishers to wait behind the publication
lock after both passed the pre-lock final-name check. Both originally reported
success, the second replaced the first append-only manifest, and later collection
deleted the blob referenced only by the lost publication. The coordinator moved the
final-name check under the lock, cleans the losing temporary file, and added a
deterministic contention regression.

Findings

  1. Baseline direct-deletion race — critical severity, high confidence. A new
    committed manifest can reference a blob after enumeration but before unlink.
    Reproduction: compare the frozen baseline with the concurrent publisher test.
    Smallest improvement: require temporary publication plus the shared lock and
    recoverable quarantine.
  2. Prototype same-name publication defect, fixed — critical severity, high
    confidence.
    Two cooperative publishers could both report success and one could
    replace the other's append-only manifest. Reproduction: run the forced-contention
    acceptance case. Smallest improvement: test the final name while holding the lock,
    reject an existing destination, and remove the losing temporary file.
  3. Malformed-input safeguard — critical severity, high confidence. Truncated or
    malformed committed manifests abort plan and apply before quarantine and identify
    the file and record. Reproduction: run the acceptance harness. Smallest
    improvement: preserve this fail-closed diagnostic in daemon integration.
  4. Process-exit recovery — high severity, high confidence for tested boundaries.
    Resume preserved referenced blobs and completed after every injected ordinary
    exit. Reproduction: run the crash matrix. Smallest improvement: qualify the real
    filesystem and power-loss behavior before durability claims.
  5. Memory evidence scope — medium severity, high confidence. The external sorter
    met the requested fixture limits, but manifest paths, chunk readers, and merge
    state grow with the corpus; this is not an input-independent bound. Reproduction:
    run the measured scale plan. Smallest improvement: state the measured workload and
    extend the corpus shape before production sizing.
  6. Poor product fit — medium severity, high confidence. Fold does not own the
    external publication/quarantine protocol. Reproduction: compare the public Fold
    model with the runnable direct solution. Smallest improvement: document the
    ownership boundary; do not infer a garbage-collection subsystem.

Decision audit

The developer selected fixed binary hashes, external chunk sorting, a streaming
merge, two-step quarantine/finalization, advisory publication locking, strict final
newline validation, and full manifest revalidation under the lock. SQLite, Fold,
probabilistic filters, direct unlink, and grace-period-only safety were rejected.

Every real publisher must cooperate with the advisory lock. Same-filesystem rename,
directory-sync support, lowercase blob filenames on a case-sensitive filesystem,
publisher crash points, concurrent planners with one name, plan-file corruption,
interrupted syscalls, kernel failure, and power loss remain unresolved.

Skeptical review

  • Claims reproduced:
    • Both targeted formatting checks, all 6 Rust tests, both strict lints, both
      release builds, both small demonstrations, carrier crash recovery, the full
      carrier fixture, the full backup scale, and the backup acceptance matrix.
    • Both no-fit conclusions against the current public BogKit contracts.
  • Claims rejected or softened:
    • Carrier recovery before the incomplete-tail fix; production PostgreSQL/network,
      concurrent-worker, callback-conflict, and power-loss implications; and a false
      zero-memory result when process inspection was unavailable.
    • Backup cooperative-publication safety before the same-name fix; a stable Python
      memory-limit breach; input-independent bounded-memory, immutable-plan, and broad
      filesystem durability wording.
  • Quality fixes required and completed:
    • Carrier incomplete-tail truncation/sync, a second-reopen regression, and a
      measurement failure when no RSS sample exists.
    • Backup final-name checking under the publication lock, losing-temp cleanup, and a
      forced two-publisher contention regression.
  • Remaining uncertainty:
    • Real PostgreSQL/carrier integration, concurrent workflow workers, callback trust
      and conflicts, hardware durability, and production distributions.
    • Universal publisher cooperation, filesystem semantics, power failure, publisher
      crash points, concurrent same-name planners, plan integrity, and corpus scaling.

No BogKit correctness defect was demonstrated. The serious findings were defects in
today's prototypes and were fixed before archival.

Cross-run synthesis

  • New evidence:
    • Carrier/PostgreSQL atomicity and external manifest publication/quarantine are
      application authority protocols, not capabilities supplied by BogKit.
    • Conservative unknown/review handling and recoverable quarantine remain
      domain-specific evidence, not proposed core subsystems.
  • Recurring evidence:
    • The public capability and operational-boundary matrix is now supported by twelve
      independent trials.
    • The component-selective project path remains supported by four trials.
    • Nameable pipeline and reader patterns remain supported by three trials.
  • Candidate improvements:
    1. Public capability matrix covering component purpose, authoritative storage,
      transaction and process scope, external authorities, cooperative filesystem
      protocols, deployment, operating-system requirements, and hard limits.
    2. Component-selective project path that avoids unrelated setup.
    3. Documentation for nameable composed pipeline and reader types.
  • Observations not yet promoted:
    • Workflow ambiguity patterns, publisher/quarantine protocols, external sorting,
      and journal-tail recovery are one-trial or lab-specific observations.
  • No-fit or positioning signals:
    • Embedded incremental data tools do not replace correctness protocols spanning an
      external payment authority and PostgreSQL, or an externally published filesystem
      corpus. Onboarding should make those ownership boundaries quickly visible.

No new candidate was promoted. Today's two trials corroborate the existing public
capability/operational-boundary matrix but do not meet a threshold for a new BogKit
API or subsystem.

Validation

  • Trial-specific tests: 3 carrier and 3 snapshot tests passed, including both
    skeptical-review regressions.
  • Strict lint and formatting: both targeted formatting checks and strict lints
    passed; the complete nested workspace passed strict Clippy with warnings denied.
  • Runnable demonstrations: carrier small/crash/full-scale runs and backup
    acceptance/small/full-scale runs passed after review corrections.
  • Nested lab workspace tests: all 69 tests passed.
  • BogKit root workspace tests: all 45 unit and documentation tests passed.
  • Archive and secret checks: git diff --check, coverage JSON parsing, marker
    uniqueness, changed-path scope, generated-output, database, archive, model, binary,
    credential-pattern, symlink, and file-size scans passed. Every intended changed
    path is under developer-simulation/.

Full archived report: developer-simulation/reports/2026-08-02.md

urcades commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-03

Two blind trials tested BogKit against substantially different existing-software
problems: transitive software-provenance revocation and security-sensitive
offline door-policy updates. Both developers reached reviewed no-fit conclusions
after beginning with the public README and examples in separate sanitized copies
of current main.

Skeptical review preserved both decisions but rejected overclaims and found a
high-severity defect in the door-policy prototype: a contiguous next version
with the wrong base was mislabeled as a missing-version gap and produced an
impossible range. The coordinator separated those failure modes, added a
no-mutation/reopen regression, moved all generated state outside the archive,
integrated both crates with the nested workspace lock, and reran all evidence.
No BogKit correctness defect was demonstrated.

Trial 1 — Provenance revocation impact

  • Persona: release-security platform engineer with intermediate Rust and
    PostgreSQL/CI experience
  • Existing system: PostgreSQL stores artifacts, dependency manifests,
    attestations, releases, and cached promotion decisions; a nightly recursive
    recomputation determines impact
  • Problem: quickly and deterministically block every release transitively
    affected by revoked, missing, or cyclic provenance without introducing a
    partially published decision generation
  • Outcome: no fit for the PostgreSQL-authoritative promotion gate
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: Fold for the local input-fact persistence probe
  • Archive:
    runs/2026-08-03--provenance-revocation-impact

Evidence

  • Commands:
    • cargo test --manifest-path developer-simulation/Cargo.toml -p provenance-revocation-reproducer --offline --locked
    • cargo clippy --manifest-path developer-simulation/Cargo.toml -p provenance-revocation-reproducer --all-targets --offline --locked -- -D warnings
    • cargo build --manifest-path developer-simulation/Cargo.toml -p provenance-revocation-reproducer --release --offline --locked
    • provenance-revocation-reproducer generate | provenance-revocation-reproducer run <temp-state> candidate
    • provenance-revocation-reproducer generate | provenance-revocation-reproducer run <temp-state> reference
    • an abort-on-panic release build with PROVENANCE_CRASH=before_commit:1
      and PROVENANCE_CRASH=after_commit:1
  • Tests: 5 passed for transitive revocation, transitive missing manifests,
    cycles, complete acyclic input, order independence, and duplicate-edge set
    semantics.
  • Demonstration:
    • The intentionally incomplete one-hop negative control matched only 1 of 3
      decisions. It incorrectly approved a transitively revoked release and a
      release containing a reachable cycle; both engines blocked the missing root.
    • The reference produced stable tested paths
      app -> middle -> revoked-base and cycle-a -> cycle-b -> cycle-a.
    • Each tiny final Fold state occupied 48 KiB; this fixed overhead is not
      evidence for the requested storage-ratio target.
    • Abort before commit exited 134 and reopened without the artifact. Abort
      after commit exited 134 and reopened with the artifact.
  • Baseline comparison:
    • The real PostgreSQL baseline was evaluated conceptually, not reproduced.
      Its recursive query can compute exact reachability from a transactionally
      consistent authoritative snapshot, but nightly publication is too slow.
    • The runnable slow reference loads Fold facts into ordinary Rust collections
      and is only a bounded correctness oracle, not a PostgreSQL or production
      performance baseline.
    • The current public Fold operators do not supply joins, recursion, a feedback
      edge, fixed-point iteration, or graph reachability. A custom Push node
      could be written, but it would implement the graph engine and still leave a
      PostgreSQL-to-local-store reconciliation protocol.

Friction trail

The developer read the root README and all four public examples. Fold's atomic
transactions, retractions, and snapshots looked useful, but the examples showed
one delta moving through static branches rather than correlation of changing
relations or recursive reachability. Public API inspection confirmed the absence
of a supported join or fixed-point abstraction. A custom low-level operator was
technically possible but would consume the prototype boundary while leaving the
authoritative database handoff unsolved.

The first standalone Cargo attempt tried to refresh the registry; cached offline
resolution succeeded. Skeptical review later integrated the archive with the
existing nested lock, reproduced all results on Fjall/lsm-tree 3.1.6, renamed
the one-hop engine as a negative control, and required process-abort evidence so
the report would not overstate ordinary Rust-unwind recovery.

Findings

  1. No public join or recursive reachability — missing capability, blocker
    severity, high confidence.
    Reproduction: inspect the public operator list
    and examples. Smallest improvement: document the boundary in the public
    capability matrix. A recursive engine remains a one-trial observation, not a
    threshold-qualified subsystem candidate.
  2. PostgreSQL/local authority split — poor product fit, blocker severity, high
    confidence.
    Reproduction: compare the stated PostgreSQL authority with
    Fold's local embedded store and process-owned stream. Smallest improvement:
    keep computation/publication in the authoritative database unless a complete
    source-offset and generation-publication protocol is assigned.
  3. One-hop negative control approves unsafe releases — prototype correctness
    defect, blocker severity, high confidence.
    Reproduction: run both engines
    over the three-query corpus. Smallest improvement: never use the negative
    control as a gate; it is not an implemented Fold pipeline or a BogKit defect.
  4. Scale and witness scope — performance/evidence limitation, major severity,
    high confidence.
    The requested 500,000-artifact/5-million-edge fixture,
    100-updates-per-second concurrency, memory bound, and storage ratio were not
    measured after correctness no-fit was established. Sorted traversal gives a
    stable first witness on tested fixtures, not a globally minimal proof.

Decision audit

The developer rejected nightly-only recomputation, a Fold side cache, a one-hop
gate, ESE, ANNy, and a bespoke recursive Push node. The reviewed direction is
an incremental reverse-dependency generation published atomically in PostgreSQL,
or another proven recursive graph system that can preserve the named authority.
The no-fit decision does not claim that arbitrary custom Rust inside Fold could
never meet the numeric targets.

Trial 2 — Offline door policy update

  • Persona: building-access systems developer, Rust beginner with production
    embedded-C experience
  • Existing system: PostgreSQL compiles a nightly sorted policy file for 2,000
    intermittently connected controllers
  • Problem: safely apply frequent revocations and temporary grants under
    duplicate, reordered, skipped, truncated, and power-interrupted delivery while
    keeping a complete prior policy active until the new generation is verified
  • Outcome: no fit for the fixed-image controller safety boundary
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: Fold for transactional keyed-state fit evidence
  • Archive:
    runs/2026-08-03--offline-door-policy-update

Evidence

  • Commands:
    • cargo test --manifest-path developer-simulation/Cargo.toml -p offline-door-policy-fit-probe --offline --locked
    • cargo clippy --manifest-path developer-simulation/Cargo.toml -p offline-door-policy-fit-probe --all-targets --offline --locked -- -D warnings
    • cargo build --manifest-path developer-simulation/Cargo.toml -p offline-door-policy-fit-probe --release --offline --locked
    • three release demonstrations using separate empty /private/tmp state roots
  • Tests: 2 passed, including the reviewer-required wrong-base/no-mutation/reopen
    regression.
  • Demonstration:
    • Zero mismatches across 20,000 generated authorization queries before and
      20,000 after 50,000 revocations.
    • The final nested-workspace runs applied 60,000 initial grants in 49-63 ms,
      applied 50,000 revocations in 48-51 ms, and checkpointed in 4-5 ms.
    • A wrong-base version 2 returned BaseMismatch, left coherent version-1
      status, did not revoke the tested grant, and preserved it across reopen.
    • The same active version was labeled SameVersionIgnoredUnverified; payload
      identity and authenticity were not claimed.
    • Old and skipped versions were rejected, contiguous repair succeeded, and
      final active version 4 survived clean checkpoint/reopen.
    • The open Fold directory had 14 files with a 67,115,332-byte sparse logical
      extent and 3,129,344 allocated bytes. Clean close left 11 files with
      3,072,778 logical and 3,104,768 allocated bytes.
    • Whole-process host RSS was 25,526,272-26,820,608 bytes. It includes the
      reference map, generated bundles, runtime, code, and database mappings, so
      the 4 MiB controller-memory bound remains not demonstrated.
  • Baseline comparison:
    • The naive in-place file baseline mixes old and new bytes after the first
      modeled 4 KiB write.
    • Fold correctly handled transactional grant retractions and version metadata
      on the host, but uses a filesystem directory rather than the fixed 16 MiB
      flash image and does not expose each physical 4 KiB write for exhaustive
      power-cut injection.
    • Signed framing, length/truncation checks, bundle identity, and key management
      are application-specific work outside the probe.

Friction trail

The developer selected Fold after the starter and time-series examples showed
atomic keyed retractions and reopen. A pre-created regular 16 MiB file could not
serve as the database path; the public constructor documents that store-open
failure may panic. Host results were fast, but neither sparse filesystem layout
nor whole-process RSS established the fixed flash or memory contract.

Skeptical review reproduced the full evidence on the final nested dependency
set, found the wrong-base classification defect, showed that rejected-gap status
is process-local, and showed that same-version content was ignored without
identity verification. The coordinator fixed the consequential wrong-base bug,
narrowed both softer claims, moved runtime/test state to caller-supplied or
unique temporary paths, and reran the suite and demonstration three times.

Findings

  1. Wrong-base classification, fixed — prototype correctness defect, high
    severity, high confidence.
    Original reproduction: after active version 1,
    version 2 based on version 0 returned Missing { expected: 2, received: 2 }
    and missing=2..1. Smallest improvement completed: return BaseMismatch,
    make no policy mutation, keep coherent status, and cover reopen.
  2. Fixed/raw-flash mismatch — poor product fit, critical severity, high
    confidence.
    Reproduction: the regular-file-path probe and absence of a raw
    block interface. Smallest improvement: retain fixed/raw flash as a documented
    non-goal and use a purpose-built controller format; do not infer a new engine.
  3. No controllable physical write-cut boundary — missing capability, critical
    severity, high confidence.
    Reproduction: Fold exposes logical transactions
    and checkpoint, not every 4 KiB physical write. Smallest improvement: build
    the controller-specific dual-generation recovery harness outside BogKit and
    keep the boundary visible in onboarding.
  4. Working-memory target not demonstrated — performance limitation, high
    severity, medium confidence.
    Reproduction: three whole-process RSS runs.
    Smallest improvement: measure a real controller implementation with allocator
    and storage telemetry; do not attribute the harness RSS to Fold.
  5. Bundle trust and status scope — missing application capability, high
    severity, high confidence.
    Signed/truncated intake, same-version payload
    identity, and persistent rejected-gap diagnostics were not implemented.
    Smallest improvement: define them in the application wire/recovery protocol;
    this does not qualify as a BogKit subsystem candidate.
  6. Durability boundary can be overread — documentation gap, medium severity,
    high confidence.
    Smallest improvement: distinguish process commit,
    checkpoint, filesystem behavior, fixed capacity, and raw-flash power cuts in
    the public capability matrix.

Decision audit

The developer rejected in-place file replacement, Fold on the controller, ESE,
ANNy, and Fold only in the central compiler. A purpose-built fixed-image format
with an inactive staging region, redundant activation record, streaming signed
verification, monotonic version rules, and torn-write recovery owns the actual
safety boundary. PostgreSQL remains the authoritative compiler input.

Skeptical review

  • Claims reproduced:
    • Both final formatting checks, all 7 new tests, strict lints, release builds,
      both provenance decisions, both abort boundaries, and three door runs.
    • The absence of public join/recursive reachability, the local transaction
      boundary, filesystem-directory storage, and both no-fit conclusions.
  • Claims rejected or softened:
    • The one-hop negative control as a real Fold composition or BogKit defect;
      the slow reference as a reproduced PostgreSQL baseline; globally minimal
      witnesses; broad crash/power-loss implications; and scale claims.
    • Door duplicate identity, persisted rejected-gap status, a Fold-specific
      memory failure, general corruption behavior, hardware timing, and a signed
      envelope as a core candidate.
  • Quality fixes required and completed:
    • Distinct wrong-base handling with non-mutation/reopen regression.
    • Caller-supplied/temporary state, nested-lock integration, removal of package
      locks/generated databases/build output, archive READMEs, and all final reruns.
  • Remaining uncertainty:
    • Production PostgreSQL ingestion/publication, correct incremental graph scale,
      global witness policy, concurrency, memory, and storage amplification.
    • Real signed bundles, controller hardware, fixed-image layout, persisted
      diagnostics, wear, erase geometry, physical write atomicity, and power loss.

No BogKit correctness defect was demonstrated. The serious defect was in the
door-policy prototype and was fixed before archival.

Cross-run synthesis

  • New evidence:
    • Exact recursive graph processing and fixed raw-flash policy activation sit
      outside the current public BogKit abstractions for these two workloads.
    • Provenance joins/recursion and signed door bundles remain one-trial
      observations; neither qualifies as a new core API or subsystem candidate.
  • Recurring evidence:
    • The public capability and operational-boundary matrix is now supported by
      fourteen independent trials.
    • The door trial independently repeats the July 29 raw-NOR/fixed-flash no-fit
      boundary.
    • The component-selective project path remains supported by four trials.
    • Nameable pipeline and reader patterns remain supported by three trials.
  • Candidate improvements:
    1. Public capability matrix covering component purpose, authoritative storage,
      transaction/process scope, joins/recursion, external authorities,
      filesystem/raw-flash boundaries, deployment, OS requirements, and hard
      limits.
    2. Component-selective project path that avoids unrelated setup.
    3. Documentation for nameable composed pipeline and reader types.
  • Observations not yet promoted:
    • Durable recursive provenance, canonical graph witnesses, signed delta
      delivery, bundle identity, and controller activation formats.
  • No-fit or positioning signals:
    • Embedded incremental state is not a substitute for an authority-safe
      PostgreSQL graph publication protocol or a fixed-capacity raw-flash recovery
      format with explicit physical fault injection.

No new candidate was promoted.

Validation

  • Trial-specific tests: all 7 passed, including the skeptical-review regression.
  • Strict lint and formatting: every nested package passed its formatting check;
    both new packages and the complete nested workspace passed strict Clippy with
    warnings denied.
  • Runnable demonstrations: the provenance negative control/reference, both
    process-abort boundaries, and all three corrected door runs passed with the
    results above.
  • Nested lab workspace tests: all 76 tests passed.
  • BogKit root workspace tests: all 45 unit and documentation tests passed after
    the sandboxed first attempt reached ESE's required model download and the
    unchanged suite was rerun with network access.
  • Archive and secret checks: git diff --check, coverage JSON parsing, marker
    uniqueness, changed-path scope, generated-output, database, archive, model,
    binary, credential-pattern, symlink, and file-size scans passed. Every intended
    changed path is under developer-simulation/.

Full archived report: developer-simulation/reports/2026-08-03.md

urcades commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-04

Two blind trials tested BogKit against substantially different existing-software
problems: mixed-version event-contract safety during rolling deployments and
security-sensitive OCR redaction remapping after text correction. Both fresh
developers began with the public README and examples in separate sanitized
copies of current main, built runnable standalone prototypes, and reached
reviewed no-fit conclusions.

Skeptical review preserved both fit decisions but rejected initial quality
claims and found four archive blockers. Duplicate raw JSON member names could
false-allow a contract. The redaction mapper could cover the wrong repeated
occurrence, trust stale completed files, and reconnect a stale checkpoint to
truncated partial output. The coordinator fixed all four classes with exact
regressions and reran the affected demos, strict checks, and full workloads.
None was a BogKit correctness defect.

Trial 1 — Mixed-version contract gate

  • Persona: deployment-platform developer with intermediate Rust and production
    Go experience
  • Existing system: repository-owned versioned JSON contracts and topology, with
    a CI script that compares only adjacent versions
  • Problem: cover every supported producer-consumer version pair during rolling
    deployment and return deterministic minimal counterexamples
  • Outcome: no fit for the bounded read-only compatibility gate
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-04--mixed-version-contract-gate

Evidence

  • Commands:
    • cargo test --offline --locked --manifest-path developer-simulation/Cargo.toml -p mixed-version-contract-gate
    • cargo clippy --offline --locked --manifest-path developer-simulation/Cargo.toml -p mixed-version-contract-gate --all-targets -- -D warnings
    • cargo build --release --offline --locked --manifest-path developer-simulation/Cargo.toml -p mixed-version-contract-gate --bins
    • the archived generator and gate against external demo and workload
      directories under /private/tmp
    • python3 .../oracle.py .../fixtures/semantic_cases.json
  • Tests: 8 passed after the reviewer-required duplicate-member regression.
  • Demonstration:
    • The independent Python oracle matched all 64 fixed semantic cases.
    • The demo evaluated all 9 mixed-version pairs and blocked exactly the 3
      producer versions incompatible with the narrowed consumer contract.
    • Raw duplicate members at the root, contract, schema, topology, fleet, and
      candidate layers now return review-required; the reviewer's exact
      {"type":"string","type":"integer"} false allow is fixed.
    • Skeptical review reversed and identically duplicated the full input and
      obtained byte-identical output. A reverse relationship produced 18 bounded
      pairs and the same 3 issues.
    • The corrected full fixture contained 300 services, 120 topics, 1,800
      contracts, 12,000 relationships, 25 candidates, and 108,000 evaluated
      version pairs. The final nested-workspace run returned 237 issues and no
      review issues in 0.52 seconds, with 119,144,448 bytes maximum RSS. Output
      SHA-256 remained
      d5cbec30584c237b56d03a507292ffb4772234c34be1f47a5ebbb3436e4fd5d7.
  • Baseline comparison:
    • The stated existing gate was modeled but not preserved as a runnable
      baseline, so no measured speed or correctness improvement over it is
      claimed.
    • The application-specific contract subset and default policy are not full
      JSON Schema, Avro, or Protobuf semantics.
    • Fold persistence would not remove strict parsing, recursive inclusion,
      witness construction, or diagnostics from this bounded immutable pass.

Friction trail

The developer read the public README and starter, timeseries, chat, and search
examples before inspecting manifests or component source. A clean-target
cargo run -p starter --offline reached ESE's model download even though the
starter source uses only Fold. The developer then confirmed that both the
starter manifest and project scaffold include all three components.

After reading Fold's public stream and pipeline interfaces, the developer kept
the gate standalone with serde and serde_json. Skeptical review reproduced
the claimed workload but defeated serde_json::Value's last-member-wins parsing
with a raw duplicate type key. The coordinator added a duplicate-rejecting
recursive JSON visitor and reran the complete evidence set.

Findings

  1. Duplicate-member false allow, fixed — prototype correctness defect, high
    severity, high confidence.
    Reproduction: the reviewer's conflicting raw
    type keys. Smallest improvement completed: reject duplicate object members
    before conversion to Value, at every nesting level.
  2. Unconditional starter/scaffold components — API friction and documentation
    gap, moderate severity, high confidence.
    Reproduction: the clean-target
    starter run plus current manifests. Smallest improvement: remove unused
    starter dependencies and make generated components opt-in; disclose ESE's
    model artifact when selected.
  3. Bounded batch analysis is poor product fit — informational severity, high
    confidence.
    Reproduction: compare the four immutable inputs and pure output
    with Fold's durable runtime delta/materialization model. Smallest improvement:
    document when plain in-memory analysis is preferable; no new core API is
    justified.
  4. Evidence scope — moderate severity, high confidence. The fixed cases are
    broad but not a formal inclusion proof; timing and memory are one generated
    workload on one host.

Decision audit

The developer chose a standalone strict parser, a documented small schema
language, receiver-materialized defaults, one smallest witness per version pair,
ordered semantic identities, fail-review behavior for unsupported input, and an
in-memory touched-pair cache. Fold, ESE, ANNy, durable cross-job caching, full
standards support, deployment orchestration, and application-semantic proofs
were rejected. Formal exhaustiveness, arbitrary deeply nested witness
minimality, and production contract semantics remain unresolved.

Trial 2 — OCR redaction remapping

  • Persona: public-records processing engineer, Rust beginner with production
    Python experience
  • Existing system: human-reviewed UTF-8 offsets are clamped into OCR text after
    Unicode cleanup and glyph regeneration
  • Problem: remap reviewed spans without exposing corrected sensitive text,
    silently guessing ambiguity, or publishing incomplete geometry
  • Outcome: no fit for the independent page-local transformation
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-04--ocr-redaction-remap

Evidence

  • Commands:
    • cargo test --offline --locked --manifest-path developer-simulation/Cargo.toml -p ocr-redaction-remap
    • cargo clippy --offline --locked --manifest-path developer-simulation/Cargo.toml -p ocr-redaction-remap --all-targets -- -D warnings
    • developer-simulation/runs/2026-08-04--ocr-redaction-remap/run_demo.sh
    • a release build plus the exact external 5,000-page workload under
      /private/tmp
  • Tests: 7 passed, including the reviewer's repeated-occurrence, changed-input,
    truncated-output, changed-audit, stale-checkpoint, and short-partial
    regressions.
  • Demonstration:
    • The final fixture run checked 244 pages: 243 exact, 1 conservative, 0
      blocked, 0 expected-rectangle mismatches, and no complete sentinel strings
      in output or audit.
    • Reversed and duplicated spans remained byte-identical. Controlled stops at
      pages 73 and 191 resumed to the uninterrupted bytes.
    • Invalid offsets, UTF-8 boundaries, missing geometry, and contradictory
      geometry blocked with zero rectangles; invalid raw UTF-8 produced only a
      static error code and line number.
    • The reviewer's changed-first repeated secret is now conservatively covered
      at the source-mapped region instead of being mislabeled exact at the
      unchanged second occurrence.
    • Completed resume now verifies exact paths, lengths, and SHA-256 values for
      input, output, and audit. Changed input, truncated output, and changed audit
      fail. Fresh runs invalidate stale state; short or changed partial prefixes
      fail instead of being extended.
    • The corrected nested-workspace 5,000-page, 20-million-ASCII-scalar,
      150,000-span workload completed in 38.82 seconds with 5,947,392 bytes
      maximum RSS. It emitted
      5,000 output lines, 5,000 audit lines, and 600,000 exact rectangles. A
      second full output and audit were byte-identical.
  • Baseline comparison:
    • baseline_clamp.py deterministically shows partial exposure for the stated
      stale-offset algorithm without printing the modeled secret. This is
      conditional evidence about that algorithm, not proof of the production
      implementation.
    • Fold owns durable incremental views, while ESE and ANNy provide semantic
      embeddings and approximate retrieval. None supplies authoritative Unicode
      correspondence or glyph-redaction geometry.

Friction trail

The developer followed the same public onboarding order independently and hit
the same unused ESE model-download coupling before inspecting manifests. After
freezing the clamping reproducer, the developer selected no BogKit component and
built a page-streaming Rust transformer with bounded edit alignment, complete
geometry validation, conservative fallbacks, content-free records, and
controlled process checkpoints.

Skeptical review reproduced the benign fixtures but constructed a changed first
occurrence followed by an unchanged duplicate. The mapper redacted only the
second occurrence. Review also changed completed input, truncated final output,
and reconnected a stale checkpoint to short partial files. The coordinator fixed
all three classes and reran every affected check and scale case.

Findings

  1. Stale-offset clamp exposure — modeled baseline correctness defect,
    critical severity, high confidence for the reproducer.
    Smallest improvement:
    stop clamping revised offsets; validate correspondence and block unresolved
    mapping or geometry.
  2. Wrong repeated occurrence, fixed — prototype confidentiality defect,
    critical severity, high confidence.
    Smallest improvement completed: compare
    literal candidates with source-position edit mapping and conservatively cover
    disagreement.
  3. Completed and partial recovery integrity, fixed — prototype correctness
    defects, critical severity, high confidence.
    Smallest improvement completed:
    strong final bindings, fresh-state invalidation, and prefix validation that
    never extends a short file.
  4. Unconditional starter/scaffold components — API friction and documentation
    gap, moderate severity, high confidence.
    This independently repeats Trial
    1's setup evidence.
  5. Page-local remapping is poor product fit — informational severity, high
    confidence.
    No current public component removes alignment, ambiguity, or
    geometry work; approximate matching would be inappropriate.
  6. Evidence scope — moderate severity, high confidence. Sentinel checks are
    not a general leakage proof, the workload is ASCII, metadata must be declared
    non-sensitive by the caller, and recovery covers controlled process restart,
    not filesystem or power-loss durability.

Decision audit

The developer chose validated UTF-8 byte spans, Unicode grapheme rectangles,
bounded normalization and edit mapping, conservative token/line coverage,
complete one-to-one geometry, sorted duplicate handling, allow-listed identifier
metadata, paired external outputs, prefix-validated checkpoints, and SHA-256-
bound completion markers. Fold, ESE, ANNy, approximate cross-page matching, PDF
processing, OCR, entity detection, bidirectional layout, and power-loss claims
were rejected. Production normalization rules, edits beyond 256, arbitrary
Unicode scale, metadata classification, and real filesystem failure remain
unresolved.

Skeptical review

  • Claims reproduced:
    • Both final no-fit decisions, starter/scaffold coupling, contract demo and
      full workload, redaction baseline reproducer, fixture demo, controlled
      resumes, malformed-input failures, and final scale run.
  • Claims rejected or softened:
    • Strict contract parsing before duplicate-key rejection; a measured
      existing-gate comparison; broad standards, Unicode, confidentiality,
      portability, memory, durability, crash, power-loss, or sentinel-leakage
      guarantees; and the original completed-resume claims.
  • Quality fixes required and completed:
    • Recursive duplicate-member rejection.
    • Source-correspondence validation for literal redaction matches.
    • SHA-256-bound completed state and stale/short partial rejection.
    • Nested-workspace integration, external generated state, archive READMEs,
      and final reruns.
  • Remaining uncertainty:
    • Formal contract inclusion and witness proofs, production schema semantics,
      multi-host performance, full Unicode production corpora, authoritative
      metadata classification, arbitrary layout, and storage/power failures.

No BogKit correctness defect was demonstrated. The serious findings belonged to
the modeled clamping baseline and the two prototypes and were fixed or narrowly
classified before archival.

Cross-run synthesis

  • New evidence:
    • Contract-language inclusion and reviewed Unicode/glyph correspondence are
      application-specific subsystems outside the current public BogKit roles.
    • Both trials independently encountered unconditional all-component starter
      and scaffold dependencies.
  • Recurring evidence:
    • The public capability and operational-boundary matrix is now supported by
      16 independent trials.
    • The component-selective project path is now supported by 6 trials.
    • Nameable pipeline and reader patterns remain supported by 3 trials.
  • Candidate improvements:
    1. Public capability and operational-boundary matrix.
    2. Component-selective project path, including a true Fold-only starter.
    3. Documentation for nameable composed pipeline and reader types.
  • Observations not yet promoted:
    • Contract-language tooling, Unicode/glyph redaction, final-output binding,
      and conservative correspondence APIs.
  • No-fit or positioning signals:
    • BogKit's persistent local data semantics, embeddings, and approximate
      retrieval should not replace bounded standards analysis or exact
      security-sensitive document transformation when those components do not own
      the authority or algorithm.

No new candidate was promoted.

Validation

  • Trial-specific tests: all 15 corrected tests passed.
  • Strict lint and formatting: both new packages and the complete nested
    workspace passed formatting and Clippy with warnings denied.
  • Runnable demonstrations: contract demo/oracle/full workload and corrected
    redaction fixture/resume/full workload passed with the scoped results above.
  • Nested lab workspace tests: all 91 tests passed.
  • BogKit root workspace tests: all 45 unit and documentation tests passed.
  • Archive and secret checks: git diff --check, coverage JSON parsing, marker
    uniqueness, changed-path scope, generated-output, database, archive, model,
    binary, credential-pattern, symlink, and file-size scans passed. Every intended
    changed path is under developer-simulation/.

urcades commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-05

Two blind trials tested BogKit against unrelated existing-software problems:
streaming paired-FASTQ demultiplexing with bounded output handles, and exact
map-tile invalidation from authoritative parcel geometry deltas. Fresh
developers began from the public README and examples in separate sanitized
copies of current main. Both built runnable standalone Rust prototypes and
reached reviewed no-fit conclusions.

Skeptical review preserved both fit decisions but found correctness and evidence
blockers in each prototype. The coordinator fixed case-insensitive output-name
collisions, incomplete FASTQ identifier validation, a false-passing file-handle
observer, duplicate JSON member acceptance, and invalid polygon topology
acceptance. All affected tests, demonstrations, observers, and scale workloads
were rerun. None of these findings was a BogKit correctness defect.

Trial 1 — Bounded FASTQ barcode spill

  • Persona: sequencing-pipeline engineer with intermediate Rust and production
    Python experience
  • Existing system: a Python demultiplexer that routes paired reads by exact
    barcode and keeps one output handle open per sample
  • Problem: stream 1,000,000 paired records to 384 sample destinations, apply
    unique Hamming-distance-one correction, preserve ambiguous ties, and keep no
    more than 24 sample output files open
  • Outcome: no fit for this stateless, one-pass byte router
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-05--fastq-barcode-spill

Evidence

  • Commands:
    • cargo test --offline --locked --manifest-path developer-simulation/Cargo.toml -p fastq-barcode-spill
    • cargo clippy --offline --locked --manifest-path developer-simulation/Cargo.toml -p fastq-barcode-spill --all-targets -- -D warnings
    • developer-simulation/runs/2026-08-05--fastq-barcode-spill/scripts/verify.sh
    • the archived measure_open_files.py observer against the release binary
    • two release-mode 1,000,000-pair runs from a non-seekable generated pipe
  • Tests: 5 Rust tests and the corrected Python integration suite passed.
  • Demonstration:
    • Clean exact-match outputs were byte-identical to the trial-created Python
      baseline for every sample and unmatched file.
    • Seeded unique one-base correction, unmatched routing, barcode-level ties
      between two whitelist entries for the same sample, and ties across samples
      all produced the expected classifications.
    • Truncation, unequal sequence and quality lengths, mismatched pair IDs,
      invalid identifiers, and colliding output aliases failed without a
      completion manifest or sensitive read/sample content in diagnostics.
    • An independent lsof observer measured 24 open FASTQ outputs across 45
      successful polls; a forced observer failure now fails the check.
    • The final mixed run processed exactly 1,000,000 pairs: 250,000 exact,
      250,000 corrected, 250,000 ambiguous, and 250,000 unmatched. It completed
      in 1.93 seconds, used 63,750,144 bytes maximum RSS, and reported 24 maximum
      open writers.
    • Two complete 387-file output trees produced the same SHA-256:
      88e0f8be6e3f5622b19d4cf530c1af20a2f69ce8ece18c3bf347a466e22cd644.
  • Baseline comparison:
    • The repository contained no production FASTQ implementation, so the Python
      baseline is a minimal trial-created model of the stated exact-match and
      all-handles-open behavior, not evidence about an unavailable production
      program.
    • Its 384-output failure under a 64-file process limit confirms the modeled
      design pressure only. No production speedup claim is made.
    • Fold maintains durable views; ESE and ANNy provide embeddings and
      approximate retrieval. None removes the required parsing, exact barcode
      classification, byte preservation, or file-descriptor management.

Friction trail

The developer read the README, starter, timeseries, chat, and search examples,
then inspected the project scaffold. Because the scaffold creates a public
example with all three components, the developer made a standalone trial and
modeled the unavailable Python baseline first. The first clean fixture contained
an extra blank line, the first parser used record-relative error lines, and a
naive correction path scanned every barcode. Those trial defects were fixed
before final evidence.

Round-robin samples caused excessive output reopening even with a bounded LRU,
so the developer added per-destination buffers while preserving read order. A
first piped benchmark raced barcode-map creation, and sandboxed macOS resource
reporting could not read the RSS counter; both measurements were corrected and
rerun. Skeptical review then reproduced the headline workload but showed that
Alpha/alpha and reserved aliases could share a filename, malformed read IDs
were accepted, and the external observer could pass without a valid sample. The
coordinator added preflight name checks, explicit supported-ID validation, and a
positive-observation requirement, then reran the suite, observer, and two full
workloads.

Findings

  1. Case-folded output collision, fixed — correctness defect, high severity,
    high confidence.
    Reproduction: use sample aliases differing only by ASCII
    case or a reserved Ambiguous/Unmatched alias. Smallest improvement
    completed: reject all case-folded filename collisions before output creation.
  2. Incomplete identifier admission, fixed — correctness defect, moderate
    severity, high confidence.
    Reproduction: empty/control identifiers or
    contradictory slash and CASAVA mate roles. Smallest improvement completed:
    validate the documented identifier subset without echoing rejected data.
  3. False-passing descriptor observer, fixed — evidence defect, high severity,
    high confidence.
    Reproduction: force the observer command to fail for every
    poll. Smallest improvement completed: require successful positive polls and
    retain the real 45-poll result.
  4. All-component scaffold — API friction and documentation gap, moderate
    severity, high confidence.
    Reproduction: inspect the current project
    generator. Smallest improvement: make components opt-in.
  5. Stateless byte routing is poor product fit — informational severity, high
    confidence.
    Smallest improvement: state this boundary in the public
    capability matrix; no new component is justified.
  6. Evidence scope — moderate severity, high confidence. Resource figures are
    one generated workload on one host, the parser intentionally supports a
    bounded identifier grammar, individual FASTQ lines are not length-capped,
    and manifest creation does not claim filesystem power-loss durability.

Decision audit

The developer chose a streaming paired-record reader, byte-preserving output,
fixed-length exact and precomputed neighbor maps, barcode-level ambiguity,
per-destination buffering, a 24-entry writer LRU, static privacy-safe errors,
and an atomically renamed completion manifest. Fold, ESE, ANNy, one writer per
sample, approximation, compression, parallel parsing, full FASTQ dialect
support, and power-loss guarantees were rejected. Maximum record length,
production barcode grammars, compressed input, and multi-host performance
remain unresolved.

Trial 2 — Parcel delta tile planner

  • Persona: civic GIS platform developer, Rust beginner with production
    TypeScript experience
  • Existing system: a stateless TypeScript job that scans map tiles touched by
    authoritative old and new parcel geometries
  • Problem: produce the exact deterministic z/x/y invalidation plan for
    Polygon and MultiPolygon deltas at zooms 12–16 without reading or writing a
    parcel store
  • Outcome: no fit for this bounded, plan-only geometry transformation
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-05--parcel-delta-tiles

Evidence

  • Commands:
    • cargo test --offline --locked --manifest-path developer-simulation/Cargo.toml -p parcel-delta-tiles
    • cargo clippy --offline --locked --manifest-path developer-simulation/Cargo.toml -p parcel-delta-tiles --all-targets -- -D warnings
    • developer-simulation/runs/2026-08-05--parcel-delta-tiles/scripts/verify.sh
    • the archived Rust CLI and TypeScript mirror on a generated 1,000-edit,
      200-distinct-vertex-per-line workload
  • Tests: 8 Rust tests passed after strict JSON and topology regressions.
  • Demonstration:
    • Rust matched the TypeScript full-scan mirror on insertion, deletion,
      translation, concavity, holes, MultiPolygon, edge/corner contact, and a
      seeded 10,000-edit case.
    • An independent analytical rectangle enumerator matched 500 edits and 34,650
      expected tiles without sharing the polygon predicate.
    • Ten permutations were byte-identical. Eleven malformed cases, including
      recursive duplicate members, self-intersection, zero area, outside holes,
      and overlapping holes, failed deterministically with empty stdout.
    • The final 7,977,879-byte workload mixed inserts, deletes, replacements,
      concave polygons, holes, MultiPolygons, four extents, and 1,000 lines with
      200 distinct vertices each. Rust and the mirror produced the same 2,922
      tile lines.
    • On that workload Rust completed in 0.14 seconds with 2,670,592 bytes maximum
      RSS. The trial-created TypeScript mirror took 11.99 seconds and 169,312,256
      bytes maximum RSS on the same host and input.
  • Baseline comparison:
    • No production TypeScript planner existed in the checkout. The archived
      TypeScript program is a trial-created correctness mirror with an independent
      full-tile enumeration but a shared intersection predicate, so it is not a
      fully independent oracle and does not substantiate the brief's reported
      production runtime.
    • Fold's durable incremental state is unnecessary because each line already
      carries authoritative old/new geometry and state access is prohibited. ESE
      and ANNy do not solve exact polygon/tile intersection.

Friction trail

The developer followed the public onboarding order, found no parcel baseline,
and avoided the public-example scaffold. Restricted network access made the
first ordinary Cargo run retry the registry, while the cached locked build
worked offline. The initial planner was made explicitly lexicographic and its
fixtures were broadened from rectangles to concavity, holes, MultiPolygons, and
boundary-only contact.

Skeptical review then defeated ordinary serde_json::Value parsing with
duplicate members and showed that bowties, zero-area rings, outside holes, and
overlapping holes were accepted. It also rejected calling the TypeScript mirror
an independent oracle and found the first scale fixture too repetitive. The
coordinator added recursive duplicate rejection and simple-topology admission,
retained the analytical rectangle check as independent evidence, narrowed the
mirror claim, diversified the scale fixture, and reran every check and both
implementations.

Findings

  1. Duplicate JSON members, fixed — correctness defect, high severity, high
    confidence.
    Reproduction: duplicate old, new, or nested geometry keys.
    Smallest improvement completed: reject duplicates recursively before value
    conversion and emit no partial plan.
  2. Invalid polygon topology, fixed — correctness defect, high severity, high
    confidence.
    Reproduction: self-intersecting or zero-area rings, outside or
    overlapping holes, or overlapping MultiPolygon exteriors. Smallest
    improvement completed: strict simple-topology validation before planning.
  3. Reference independence overstated, fixed — evidence defect, moderate
    severity, high confidence.
    Smallest improvement completed: label the
    TypeScript program a mirror and separate the analytical rectangle oracle.
  4. All-component scaffold — API friction and documentation gap, moderate
    severity, high confidence.
    This independently repeats Trial 1.
  5. Authoritative stateless delta planning is poor product fit — informational
    severity, high confidence.
    Smallest improvement: document the boundary;
    adding durable state would make this command less faithful to its contract.
  6. Evidence scope — moderate severity, high confidence. The implementation
    handles strict simple planar GeoJSON polygons on synthetic data, not general
    validity, antimeridian wrapping, geodesic edges, or production GIS corpora;
    performance is one host and fixture.

Decision audit

The developer chose strict line-at-a-time JSON admission, simple-topology
validation, geometry-bounded candidate enumeration, closed-boundary contact,
deduplication, and lexicographic rendering. Fold, ESE, ANNy, parcel persistence,
rendering/publishing, deletion, spatial indexes, general polygon repair, and
silent malformed-input recovery were rejected. Antimeridian semantics,
geodesics, arbitrary GeoJSON, production topology policy, and multi-host scale
remain unresolved.

Skeptical review

  • Claims reproduced:
    • Both no-fit decisions; README/example/scaffold discovery; clean FASTQ
      comparison; correction/tie/unmatched routing; million-pair counts and
      checksum; descriptor bound; named GIS cases; analytical rectangles;
      malformed behavior; deterministic permutations; and scale outputs.
  • Claims rejected or softened:
    • Production-baseline comparisons, broad FASTQ parsing and memory claims,
      descriptor evidence before positive-poll enforcement, an independent GIS
      oracle, general GeoJSON validity, and multi-host performance.
  • Quality fixes required and completed:
    • Case-folded and reserved output-name rejection.
    • Supported FASTQ identifier validation and reliable external observation.
    • Recursive duplicate-member rejection and strict simple-topology admission.
    • More varied GIS scale data, nested-workspace integration, root-relative
      archive scripts, external generated state, and complete evidence reruns.
  • Remaining uncertainty:
    • Production inputs and baselines, adversarial record sizes, compressed
      sequencing data, storage failures, general GIS validity, antimeridian and
      geodesic semantics, and performance beyond the measured host.

No BogKit correctness defect was demonstrated. The blockers belonged to the two
trial prototypes and were fixed before archival.

Cross-run synthesis

  • New evidence:
    • Stateless, one-pass exact transformations remain outside the useful public
      roles of BogKit when the job has no durable read model, embedding, or
      approximate-retrieval need.
    • Strict input admission and an independent or analytically separable oracle
      are essential before claiming exactness for streaming or geometry tools.
  • Recurring evidence:
    • The public capability and operational-boundary matrix is now supported by
      18 independent trials.
    • The component-selective project path is now supported by 8 trials.
    • Nameable pipeline and reader patterns remain supported by 3 trials.
  • Candidate improvements:
    1. Public capability and operational-boundary matrix.
    2. Component-selective project path, including a true Fold-only starter.
    3. Documentation for nameable composed pipeline and reader types.
  • Observations not yet promoted:
    • Strict JSON-member admission, FASTQ parsing, output-spill management,
      geometry validity, and tile-intersection utilities.
  • No-fit or positioning signals:
    • Do not add durable state merely to adopt the toolkit when authoritative
      deltas are already present and the result is a bounded exact transformation.

No new candidate was promoted.

Validation

  • Trial-specific tests: all 13 corrected Rust tests and both archived integration
    verifiers passed.
  • Strict lint and formatting: both new packages passed formatting and Clippy
    with warnings denied; the complete nested workspace passed Clippy. The broad
    repository formatting check also reports an existing formatting difference in
    examples/search/src/main.rs, which is outside the automation archive boundary
    and was left unchanged.
  • Runnable demonstrations: corrected FASTQ clean/mixed/malformed/observer/full
    workloads and GIS named/analytical/malformed/permutation/full workloads passed
    with the scoped results above.
  • Nested lab workspace tests: all 104 tests passed.
  • BogKit root workspace tests: all 45 unit and documentation tests passed after
    the required ESE model artifact was fetched outside the restricted sandbox.
  • Archive and secret checks: git diff --check, coverage JSON parsing, marker
    uniqueness, changed-path scope, generated-output, database, archive, model,
    binary, credential-pattern, symlink, and file-size scans passed. Every intended
    changed path is under developer-simulation/.

urcades commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-06

Published commit: b2159b3

Two blind developers tested BogKit from fresh sanitized current-main copies, followed by a separate skeptical review. Both trials produced runnable prototypes; neither established a full production fit, and no BogKit core or existing-example defect was demonstrated.

Trial results

  1. CalDAV recurrence and time-zone correctness — no fit for the full SQLite-authoritative scenario. Fold was useful only for the event-master keyed upsert/remove boundary. The corrected standalone prototype has 10 tests, strict formatting/Clippy, a 5,000-master / 2,000,000-candidate release run in 3.85s at 17,547,264 bytes peak RSS, fingerprinted-shard reuse in 2.93s, and byte-identical UTC/Honolulu outputs. The assigned oracle, production SQLite service, and reference machine were absent, so those claims remain unverified.

  2. HTTP cache revalidation and tag-purge safety — no fit for the acceptance-critical cross-store cache boundary; no BogKit component was used. The corrected model has 12 tests, strict formatting/Clippy, passing demo and quota traces, privacy-safe hashed output, and a compact shape check of 2M objects / 1M requests / 100K purges at 65,634,304 bytes RSS. It does not claim real HTTP, body-file, SQLite, fsync, restart, lease-expiry, distributed-worker, or semantic-scale behavior.

Review and synthesis

The reviewer’s prototype blockers were fixed before archival: calendar all-day intersection, canonical/unseen overrides, shard fingerprint verification, expansion preflight, and help/evidence corrections; cache purge reordering and strict trace-parser admission. The capability/operational-boundary finding now has 20 independent trials; component-selective scaffolding has 10. No new candidate was promoted.

Validation

  • Both archived trial suites: 10 and 12 tests passed.
  • Full nested simulation workspace: strict Clippy and all tests passed.
  • BogKit root workspace: all unit and documentation tests passed after the required ESE model download in the network-enabled retry.
  • Archive, secret, symlink, size, generated-output, and scope checks passed; only developer-simulation/ changed.

Full report: developer-simulation/reports/2026-08-06.md

urcades commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

BogKit simulated-developer report — 2026-08-07

Commit: 4ba1559 · full archived report

Two blind developers worked from fresh sanitized current-main copies, followed by an independent skeptical review. Exactly two deep trials were completed.

Outcomes

  1. Localization catalog compiler — partial local proof only; no production adoption decision.
    A dependency-free prototype validated structured catalogs, fallback rules, diagnostics, deterministic tables, and a 100,000-message / 18-locale stress shape. The reviewer found and the coordinator fixed incomplete traversal of comma-separated fallback locales, with a regression test. BogKit components considered: Fold, ESE, ANNy. Used: none.

  2. Multi-tenant webhook delivery scheduler — no fit for BogKit.
    A dependency-free in-memory scheduler modeled ordering, bounded queues, rate limits, retries, crash recovery, and outage traces. The reviewer found and the coordinator fixed jitter exceeding the configured retry cap, with regression coverage. The corrected prototype remains exploratory only; it does not claim HTTP, PostgreSQL, durable process recovery, or production throughput. BogKit components considered: Fold, ESE, ANNy. Used: none.

Review and synthesis

  • No BogKit correctness defect was demonstrated.
  • No new recurring BogKit improvement or dashboard candidate was promoted.
  • Existing thresholds remain unchanged: capability/operational-boundary matrix (20 prior trials), component-selective project path (10), and nameable pipeline/reader patterns (3).
  • Prototype defects were fixed before archival; production semantics and external-system evidence remain explicitly unproven.
  • No BogKit core code or existing examples were changed.

Validation

  • Catalog: 4 tests, valid/invalid demonstrations, two deterministic stress compiles with matching SHA-256, formatting, strict Clippy, and release build passed.
  • Webhook: 8 debug tests, 8 release tests, formatting, strict Clippy, and 1,000 release repetitions passed.
  • Locked offline nested-workspace tests and strict Clippy passed; BogKit root workspace tests passed.
  • Archive scope, diff, credential, binary, symlink, large-file, and generated-artifact checks passed. The broad formatter still reports an existing unrelated difference in `examples/search/src/main.rs), which was left unchanged.

The daily report is immutable; the rolling dashboard comment is updated separately.

urcades commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

BogKit simulated-developer report — 2026-08-08

Commit: e1916d1 · full archived report

Two blind developers worked from fresh sanitized current-main copies, followed by one independent skeptical reviewer. Exactly two deep trials were completed. Both reached evidence-backed no-fit decisions for BogKit's role in the core computation.

Outcomes

  1. Conservative remittance-to-claim reconciliation — no fit.
    A standalone exact-cent matcher resolved bounded candidate clusters, quarantined uncertainty, and compared against an exact-reference-first greedy baseline. From the final archive path, 6 library and 17 adversarial tests passed; the authored 62,000-claim / 50,000-remittance run linked all 49,995 authored unambiguous truth links with no verifier or privacy failures in 4.628s, and ten shuffles were byte-identical. The production Java/SQL executable, external fixture, independent truth, specified hardware, and peak RSS were unavailable, so no production accuracy or speed claim is made. BogKit components considered: Fold, ESE, ANNy. Used: none.

  2. Advisory container-yard reshuffling — no fit.
    A standalone deterministic proposal generator validated placement rules, separately replayed moves, and atomically published either a complete plan or a review result. Its 1 library and 15 acceptance tests passed. One micro-geometry improved from 3 baseline relocations to 2, but the preserved three-stack witness proves the bounded heuristic can reject a feasible wave. The 30 production snapshots, C# results, production 20% aggregate, and hard end-to-end 10-second guarantee were not established. BogKit components considered: Fold, ESE, ANNy. Used: none.

Skeptical review and fixes

The initial audit found 4 blockers, 8 important limits, and 3 minor scope issues. Before archival:

  • remittance decisions were restricted to candidates supported by both nonempty identity sources;
  • duplicate remittance IDs and claim revision keys were quarantined before indexing;
  • the comparator was corrected to stage exact references before fallback;
  • yard output publication became atomic and exclusive across success, review, malformed input, timeout, replay rejection, and injected write failure;
  • the yard verifier was expanded to check all executable metadata and explanations;
  • broad split, reversal, feasibility, timing, baseline, verifier-independence, and production claims were narrowed;
  • both temporary child workspaces and lockfiles were removed in favor of the lab's single nested workspace and lockfile.

The same reviewer reproduced the focused counterexamples and controls, found no new critical or important breakage, and returned APPROVED_FOR_PUBLICATION.

Synthesis

  • No BogKit correctness defect was demonstrated.
  • No new recurring finding count changed and no candidate was promoted.
  • Existing thresholds remain: capability/operational-boundary matrix (20 independent trials), component-selective project path (10), and nameable pipeline/reader patterns (3).
  • Exact reconciliation/flow machinery and general state-space planning remain distinct one-trial observations, not justification for a new BogKit subsystem.
  • The no-fit signal is useful: BogKit should not gain durable or approximate machinery merely to participate in a frozen exact computation.

Validation

  • Both corrected trial suites, package formatting, strict Clippy, demos, permanent remittance overlap reproduction, full remittance acceptance, and ten shuffles passed from final archive paths.
  • Locked offline tests and strict Clippy passed the full nested simulation workspace.
  • All 45 BogKit unit and documentation tests passed.
  • Scope, diff, coverage JSON, credential, binary, symlink, large-file, generated-output, child-lock/workspace, and archive checks passed; only developer-simulation/ changed.

This daily report is immutable. The rolling dashboard is updated separately.

urcades commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

BogKit simulated-developer report — 2026-08-09

Commit: 511e4d0 · full archived report

Two blind developers worked from fresh sanitized current-main copies, followed by one independent skeptical reviewer. Exactly two deep trials were completed. Both reached evidence-backed no-fit decisions for BogKit's role in the core computation.

Outcomes

  1. Incremental syntax highlighting without stale tokens — no fit.
    A standalone line-cache prototype matched its authoritative full lexer across 100 documents / 10,000 edits, 9 handwritten cases, the reviewer's independent 100,000-edit corpus, and a full 10 MiB / 2,000-edit oracle run. All 1,900 localized edits met the declared locality thresholds; the lexer-byte ratio was 2.001425%; exact reproducibility compared equal 20,576,672-byte canonical values. The 58.7367 MiB figure is only a retained-index estimate, not peak RSS, and the prototype does not establish production-editor semantics or whole-editor latency. BogKit components considered: Fold, ESE, ANNy. Used: none.

  2. Ordered firewall policy impact analyzer — no fit.
    A standalone exact first-match analyzer found the synthetic region missed by packet sampling, replayed deterministic change/reachability witnesses, and passed 14 acceptance tests plus 2.56M exhaustive tuples and 2M full-width probes with zero disagreements. After archive-profile normalization, the 50,000/50,000-rule fixture analyzed in 0.13s and verified 25,001 witnesses in 0.81s; archived-profile RSS was unavailable, and the result is not a universal bound for fragmented policies or real vendor semantics. BogKit components considered: Fold, ESE, ANNy. Used: none.

Skeptical review and fixes

The audit counted 4 blocker/critical, 9 important, and 3 minor findings. Before archival:

  • the lexer fixed an EOF-reaching cache-partition defect that split one String token into 5..14 and 14..15 instead of 5..15;
  • lexer reproducibility was strengthened from digest comparison to exact canonical byte comparison, and memory/counter claims were narrowed;
  • the firewall analyzer reserved the implicit default-deny identifier and closed every report schema layer;
  • stale requested reports are removed after failed reruns;
  • direct, canonical, symlink, and hard-link report/input aliases are rejected without changing either input;
  • report publication now exclusively creates, syncs, and atomically renames an owned temporary file while preserving foreign collision files;
  • both standalone child workspaces, locks, and package release profiles were removed before final-path remeasurement.

The same reviewer reproduced the fixes and returned APPROVED_FOR_PUBLICATION.

Synthesis

  • No BogKit correctness defect was demonstrated.
  • No recurring finding count changed and no candidate was promoted.
  • Existing thresholds remain: capability/operational-boundary matrix (20 independent trials), component-selective project path (10), and nameable pipeline/reader patterns (3).
  • Incremental lexer convergence and exact static firewall analysis remain separate one-trial domain needs, not justification for new BogKit subsystems.
  • The useful no-fit signal remains narrow: do not add durable or approximate machinery when an exact local algorithm over one buffer or complete snapshot carries the load-bearing correctness.

Validation

  • Final archive paths passed 6 syntax tests, 14 firewall tests, focused formatting, strict release Clippy, runnable demos, permanent adversarial regressions, and both large workloads.
  • Locked offline release tests and strict Clippy passed the full nested simulation workspace.
  • All 45 BogKit unit and documentation tests passed.
  • Scope, diff, coverage JSON, credential, binary, symlink, large-file, generated-output, child-lock/workspace/profile, and archive checks passed; only developer-simulation/ changed.

This daily report is immutable. The rolling dashboard is updated separately.

urcades commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-10

Two blind developers evaluated substantially different existing-software
problems from separate sanitized copies of current main: a bilingual
support-case duplicate finder beside authoritative PostgreSQL, and a fully
offline repair-cafe lending ledger. Both built working BogKit integrations,
but neither justified production adoption over its existing baseline. A
separate skeptical reviewer confirmed one narrow Fold correctness defect and
found one important prototype ordering bug plus three minor evidence/reporting
issues. The original developer fixed the prototype bug and both archives passed
fresh normalized-workspace checks. No BogKit core or existing example changed.

Trial 1 — Bilingual support-case duplicate finder

  • Persona: customer-support platform developer with production TypeScript
    experience and six months of Rust
  • Existing system: PostgreSQL full-text search over a privacy-scrubbed
    projection of resolved support cases
  • Problem: improve English/Spanish duplicate-case recall without moving
    authority, permissions, or workflow ownership out of PostgreSQL
  • Outcome: local mechanics proof only; no production fit on current evidence
  • BogKit components considered and used: Fold, ESE, and ANNy
  • Archive:
    runs/2026-08-10--support-case-finder

Evidence

  • Quality commands:
    • cargo fmt -p support-case-finder -- --check
    • cargo test --offline --locked --release -p support-case-finder --all-targets
    • cargo clippy --offline --locked --release -p support-case-finder --all-targets -- -D warnings
  • Tests: 2 unit tests passed; the separate panic reproducer compiled and the
    complete nested workspace test suite also passed.
  • Disclosed retrieval fixture: lexical, semantic, and hybrid each returned a
    known case for 6/6 queries at rank 5. At the more discriminating rank 1,
    lexical scored 5/6, semantic 4/6, and hybrid 5/6. The hybrid therefore did
    not improve on lexical even in the toy comparison.
  • Mechanics: three reopened processes emitted byte-identical ordered IDs; an
    aborted child left the prior index queryable; retry made an edit and insert
    visible and removed a deletion; every displayed excerpt was an exact
    substring of approved fixture text.
  • Normalized archive scale run: 75,000 synthetic cases built and checkpointed
    in 80.43 seconds, occupied 321,337,660 bytes, applied 2,000 mixed changes in
    1.78 seconds, and served the repeated warm query at 17.24 ms p95. Developer
    and reviewer runs observed 254.2–321.3 MB, so storage is reported as a range,
    not a stable size. The host was not constrained to two cores and peak memory
    was unavailable.
  • Primary acceptance gate: not run. The private 200-query English/Spanish
    judgments and the production 58% recall@5 baseline implementation were not
    available, so the required 72% recall@5 gain remains unestablished.

Friction trail

The public search example made Fold, ESE, and ANNy look unusually direct for a
hybrid side index. The first offline Cargo build still attempted ESE's network
download until an existing model/tokenizer cache was supplied. Source
inspection then showed that the embedded model is English-specific, while the
brief is explicitly bilingual. The developer used a small disclosed fixture
and retained PostgreSQL as authority rather than turning synthetic success into
a production claim.

Crash testing uncovered the strongest result. A child-process abort during an
uncommitted refresh preserved the old index and allowed retry. But if the
service catches a panic resumed by Fold's wtx, the committed row remains
readable and the next write panics with Fjall's poisoned tx lock. The
reviewer independently reproduced the minimal expected-failure binary and
traced the cause to resuming the user panic while the transaction's mutex guard
is still alive.

Findings

  1. Caught write panic poisons later writes — BogKit correctness defect,
    important severity, high confidence.
    Run
    cargo run --offline --locked --release -p support-case-finder --bin panic_poison;
    it exits 101 after proving rollback and then prints poisoned tx lock. The
    smallest correction is to drop the underlying write transaction before
    resuming the panic and add both Stream and KeyedStream regressions that
    prove rollback plus a successful later write.
  2. ESE model and offline-build boundaries — documentation gap, important
    severity, high confidence.
    The public onboarding does not name the model,
    its English scope, cache path, first-build network behavior, or a hermetic
    build recipe. Document those facts before considering a model-selection API.
  3. Named pipeline reuse — API friction, important severity, high
    confidence.
    Closure-bearing pipeline types again forced macros around
    ordinary open/search helpers. This independently strengthens the existing
    documentation-first candidate; it does not justify type erasure.
  4. Production retrieval and resource gates — evidence limitation, important
    severity, high confidence.
    Synthetic text, an unconstrained host, missing
    peak memory, fixed top-K candidate behavior, and absent private judgments
    prevent an adoption claim.

Decision audit

The developer chose ESE dim-64/quant-8, default HNSW tuning with seed 42,
default ASCII-oriented BM25, equal-weight reciprocal-rank fusion, application
sorting by case ID, unweighted concatenated fields, fixed exact excerpts, and
one large daily transaction. Language/product filtering, candidate-cutoff tie
completeness, versioned index swapping, memory ceilings, concurrent refresh
traffic, and a restart policy remain unresolved. The local index is replaceable
and advisory; PostgreSQL remains the only authority.

Trial 2 — Repair-cafe tool lending kiosk

  • Persona: sole volunteer developer with strong Python/SQL and intermediate
    Rust experience
  • Existing system: CSV event files plus a Python-generated current-inventory
    snapshot
  • Problem: keep ordered history and current state atomic, replayable,
    inspectable, searchable, and fully offline on one small laptop
  • Outcome: no fit for the authoritative store; Fold is a successful local
    proof, but embedded SQL remains the better production choice
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: Fold only at the application level
  • Archive:
    runs/2026-08-10--repair-cafe-kiosk

Evidence

  • Quality commands:
    • cargo fmt -p repair-cafe-fold-trial -- --check
    • cargo test --offline --locked --release -p repair-cafe-fold-trial --all-targets
    • cargo clippy --offline --locked --release -p repair-cafe-fold-trial --all-targets --all-features -- -D warnings
  • Tests: 6 tests passed, including transition/rejection atomicity, reopen and
    historical queries, canonical lookup, live order guards, and matching import
    order guards.
  • Process interruption: a child flushed pending event and projection views
    inside an uncommitted transaction, then aborted. Reopen exposed the complete
    old state; a later committed event reopened with both new views.
  • Normalized archive benchmark: 8,000 items, 1,200 borrowers, and 100,000
    accepted events imported in 1.18 seconds; five injected bad rows were all
    reported; reopen took 0.198 seconds; 1,000 point reads measured 0.003 ms p95;
    the synthetic search set scored 50/50 in 3.53 seconds; three fresh processes
    reproduced byte-identical current views. Peak RSS was 172.0 MiB for the
    coordinator and 171.3–172.8 MiB for replay children on this host.
  • Baseline comparison: not benchmarked. Embedded SQL still provides familiar
    uniqueness/foreign-key constraints, indexed ordered history, inspection,
    backup, and recovery; the report does not invent comparative speed or size.

Friction trail

Fold genuinely kept accepted events and the current/history projection in one
transaction, while ordinary deterministic matching made ESE and vector search
unnecessary. However, a Fold-only application still resolved ANNy through
Fold's unconditional dependency. Naming the reusable pipeline and reader also
required about forty lines of nested aliases and function-pointer casts.

The reviewer found that the first implementation validated a return against
current state, then sorted all history by caller-supplied sequence during
materialization. Checkout sequence 10 followed by return sequence 5 therefore
passed preflight and panicked during replay. The developer chose an explicit
global append-only policy, rejecting reused or non-increasing sequences before
wtx in both live writes and imports. Permanent regressions prove both errors
leave state/history unchanged and that a later valid write still succeeds. The
small proof scans all events for this check; that O(total events) cost is
disclosed rather than hidden behind a new subsystem.

Findings

  1. Out-of-order event panic, fixed — prototype correctness defect,
    important severity, high confidence.
    Strict append-order validation and
    two permanent regressions close the hole without changing BogKit core. The
    no-fit recommendation remains.
  2. Fold-only consumers still pull ANNy — API/packaging friction, important
    severity, high confidence.
    cargo tree independently confirmed
    application → Fold → ANNy even though the kiosk rejects vector search. This
    strengthens the existing component-selective candidate; the narrowest
    improvement is an optional HNSW/ANNy integration with a simple default.
  3. Durability and validation boundaries — documentation gap, important
    severity, high confidence.
    Public onboarding should distinguish process
    atomicity, checkpoint/power-loss durability, transactional reads, and why
    pre-transaction validation depends on the stated single-writer rule.
  4. Relational ledger — poor product fit, high confidence. Fold can perform
    the workload, but it moves uniqueness, references, ordering, inspection,
    and recovery into bespoke Rust without demonstrating an advantage over the
    persona's embedded-SQL baseline. Do not add relational or SQL-like machinery
    to BogKit based on this trial.

Decision audit

The developer chose one global strictly increasing sequence, append-only
correction events, application validation, a per-item history vector, day-level
time, checkpoint after bulk work, and an O(8,000) normalized edit-distance
search. Backdated insertion, master-data mutation, real CSV parsing, live-copy
backup, corruption/version migration, power loss, production volunteer query
logs, and the actual old laptop remain untested. The full-line/history rewrite
and O(total-events) order check are proof-scale choices, not production advice.

Skeptical review

  • Claims reproduced: both narrow non-production outcomes; Fold multi-view
    atomicity; keyed edit/delete and process reopen; deterministic output;
    exact-source/privacy shape; the caught-panic writer poisoning; 75,000-case
    scale mechanics; kiosk rejection nonmutation; transaction abort/reopen;
    100,000-event import; exact replay; lookup; memory; and the transitive ANNy
    dependency.
  • Claims rejected or softened: production retrieval gain, Spanish embedding
    quality, stable storage size, declared-host performance, Trial 1 peak memory,
    every crash boundary, real kiosk search language, power-loss durability, and
    measured superiority over embedded SQL.
  • Review count: 0 blocker/critical, 2 important, and 3 minor findings. One
    Important is the confirmed Fold defect; the other is the fixed kiosk order
    bug. Minor corrections covered transitive-dependency wording, generated-data
    location, and the exact boundary exercised by Trial 1's abort check.
  • Final verdict: both archives approved after the original kiosk developer's
    correction and an independent scoped re-review.

Cross-run synthesis

  • New confirmed defect candidate: Fold's caught-wtx writer poisoning meets
    the one-serious-reproducer threshold. The proposal is only the drop-before-
    resume correction and two regression tests; the lab does not modify core.
  • Recurring storage/product-fit boundary: 22 independent trials now support a
    concise public capability matrix. Today's two cases distinguish a replaceable
    advisory side index from a relational source of truth.
  • Component-selective setup: 11 independent sources. The kiosk adds direct
    evidence that choosing Fold alone still brings unused ANNy code.
  • Nameable pipelines/readers: 5 independent sources, now high confidence.
    Keep the candidate documentation-first: show aliases, function pointers,
    macros, and ordinary helper signatures before considering a new abstraction.
  • Observation not yet promoted: ESE model provenance, English-only suitability,
    and first-build download behavior deserve documentation, but this is the
    first archived developer trial that selected ESE. Multilingual model choice,
    metadata filtering, index swapping, relational constraints, and SQL-like
    range queries remain one-scenario needs or explicit no-fit boundaries.
  • Outcome totals: 28 trials — 25 no_fit and 3
    local_proof_only_no_production_fit.

Validation

  • New packages: 8 focused tests passed; formatting and strict release Clippy
    passed with warnings denied.
  • Runnable evidence: support retrieval/update/crash/determinism/privacy demo,
    75,000/2,000 scale run, expected writer-poisoning reproducer, kiosk demo,
    transaction-interruption run, and 100,000-event benchmark all ran from the
    normalized archive paths.
  • Nested lab workspace: child workspaces, child lockfiles, and package release
    profiles were removed. One lock resolved both unique packages; locked offline
    release tests and strict Clippy passed the entire archive workspace.
  • BogKit root workspace: cargo test --workspace passed all 45 unit and
    documentation tests against 80fd3c9a023e877fff2e5d127accca386d437af0.
  • Archive boundaries: final checks cover diff whitespace, valid JSON,
    changed-path scope, child workspace/locks, generated build/database/binary
    output, large files, symlinks, and credential patterns. Every retained change
    is under developer-simulation/.

Full branch report: developer-simulation/reports/2026-08-10.md

urcades commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-11

Two blind developers evaluated substantially different existing-software
problems from separate sanitized copies of current main: repairing derived
cold-chain incident state from an authoritative NDJSON archive, and admitting
receiving-dock bookings whose command result and audit must commit together in
PostgreSQL. Both built runnable Fold integrations or boundary reproducers and
reached grounded production no-fit conclusions. A separate skeptical reviewer
confirmed one serious Fold persisted-value error-handling defect and found
three Important scheduling-model defects. The original dock-slot developer
fixed those defects, the same reviewer independently approved the corrections,
and both normalized archives passed fresh workspace checks. No BogKit core or
existing example changed.

Trial 1 — Cold-chain excursion state repair

  • Persona: regional food-distributor platform engineer with six years of
    production Python and one year of Rust
  • Existing system: append-only NDJSON observation/configuration archive with a
    disposable derived SQLite state database
  • Problem: obtain exact canonical incident state under duplicates, late
    uploads, linked corrections, and backdated temperature limits
  • Outcome: no fit for the production reducer; the application-specific replay
    remains simpler and materially lighter
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: Fold
  • Archive:
    runs/2026-08-11--cold-chain-repair

Evidence

  • Quality commands:
    • cargo fmt --manifest-path developer-simulation/Cargo.toml -p cold-chain-repair -- --check
    • cargo test --manifest-path developer-simulation/Cargo.toml --locked --offline --release -p cold-chain-repair --all-targets
    • cargo clippy --manifest-path developer-simulation/Cargo.toml --locked --offline --release -p cold-chain-repair --all-targets -- -D warnings -D clippy::all -D clippy::pedantic
  • Tests: seven acceptance tests and two CLI/subprocess tests passed. The
    100-seed differential test used fresh Fold databases for chronological
    reference, shuffled candidate, and repeated candidate batches.
  • Canonical behavior: hand-checked five- and ten-minute boundaries, exact
    observation/configuration provenance, correction and backdated-limit
    isolation, duplicate idempotency, malformed-batch rejection, and byte-stable
    NDJSON-to-JSON repair all passed.
  • Process exits: an exit inside an uncommitted Fold transaction, immediately
    before snapshot rename, immediately after rename, and restart convergence
    all exposed one complete prior-or-next snapshot for the tested boundaries.
  • Normalized archive demonstrations: the fixture demo exactly matched the
    separate replay oracle, verify 100 passed, and the process-exit demo passed.
  • One-million-observation/400-freezer measurements were independently
    reproduced. The reference used 0.716–0.727 seconds inside the reducer and
    511.46–511.51 MB peak RSS; the Fold-backed candidate used 3.230–3.320 seconds
    and 1.155–1.156 GB peak RSS. Both met the 60-second target and failed 384 MiB;
    the candidate also exceeded the stated 512 MiB server. Fixture generation,
    validation, and replay remain in memory, so this is an architecture failure,
    not an isolated Fold memory benchmark.

Friction trail

The public Fold examples made a durable keyed record index plausible, while ESE
and ANNy were clearly unrelated to deterministic temperature state. Source
inspection confirmed useful local multi-view atomicity, but no operator for
invalidating and replaying an event-time suffix after an old observation or
configuration changes. The authoritative archive append and canonical snapshot
also remain outside Fold's transaction, so the hardest domain and recovery
logic stayed in the application.

The first ordinary persisted enum compiled under Fold's public
Serialize + DeserializeOwned bounds and committed, then table iteration
panicked with Postcard WontImplement. Changing to external enum tagging and
omitting a None field with skip_serializing_if produced a second read-time
panic, DeserializeUnexpectedEnd. The developer retained both minimal
reproducers and used external tagging with the option always serialized in the
working proof.

The reviewer reproduced both panics and then called Postcard directly without
Fold. Postcard returned the same two expected errors: its non-self-describing
format cannot decode an internally tagged enum or a schema field omitted at
serialization. The consumer-facing Fold defect is therefore the undisclosed
narrower storage contract plus an internal unwrap that converts a predictable
type incompatibility into an unrecoverable read-time panic after data commits;
the underlying format limitations are not Postcard correctness bugs.

Findings

  1. Persisted-value incompatibility panics at read time — BogKit/Fold
    correctness and error-handling defect, important severity, high
    confidence.
    Both compact reproducers deterministically exit 101 at
    TableReader::iter. First document Postcard-compatible Serde shapes at every
    persisted-value entry point; then replace serialization/deserialization
    unwraps with recoverable typed errors where an API-compatible path exists.
    This serious reproducer satisfies the charter's single-defect candidate
    exception without implying a new codec subsystem.
  2. Straightforward Fold record index misses the memory ceiling — prototype
    performance failure, high confidence.
    The reference is already too large,
    and the duplicate index makes the candidate substantially worse. Stream the
    application-specific reducer rather than adding a BogKit subsystem.
  3. External authority and ordered temporal repair — poor product fit, high
    confidence.
    Fold does not own the archive/snapshot publication boundary
    and does not remove correction/configuration replay. Do not add a temporal
    database or suffix-repair primitive from this one workload.
  4. Named pipeline helpers — recurring API friction, high confidence. The
    closure-composed pipeline stayed inline because ordinary helper types were
    awkward. This strengthens the existing documentation-first candidate, not
    a type-erasure proposal.

Decision audit

The developer chose integer-second timestamps, (observed_at, observation_id)
tie ordering, continuous classification across unspecified sampling gaps,
transition time at the first qualifying observation, conflicting-duplicate and
ambiguous-correction rejection, backdated configuration lookup by effective
time, fixed-shape canonical JSON, 4,096-record Fold transactions, and
same-directory synced snapshot rename. A seven-day correction fixture exists,
but no wall-clock cutoff policy was invented. The process-exit suite does not
cover power loss or the boundary after some index chunks commit and before the
full archive is indexed; restart convergence at that exact intermediate point
is reasoned rather than executed. The oracle has a separate replay loop but
shares record types, validation inputs, and fixture structure, so it is an
algorithmically separate oracle rather than a fully independent production
implementation.

Trial 2 — Receiving-dock slot admission

  • Persona: warehouse-management backend engineer with eight years of
    TypeScript/PostgreSQL experience and three months of Rust
  • Existing system: six service replicas using serializable PostgreSQL and
    advisory locks, with Redis-based hold expiry outside the commit boundary
  • Problem: make hold, confirm, cancel, reschedule, expiry, idempotency, booking,
    and audit decisions atomic and deterministic
  • Outcome: no fit for the production authority; keep every accepted command
    and audit decision in one PostgreSQL transaction
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: Fold in the split-commit reproducer
  • Archive:
    runs/2026-08-11--receiving-slot-admission

Evidence

  • Quality commands:
    • cargo fmt --manifest-path developer-simulation/Cargo.toml -p receiving-slot-admission -- --check
    • cargo test --manifest-path developer-simulation/Cargo.toml --locked --offline --release -p receiving-slot-admission --all-targets
    • cargo clippy --manifest-path developer-simulation/Cargo.toml --locked --offline --release -p receiving-slot-admission --all-targets -- -D warnings -D clippy::all -D clippy::pedantic
  • Tests: eight tests passed after review, including expiry without an explicit
    sweep, reschedule after an unrelated expiry, zero/nine-duration nonmutation,
    distinct stale-version and payload-mismatch results, exact retries,
    deterministic fixtures, and local collision invariants.
  • Domain-model smoke: a deterministic 10,000-command fixture plus 100 generated
    seeds passed; four threads over 30 runs preserved invariants behind one local
    mutex. These are explicitly not PostgreSQL or six-process evidence.
  • Split-commit reproducer: real child exits immediately before and after a
    mocked atomic authority commit. An exit after the authority commit and before
    the real Fold mirror write leaves booking/audit present with no Fold row.
    Retry can repair the mirror but cannot make the original commits atomic.
  • Normalized archive demo: all model and split-commit checks passed. The local
    repeat-heavy 100,000-command run measured about 2.69–2.71 million commands/s
    and 0.000459–0.000500 ms p95. These numbers are invalid for the PostgreSQL
    acceptance gate and are retained only as a runnable smoke check.
  • PostgreSQL gate: not run. psql, postgres, and pg_isready were absent;
    port 5432 was closed; no connection setting existed; and no service was
    installed or started. Serializable locking, database timestamps, four real
    clients, six replicas, schema size, and database latency remain unverified.

Friction trail

The public examples showed that Fold atomically maintains views inside an
embedded store, but did not state the external transaction or writer boundary.
Minimal source inspection found a concrete Fjall SingleWriterTxDatabase:
Stream::wtx creates and commits its own transaction and exposes no
PostgreSQL/caller-owned transaction, prepare/commit, or storage-adapter hook.
ESE and ANNy were irrelevant to exact scheduling. A downstream outbox-fed Fold
projection remained possible, but no measured read need justified operating a
second persisted system.

The first model also demonstrated why skeptical review is required. Expired
holds continued consuming capacity unless an explicit sweep ran, zero-duration
reschedules were accepted, and the exact pedantic lint gate failed with 19
diagnostics. The original developer moved logical-time expiry into every new
non-replayed authority command, added range validation/nonmutation, replaced
lossy conversions, and separated a genuine stale-version assertion from
payload mismatch. Eight tests, strict lint, the release demo, and the real
split-commit boundary then passed independently.

Findings

  1. Expired-capacity and invalid-reschedule bugs, fixed — prototype
    correctness defects, important severity, high confidence.
    Logical
    expires_at <= command_now processing and duration 1..=8 validation now
    occur before mutation. Permanent regressions cover both fixes; no BogKit
    core change was involved.
  2. External transaction/writer boundary — documentation gap, important
    severity, high confidence.
    State concisely that Fold transactions are
    scoped to the embedded Fjall store and do not join an external database
    transaction, and name the intended writer topology. Do not add a PostgreSQL
    adapter based on this no-fit workload.
  3. Component-selective setup and nameable pipelines — recurring API
    friction, high confidence.
    Only Fold was relevant, yet the starter bundles
    unrelated search components, and closure-bearing pipeline types remain hard
    to name for ordinary helpers. Keep both improvements narrow and
    documentation/scaffolding-first.
  4. PostgreSQL command authority — poor product fit, high confidence. The
    existing database already provides the only commit boundary that can satisfy
    the requirement. No BogKit dependency is the lean result.

Decision audit

The model chose operation-scoped idempotency keys, stable payload hashes,
confirmed-booking exclusivity despite pallet spare capacity, ascending door
selection, time-conflict precedence, expiry at equality, explicit and implicit
expiry, injected logical command time, one decision row per stored command,
and eventual mirror repair only for demonstration. Payload-mismatch attempts
remain visible but do not append a second durable decision; carrier priority,
database schema/indexing, HTTP serialization, migrations, connection pooling,
real lock ordering, and production retry behavior remain unresolved. The
100-seed comparison repeats the same model, four workers serialize through one
mutex, and the atomic file is only a benign authority model. None is presented
as database evidence.

Skeptical review

  • Trial 1: APPROVED_FOR_ARCHIVE. Nine tests, formatting, strict pedantic
    Clippy, demo, 100 seeds, process-exit boundaries, independent one-million
    measurements, and both Fold/Postcard reproducers passed. Two Minor evidence
    limits remain disclosed.
  • Trial 2 initial verdict: REJECTED_UNTIL_FIXED. Three Important items were
    the expired-hold capacity bug, invalid reschedule duration, and failed strict
    lint gate. One Minor evidence-name issue was also corrected.
  • Trial 2 final verdict: APPROVED_FOR_ARCHIVE_AFTER_FIXES. Eight tests,
    formatting, strict pedantic Clippy, logical-time source inspection, release
    demo, split-commit boundary, documentation, and cleanliness all passed the
    same reviewer's scoped rerun. The PostgreSQL evidence limit remains.
  • Consolidated review count: 0 blocker/critical; 3 Important fixed and 0
    remaining; 4 Minor notes, of which one was fixed and three remain as explicit
    evidence limits.

Cross-run synthesis

  • New confirmed candidate: Fold's persisted-value codec panic meets the
    one-serious-reproducer threshold. Keep it separate from the caught-wtx
    writer-poisoning candidate because the roots and corrections differ.
  • Storage, transaction, and concurrency boundaries: 24 independent trials.
    Today's runs add an authoritative external archive/snapshot boundary and an
    external PostgreSQL/multi-process boundary.
  • Component-selective setup: 12 independent sources. The dock-slot developer
    again rejected unrelated ESE/ANNy setup for a Fold-only evaluation.
  • Nameable pipeline/readers: 7 independent sources, now strengthened by both
    developers. Preserve the documentation-first scope.
  • Do not promote event-time suffix replay, a PostgreSQL adapter, distributed
    writer coordination, or a general constraint solver. Each is either a
    one-scenario missing capability or an explicit product-fit boundary.
  • Outcome totals: 30 trials — 27 no_fit and 3
    local_proof_only_no_production_fit.

Validation

  • New packages: all 17 focused tests passed; formatting and strict release
    Clippy passed with warnings, clippy::all, and clippy::pedantic denied.
  • Runnable evidence: cold-chain fixture/reference, 100-seed differential,
    process-exit recovery, both expected Fold panic reproducers, dock-slot domain
    model, local collision smoke, and real Fold split-commit boundary ran from
    normalized archive paths.
  • Nested lab workspace: child workspaces, child lockfiles, and package release
    profiles were removed. One lock resolves both unique packages. Locked offline
    release tests passed the complete nested workspace, and workspace-wide Clippy
    passed with warnings denied. The unchanged ESE model/tokenizer were supplied
    only through an external build cache; no model or generated data was archived.
  • BogKit root workspace: cargo test --workspace --locked --offline passed all
    45 unit and documentation tests against
    80fd3c9a023e877fff2e5d127accca386d437af0.
  • Archive boundaries: final checks cover diff whitespace, valid JSON,
    changed-path scope, child workspace/locks/profiles, generated build/database/
    binary output, large files, symlinks, and credential patterns. Every retained
    change is under developer-simulation/.

Full branch report: developer-simulation/reports/2026-08-11.md

urcades commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-12

Two blind developers evaluated substantially different existing-software problems from separate sanitized copies of current main: an offline authoritative-DNS change-admission gate and a hard-real-time MIDI event scheduler. Both independently rejected Fold, ESE, and ANNy, built compact dependency-free Rust evidence artifacts, and retained the existing production baseline.

A separate skeptical reviewer rejected both first handoffs, finding one Critical and eight Important prototype defects. The original developers repaired every serious issue, the same reviewer independently reproduced the corrections, and both normalized archives received final publication approval. No BogKit core or existing example changed.

Trial 1 — Authoritative DNS change-admission gate

  • Persona: hosting-platform reliability engineer with seven years of Go/shell and four months of Rust
  • Baseline: named-checkzone remains authoritative; humans inspect diffs before deployment
  • Problem: produce a deterministic, fail-closed semantic report over old and proposed BIND master-file snapshots
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Outcome: no fit. Keep the existing authority. The standalone gate is only an advisory prototype over immutable input snapshots.
  • Archive: DNS change gate

Evidence

  • 22 black-box tests and one fixture-generator integration passed in debug and release.
  • Strict formatting and Clippy passed with warnings, clippy::all, and clippy::pedantic denied.
  • The release CLI produced the checked-in 853-byte oracle exactly.
  • Maximum-u32 and stepped $GENERATE ranges fail promptly without panic or report replacement.
  • Decoded 63-octet escaped labels canonicalize exactly; 64-octet labels fail closed.
  • Static containment, input/output identity, collision-safe temporary publication, repeated publication, and complete old-or-new report behavior passed.
  • A synthetic 10,000-zone, 2,000,000-record-per-snapshot corpus ran three times in 3.38–3.45 seconds at 18.63–18.79 MB peak RSS after warm-up. This was an Apple M4 Pro macOS host, not the declared four-core Linux/256 MiB target, and the corpus was not the requested exhaustive semantic oracle.

Friction and findings

The public material made the component mismatch clear: embeddings and approximate search cannot make exact DNS decisions, while a persistent Fold store adds state and failure boundaries without parsing master files or improving a bounded comparison. A standard-library sorted representation was smaller and easier to audit.

Test-first work caught omitted $TTL inheritance and ambiguous per-snapshot record-cap accounting. Skeptical review found the more valuable issues:

  1. Critical range-cardinality overflow, fixed. Inclusive expansion count now uses checked wide arithmetic before any iteration or cap comparison.
  2. Escaped-name wire-length error, fixed. Length is measured from decoded DNS octets rather than the longer canonical display.
  3. Overstated containment, corrected. Pathname-based reads do not prove safety under concurrent replacement. The prototype now explicitly requires immutable, non-racing snapshots and marks the original race-safe requirement unproven.
  4. Report publication lifecycle, fixed. Temporary reports are exclusively created, collision-safe, ownership-tracked, synced, and cleaned on ordinary failures.
  5. Input/output aliases, fixed. Policy, root, master, include, symlink, and Unix hard-link identities are rejected before publication without changing inputs.

No item is a BogKit defect. Production evidence still lacks real named-checkzone comparison, the requested 1,000/60/40 oracle tables, concurrent-mutation containment, and target-host qualification.

Trial 2 — Hard-real-time MIDI event scheduler

  • Persona: ten-year production C++ audio developer with Rust Book and hobby-crate experience but no shipped Rust callback
  • Baseline: production C++ scheduler plus a slower rational test oracle
  • Problem: make exact timing, transport cleanup, overload, duplicate handling, and plan handoff executable without weakening the audio callback contract
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Outcome: no fit for BogKit and no production Rust replacement. Keep C++ until lock, syscall, FFI, and genuinely concurrent reclamation gates are proved.
  • Archive: MIDI scheduler model

Evidence

  • All 29 tests passed after repair: 14 callback, six validation, four modeled publication, two oracle/determinism, two real-time instrumentation, and one representative-shape test.
  • Strict formatting and Clippy passed with warnings, clippy::all, and clippy::pedantic denied.
  • One hundred fixed seeds agreed with a separately written exact rational oracle.
  • Focused cases cover half-open frames, collapsed events, strictly increasing successful tokens, transport resets, identity-checked notes, retryable failed callbacks, 4,096-event saturation, and ordinary/mixed termination overload with zero stuck active notes.
  • The release representative test prepared 180,000 events and 20,000 tempo nodes for 45 minutes with 250 modeled replacements.
  • An uncontrolled, unpinned 10,000,000-call macOS run observed p99 125 ns and maximum 49.791 microseconds. This is local smoke evidence, not a production comparison or portable guarantee.

Friction and findings

The public component contracts were direct no-fits: Fold is persistent transactional state, ESE allocates text embeddings, and ANNy performs allocating approximate search. None belongs in an exact fixed-capacity callback. The useful work was making the existing callback contract executable in a small standard-library model.

The first 23 tests missed four Important state defects that skeptical review exposed:

  1. Ordinary termination overload, fixed. Excess required terminations consolidate to one channel reset per affected channel, suppress note lifecycle output, and leave no active notes stuck.
  2. Termination identity mismatch, fixed. Note instances bind instance_id, channel, and key; mismatches reject with a stable error.
  3. Non-consecutive token replay, fixed. Successful callback tokens must increase strictly; old/equal tokens emit nothing and do not mutate state.
  4. Failed-call token consumption, fixed. The token commits only after successful scheduling, so a too-small termination output remains retryable with the same token.

The fake handoff is deliberately single-threaded. Lock attempts, forbidden system calls, FFI unwind behavior, and real concurrent reclamation were not instrumented. The test allocator covers allocation, zeroed allocation, and reallocation for 100,000 calls, not deallocation or the 10-million-call timing run. No item is a BogKit defect.

Skeptical review

  • Initial verdicts: DNS REJECTED_PENDING_FIX with one Critical, four Important, and two Minor; MIDI REJECTED_PENDING_FIX with four Important and three Minor.
  • Final verdicts: both APPROVED_FOR_ARCHIVE_AFTER_FIXES.
  • One Critical and eight Important findings are fixed; zero Critical or Important findings remain.
  • Five Minor evidence limits remain explicitly disclosed.
  • Final normalized-archive audit: APPROVED_FOR_PUBLICATION.

Cross-run synthesis

  • Outcome totals: 32 trials29 no fit, 3 useful local proof only, 0 adopted, 0 blocked.
  • Public capability and operational-boundary matrix: 26 independent trials.
  • Component-selective project path: 14 independent trials.
  • Nameable pipeline/reader patterns: unchanged at 7.
  • Confirmed BogKit defects remain unchanged at two: caught Fold write-transaction panic poisoning and persisted-value codec panic.
  • No new BogKit correctness defect or candidate improvement is justified today.
  • Do not promote a DNS parser, in-memory Fold mode, descriptor traversal layer, MIDI scheduler, callback publication primitive, or general real-time API.

Validation

  • New packages: 52 focused tests, strict pedantic lint, and both runnable demonstrations passed after archive normalization.
  • Full nested lab: all 32 packages passed release workspace tests; workspace-wide Clippy passed with warnings denied.
  • BogKit root: all 45 unit and documentation tests passed against 80fd3c9a023e877fff2e5d127accca386d437af0.
  • One nested workspace and lockfile resolve both new packages; no child workspace, lock, or package-local profile remains.
  • JSON, whitespace, changed-path scope, generated output/database/binary residue, large files, symlinks, and credential scans passed.
  • Every retained change is under developer-simulation/.

Full branch report: developer-simulation/reports/2026-08-12.md

urcades commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-13

Two blind developers evaluated substantially different existing-software
problems from separate sanitized copies of current main: advisory return and
refund reconciliation across carrier, warehouse, and payment evidence, and
municipal water-meter billing repair from cumulative-reading exports. Both
built dependency-light exact batch tools, rejected Fold, ESE, and ANNy, and
retained the existing external database and approval flows. A separate
skeptical reviewer rejected both initial handoffs, found one Critical and five
Important prototype or evidence defects, and approved the archives only after
two fix rounds by the original developers. No BogKit core or existing example
changed.

Trial 1 — Multi-carrier return refund reconciler

  • Persona: commerce-platform backend developer with seven years of
    TypeScript/PostgreSQL and four months of Rust
  • Existing system: PostgreSQL remains authoritative; a nightly Python/CSV job
    creates an operator review sheet and a separate approved service executes
    refunds
  • Problem: derive an exact, deterministic advisory refund plan from split
    parcels, warehouse corrections, substituted items, and ambiguous payment
    results without issuing a second refund
  • Outcome: no fit; retain the direct read-only snapshot/reference approach and
    keep payment execution outside the prototype
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-13--multi-carrier-return-refund-reconciler

Evidence

  • Quality commands:
    • cargo fmt --manifest-path developer-simulation/Cargo.toml -p return-reconciler-trial1 -- --check
    • cargo test --manifest-path developer-simulation/Cargo.toml --locked --offline --release -p return-reconciler-trial1 --all-targets --all-features
    • cargo clippy --manifest-path developer-simulation/Cargo.toml --locked --offline --release -p return-reconciler-trial1 --all-targets --all-features -- -D warnings -D clippy::all -D clippy::pedantic
  • Tests: 20 release tests passed after repair. Permanent regressions cover
    exact/conflicting retries, correction order, split parcels, substituted and
    excess units, payment states, cent and quantity caps, checked report totals,
    invalid payment ownership, verifier provenance, input/output aliases, normal
    publication failures, and abrupt child exits.
  • Representative shape: the generator produced exactly 25,000 returns, 40,000
    parcels, 70,000 authorized lines, 250,000 events, and 500 labeled adversarial
    returns. Candidate and separately structured reference agreed, and ten input
    shuffles produced byte-identical reports. The final trial run observed about
    273 ms candidate work, 303 ms reference work, and 5.844 seconds for the ten
    shuffles.
  • Memory evidence: three reviewer child-process high-water measurements were
    506,707,968–523,190,272 bytes (483.2–499.0 MiB), leaving only 13–29 MiB below
    the 512 MiB target. This is a narrow host-specific soft pass, not a portable
    resource guarantee.
  • Baseline comparison: the production Python/SQL implementation and real
    exports were unavailable, so no runtime or operational superiority was
    measured. The prototype adds executable audit rules but does not displace
    the authoritative baseline.

Friction trail

The public examples made ESE and ANNy direct no-fits for exact IDs, units, and
cents. Fold offered durable transactions and retractions, but the input is an
immutable PostgreSQL export and the output an advisory file: persisting another
copy would not remove reconciliation, quarantine, allocation, provenance,
reference verification, or external publication logic.

Test-first work found a relative-output false failure after a successful rename
and an underpowered provenance verifier. Skeptical review then found the
financially important gaps: return caps could report a full unit with only
partial-unit cents; report-wide u64 totals silently wrapped; and a pending
payment result for a nonexistent or wrong-return line was ignored while a
refund remained proposed. The repaired candidate, reference, and verifier now
fund whole units only, use checked report totals, and validate every raw payment
line/return identity before derivation and publication.

Findings

  1. BogKit does not improve this authoritative-snapshot boundary — poor
    product fit, high confidence.
    Fold adds a second durable store without
    sharing PostgreSQL's authority or removing exact domain rules; ESE/ANNy are
    unsafe or irrelevant for exact accounting. Retain no component.
  2. External authority and report lifecycle remain onboarding boundaries —
    documentation gap, high confidence.
    Add this source to the existing public
    capability/operational-boundary matrix; do not infer a new report-publisher
    or refund subsystem.
  3. Initial financial-accounting defects, fixed — prototype correctness, high
    confidence.
    One Critical overflow and two Important unit/payment issues
    now have permanent fail-before-publication regressions. They are not BogKit
    defects.
  4. Resource and production evidence are narrow — performance evidence limit,
    high confidence.
    The synthetic local run has little memory headroom and
    no real Python/SQL comparison, Linux qualification, operator study, or
    production-policy approval.

Decision audit

The prototype uses checked integer cents, deterministic whole-unit funding,
stable remainder-cent allocation, line-ID order for same-SKU authorizations,
explicit conflict quarantine, source/ingestion/event ordering for named
corrections, success-dominant prior payment results, a conservative pending
block, and atomic verified sibling-file publication. Money-field semantics,
remainder policy, multiple-correction policy, same-SKU allocation, return-cap
allocation, carrier-status authority, and operator workflow remain product
decisions rather than demonstrated production rules.

Trial 2 — Municipal water-meter billing repair

  • Persona: municipal-utility data engineer with nine years of SQL/Python and
    one year of Rust
  • Existing system: an authoritative vendor billing database plus understood
    SQL views, Python repair scripts, and a separate adjustment-approval import
  • Problem: deterministically reconstruct cumulative consumption across meter
    rollover, replacement, estimates, corrections, and already billed intervals
  • Outcome: no fit; retain the auditor-understood external authority and use the
    prototype only as an exact policy/evidence model
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-13--municipal-water-meter-billing-repair

Evidence

  • Quality commands:
    • cargo fmt --manifest-path developer-simulation/Cargo.toml -p water-repair -- --check
    • cargo test --manifest-path developer-simulation/Cargo.toml --locked --offline --release -p water-repair --all-targets
    • cargo clippy --manifest-path developer-simulation/Cargo.toml --locked --offline --release -p water-repair --all-targets --all-features -- -D warnings -D clippy::all -D clippy::pedantic
  • Tests: all 19 passed after repair. They cover rollover/regression,
    replacement boundaries, supersession, duplicates/conflicts, exact prior
    adjustments, ambiguity isolation, ordering and batch determinism, path
    identity, globally unique adjustment IDs, exact retries, and four real-child
    SIGABRT publication cases.
  • Correctness: the disclosed demo produced five repairs and one review and
    matched its hand-written canonical oracle byte-for-byte. A separately
    structured chronological reference agreed on 100 deterministic seeds and ten
    permutations at batch sizes 1, 7, and 64.
  • Process exits: real children terminated before staging and after a complete
    synced temporary file immediately before rename. Existing finals remained
    byte-identical, absent finals stayed absent, and restart removed the orphan
    stage before publishing one complete report.
  • Requested-count observation: a synthetic pre-sorted workload contained
    100,000 service points, 120,000 meters, 2,000,000 readings, and 200,000 billed
    intervals and completed in 0.159 seconds at 139.8 MiB peak RSS. It has only two
    intervals per point, no corrections or prior adjustments, and zero repairs or
    reviews. This is explicitly a partial best-case traversal/publication check,
    not representative repair performance.

Friction trail

Fold was the only plausible component because keyed updates and retractions
match corrected readings. Narrow source inspection confirmed those primitives,
but every consequential rule—installation validity, register rollover,
supersession, causal interval repair, provenance, per-service refusal,
canonical output, and external publication—remained application code. Durable
embedded state added lifecycle cost without measured benefit. ESE and ANNy were
unrelated to exact integer/time reconstruction.

The initial prototype passed ordinary tests but modeled publication failures as
returned errors rather than process exits and overstated a trivial large-shape
benchmark. Review also found candidate/reference disagreement when one
adjustment ID was reused across billed intervals. The repair adds real abrupt
child exits, labels the scale result partial/best-case, and establishes one
global snapshot adjustment-ID contract: identical rows are idempotent retries;
any differing reuse fails deterministically before derivation or publication.

Findings

  1. Fold does not remove temporal or external-publication ownership — poor
    product fit, high confidence.
    Keep the direct chronological reducer and
    external billing authority; do not add durable state for this batch job.
  2. Operational-boundary guidance is recurring — documentation gap, high
    confidence.
    This independently strengthens the existing capability matrix
    for immutable offline transformations and caller-owned publication.
  3. Initial lifecycle and identity defects, fixed — prototype correctness and
    evidence defects, high confidence.
    Three Important findings now have
    permanent subprocess, identity, retry, and order regressions. None is a
    BogKit defect.
  4. Scale and policy evidence remain partial — performance/validation limit,
    high confidence.
    The requested-count run is deliberately best-case; no
    production export, regulatory review, approved rollover policy, baseline
    timing, concurrent writer, or power-loss campaign exists.

Decision audit

The model uses exact integer volumes, canonical source ordering, a conservative
top-quarter-to-bottom-quarter rollover rule, fail-closed installation gaps and
ambiguous equal timestamps, explicit old/new meter boundary readings, named
correction targets, two adjacent causally affected intervals, global reading
and adjustment IDs with exact-retry collapse, prior adjustments added to old
usage, per-service ambiguity isolation, and one-writer atomic report
publication. Rollover, replacement-boundary, ambiguity, and adjustment policies
require utility approval. Stale-stage cleanup is not a multi-writer protocol.

Skeptical review

  • Trial 1 initial verdict: REJECTED_UNTIL_FIXED with one Critical, two
    Important, and two Minor findings. Trial 2 initial verdict:
    REJECTED_UNTIL_FIXED with three Important and two Minor findings.
  • Fix round 1 closed whole-unit funding, abrupt-exit evidence, and performance
    wording, but the reviewer found overflow, invalid payment ownership, and
    adjustment-ID divergence through independent harnesses.
  • Fix round 2 independently reproduced the repaired counterexamples. Final
    verdicts: both APPROVED_FOR_ARCHIVE_AFTER_FIXES, with zero Critical or
    Important and two disclosed Minor evidence limits per trial.
  • No BogKit correctness defect or new candidate improvement is justified.

Cross-run synthesis

  • Storage and concurrency boundaries: 28 independent trials. Both runs add an
    immutable external-database export and caller-owned canonical-report
    publication boundary.
  • Component-selective setup remains 14; neither trial substantially integrated
    a Fold pipeline, so today's observations do not justify an increment.
  • Nameable pipeline/readers remains seven.
  • Confirmed BogKit defects remain unchanged: caught Fold write-transaction
    panic poisoning and persisted-value codec panic.
  • Outcome totals: 34 trials — 31 no_fit and three
    local_proof_only_no_production_fit.
  • Do not promote a refund reconciler, water-billing repair subsystem, generic
    report publisher, in-memory Fold mode, or domain-specific temporal API.

Validation

  • New packages: 39 focused release tests pass after archive normalization;
    formatting and strict pedantic Clippy pass.
  • Runnable evidence: the return demo/verifier, exact representative/reference
    and ten-shuffle gate, water disclosed byte oracle, 100 seeds, ten-by-three
    determinism, real-child publication suite, and focused overflow/payment/
    adjustment-identity regressions pass from normalized archive paths.
  • Nested lab workspace: one workspace and lockfile resolve both unique new
    packages; no child workspace, lockfile, or package-local profile remains.
  • BogKit root workspace: cargo test --workspace --locked --offline passes all
    45 unit and documentation tests against
    80fd3c9a023e877fff2e5d127accca386d437af0.
  • Archive boundaries: final checks cover valid JSON, diff whitespace,
    changed-path scope, secrets, large/binary files, symlinks, child workspaces,
    locks/profiles, and generated build/database/runtime residue. Every retained
    change is under developer-simulation/.

Full branch report: developer-simulation/reports/2026-08-13.md

urcades commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-14

Two blind developers worked from separate sanitized copies of current main at 80fd3c9. One evaluated crash-safe branching undo history for a vector editor; the other evaluated an epoch-safe local mailbox mirror beside an authoritative server. A separate skeptic rejected both first handoffs, reproduced one Critical, six Important, and one Minor issue, and approved both archives only after focused repairs by the original developers.

Trial 1 — Crash-safe branching undo history

Persona: vector-editor document-core maintainer.
Outcome: useful local proof only; no production fit.
Archive: crash-safe undo history

Fold was used centrally as an atomic whole-state store and consistent read view. ESE and ANNy were rejected as no-fits for exact structured history. The retained proof keeps canonical JSON as interchange and compares Fold with a checksummed JSONL-plus-snapshot baseline and an in-memory semantic model.

Evidence:

  • All 24 tests, strict formatting, and strict pedantic lint passed.
  • Candidate, baseline, and model agreed after 1,000 generated actions: document digest b426134b815e9add, head 843, undo 784, redo 0.
  • Candidate p95 edit-submission latency was 17.050 ms versus 5.395 ms for the baseline.
  • Reopen was 19.528 ms versus 23.743 ms, only 1.22x faster.
  • Pre-compaction storage was 37,533,711 bytes versus 535,051; after compaction it was 2,111,098 versus 2,108,204.
  • The requested 60,000-object/250,000-action, 30-seed, 400-exit, concurrent-reader, memory, and full fault campaign remains unproved.

Friction and findings:

  • Fold supplied the narrow atomic persistence proof, but command validation, inverse construction, branching, exact retry identity, canonical JSON, retention, and compaction all remained application code.
  • Public examples do not demonstrate reopen, acknowledgement, checkpoint, or recovery diagnostics.
  • Skeptical review found a bad-tail acknowledgement-loss bug, extreme-coordinate panic, retry snapshot drift, and probabilistic request identity in the prototype. The repaired archive physically truncates and syncs recognized final-tail damage, prevalidates arithmetic, makes retries write-free, and persists exact canonical command bytes.
  • Those were prototype defects, not BogKit defects.

Decision audit: do not adopt the tested full-state layout. A normalized key-per-document/history/outcome layout remains untested. The smallest BogKit improvement is documentation: add a public reopen/checkpoint example before considering any API work.

Trial 2 — Epoch-safe mailbox mirror repair

Persona: desktop mail-sync maintainer experienced with SQLite.
Outcome: no fit; retain SQLite.
Archive: epoch-safe mailbox mirror

The SQLite baseline and independent Rust model were implemented before minimal Fold inspection. Fold was used only for a decisive transaction-boundary reproducer; ESE and ANNy were no-fits.

Evidence:

  • All 22 ordinary tests passed; the explicit local measurement passed separately. Strict formatting and pedantic lint passed.
  • One full-shape seed covered 80 mailboxes, 300,000 live messages, 600,000 responses, 30,000 duplicates, 12 epoch changes, 20 renames, 10 delete/recreate lifecycles, one disconnect, and five checkpoints.
  • SQLite matched the independent reference exactly in 49.101 seconds. Full replay admitted zero new batches and preserved manifest hash ca7ad005908e1699ae14787552eb4880a2fa1c551845cffbcb528b6f30349909.
  • Fresh bounded archive evidence observed 10,000 messages in 913.095 ms, 10,973.7 responses/s, 3.337 ms count p95, 2.483 ms reopen, and 2,867,200 bytes. These are local observations, not stable performance guarantees.
  • The requested 30 full seeds, 400 abrupt exits, peak RSS, write amplification, transcript cuts, and safe Fold candidate comparison remain unproved.

Friction and findings:

  • A Fold write closure commits prior writes whenever it returns normally, including a returned Err. That is documented behavior, but fallible external batches therefore need prevalidation or an overlay.
  • The trial independently confirmed the already tracked caught-panic writer-poisoning defect: the rejected row rolls back, but the next write on the same stream panics with poisoned tx lock. This is now the second independent source.
  • The skeptic also found prototype-only SQLite integer coercion and abandoned staging rows. The archive now rejects every unsigned value outside 0..=i64::MAX before admission and removes staging rows in the admitted delete transaction.
  • A complete Fold adapter would still own custom admission, staging inventories, cursor rules, replay state, publication state, and multi-key projections.

Decision audit: keep SQLite. The narrow existing BogKit correction remains sufficient: drop the underlying write transaction before resuming a user panic, then add Stream and KeyedStream rollback-plus-successful-later-write regressions. Do not infer a mailbox subsystem or new transaction API.

Skeptical review and synthesis

The reviewer independently reproduced all eight initial findings, verified the fixes, and completed a final byte-level archive audit. Final verdict: both APPROVED_FOR_ARCHIVE_AFTER_FIXES, with zero retained Critical, Important, or Minor archive issues.

Corpus totals are now 36 trials: 32 no-fit, four useful local proofs, and zero production adoptions. Operational/storage boundaries recur in 30 trials; component-selective setup in 15; nameable pipeline/reader patterns remain at seven. The two distinct confirmed BogKit defects remain caught-panic writer poisoning and persisted-value codec panic. No new subsystem or domain API candidate was promoted.

Full report: developer-simulation/reports/2026-08-14.md

Validation: the 46 new ordinary tests, full 36-package nested lab workspace, strict workspace lint, both runnable demos, and all 45 BogKit unit/documentation tests passed. All retained changes are under developer-simulation/; no build output, databases, generated runtime data, credentials, binaries, symlinks, or large files were committed.

urcades commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-15

Two blind developers worked from separate sanitized copies of current main at 80fd3c9. One evaluated a crash-safe calculated-state cache for a financial workbook engine; the other evaluated revision- and applicability-safe offline search for an aircraft-maintenance tablet. Both reached evidence-backed no-fit decisions. A separate skeptic rejected both initial handoffs, the original developers repaired the findings test-first, and the skeptic approved the normalized archive for publication.

Trial 1 — Incremental calculation cache

Persona: financial-planning document-core maintainer.
Outcome: no fit for the full brief; Fold was useful only as a bounded atomic publication primitive.
Archive: incremental calculation cache

The existing checksummed edit journal remained authoritative. The prototype used Fold to publish a complete calculated generation, but formula evaluation, dependency handling, error semantics, recovery, and application-reader snapshots remained application code.

Evidence:

  • 15 acceptance tests and one runnable demo passed, with strict formatting and strict pedantic lint.
  • Five clean logical builds and all five requested chunk groupings agreed.
  • Twelve reader tasks completed 24,000 reads with zero mixed generations.
  • One-way initialization and recovery-continuation gates passed.
  • The tiny closed store occupied 7,570 bytes; after 100 full-state publications it occupied 67,115,339 bytes. This is not a million-cell estimate.
  • The million-cell production oracle, 10,000-batch trace, process-exit campaign, corruption, disk-full, memory, long-concurrency, and latency gates remain unrun.

Friction and findings:

  • The all-components starter could not serve as a minimal offline Fold introduction because its unused ESE dependency entered model-download setup. The Fold-only package built offline.
  • The prototype identified affected cells but still fully recalculated and republished all cells.
  • The inspected API did not provide an independently clonable reader beside the exclusively owned writer, so the application published its own cloned immutable snapshot.
  • Skeptical review found repeated-initialization sequence rewind and restart divergence after a prior formula replacement. The repaired prototype makes initialization one-way and requires exact authoritative-journal reconstruction before resumed writes.
  • Those were prototype defects, not BogKit defects.

Decision audit: do not adopt the tested cache. The narrow proof supports clearer component-selective setup and storage/concurrency documentation, not a workbook API.

Trial 2 — Revision-safe aircraft-manual search

Persona: aircraft-maintenance tablet search maintainer.
Outcome: no fit; the tested global candidate surfaces cannot enforce the hard eligibility boundary.
Archive: revision-safe aircraft-manual search

The existing SQLite full-text baseline filters against separately validated revision and applicability metadata. The prototype used real Fold BM25 only to reproduce the decisive bounded-candidate failure; ESE and ANNy were not retained.

Evidence:

  • Ten acceptance tests and one runnable demo passed, with strict formatting and strict pedantic lint.
  • Twenty ineligible documents outranked the only eligible document.
  • Global top-20 followed by filtering returned zero results.
  • The complete 21-row toy ranking, filtered before the final top ten, returned the one eligible revision with exact evidence AMM-32|32-40-21|R-21|1700000000..1800000000.
  • The private 600-query judged set and the production generation, crash, memory, disk, latency, and sixteen-reader update gates were unavailable and remain unrun.

Friction and findings:

  • Fold BM25 accepts query plus limit, HNSW exposes a fixed top-K, and the inspected pipeline filter is fixed at ingestion rather than supplied per query. Bounded post-filtering can therefore lose the only eligible record.
  • ESE's clean build fetches model and tokenizer assets from mutable URLs without verified checksums.
  • Skeptical review found duplicate canonical document IDs were order-dependent and duplicate ranked IDs could emit duplicate evidence. The repaired boundary rejects both and preserves metadata-order independence.
  • Those were prototype defects, not BogKit defects.

Decision audit: retain the safety-first baseline. First document the global-candidate cutoff and hermetic-model boundary; only then evaluate a narrow allowed-key or query predicate. Do not infer a general search or generation subsystem.

Skeptical review and synthesis

Initial review found 0 Critical, 3 Important, and 1 Minor prototype/evidence issues. After one focused repair round, the same reviewer reproduced the fixes and completed a normalized-archive audit. Final verdict: both APPROVED_FOR_ARCHIVE_AFTER_FIXES, with 0 Critical, 0 Important, and 0 Minor findings remaining.

Corpus totals are now 38 trials: 34 no-fit, four useful local proofs, and zero production adoptions. Storage and concurrency boundaries recur in 32 trials; component-selective setup in 16; nameable pipeline/reader patterns remain at seven.

Two candidates crossed the evidence threshold today:

  1. Query-time filtered search — 2 independent sources. Document bounded post-filter loss, then evaluate a narrow allowed-key or predicate design.
  2. Hermetic ESE model assets — 2 independent sources. Document model identity, language scope, cache, and first-build behavior; pin immutable assets and verify checksums.

The confirmed BogKit defects remain caught Fold write-panic poisoning (two sources) and persisted-value codec panic (one source). Neither was encountered today, and no new defect was added.

Full report: developer-simulation/reports/2026-08-15.md

Validation: all 27 new tests, the full 38-package nested lab workspace, strict workspace lint, both runnable demos, and all 45 BogKit unit/documentation tests passed. All retained changes are under developer-simulation/; no build output, databases, generated runtime data, credentials, binaries, symlinks, or large files were committed.

urcades commented Aug 16, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-16

Two blind developers worked from separate sanitized copies of current main.
One evaluated causal history compaction for a collaborative canvas; the other
evaluated offline verification and durable advancement for cryptographic
transparency checkpoints. Both established the existing baseline first and
reached evidence-backed no_fit decisions.

A separate skeptical reviewer rejected both initial handoffs with three
Critical and five Important prototype or evidence findings. The original
developers repaired all eight findings test-first, and the same reviewer
approved both stable snapshots after a scoped re-review with zero Critical,
Important, or Minor findings remaining. No BogKit core or existing example
changed.

Trial 1 — Causal canvas-history compaction

  • Persona: collaborative-canvas synchronization maintainer with seven years of
    TypeScript and Go and nine months of Rust
  • Existing system: PostgreSQL operation authority plus a TypeScript reducer,
    periodic full snapshots, 30-day retained operations, and bespoke compaction
  • Problem: reduce history and tombstones without changing causal semantics,
    reconnect behavior, operation identity, or canonical JSON
  • Outcome: no_fit
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: Fold only for a development-time arrival-order
    characterization
  • Archive:
    runs/2026-08-16--causal-canvas-compaction

Evidence

  • Quality commands:
    • cargo fmt --manifest-path developer-simulation/Cargo.toml --package causal-canvas-compaction -- --check
    • cargo test --offline --locked --manifest-path developer-simulation/Cargo.toml --package causal-canvas-compaction --all-targets
    • cargo clippy --offline --locked --manifest-path developer-simulation/Cargo.toml --package causal-canvas-compaction --all-targets -- -D warnings -D clippy::all -D clippy::pedantic
  • Tests: 22 all-target tests pass. Permanent coverage includes causal
    buffering, all six literal operation kinds, duplicate and conflicting
    identity handling, tombstones, bounded input, deterministic batching,
    compaction/reopen, complete-manifest integrity, and the in-process
    publication returned-error model.
  • Reduced oracle: 1,000/1,000 scalar comparisons pass across 50 histories and
    20 schedules, with stable digest 5a36a712...e8141. The production
    TypeScript reducer and canonical fixtures were unavailable, so this is not
    production parity.
  • Representative release observation: 500,000 operations completed in 776 ms
    with 343 microseconds p95 for a 200-operation batch and 440,688 KiB sampled
    peak RSS. The required 5,000,000-operation run was not attempted because the
    architecture still retains accepted history in memory.
  • Largest reduced shape: 40,000 objects and 20,000 list elements reopened in
    69 ms, with 195,440 KiB sampled peak RSS and 12,529,568 compacted bytes.
  • Decisive gate: permanent operation fingerprints made the corrected retained
    artifact 3,719/8,670 bytes, or 42.90%. That fails the brief's at-most-40%
    requirement and strengthens the no-fit decision.

Friction trail

The developer began from the public README and examples, then inspected the
smallest Fold surface needed to test the hypothesis. Fold could atomically
store ordinary keyed views, but the application still had to decide causal
winners, preserve permanent operation identity, prove tombstone safety, and
publish complete generations beside PostgreSQL. ESE and ANNy were unrelated to
the exact replay workload.

The first stable prototype appeared to reject conflicting identity reuse and
validate generation data. Skeptical review showed that compaction discarded the
only fingerprint for an old operation ID, and that semantic manifest metadata
could be changed without touching the selected payload digests. The repair now
retains SHA-256 fingerprints for every accepted operation ID and binds every
canonical manifest byte with a separate SHA-256 file. The reviewer also found
that the initial crash and disk-full language described only ordinary returned
errors; the code and evidence now use that exact, narrower label.

Findings

  1. The production semantic oracle is absent. Category: documentation gap.
    Severity: Critical for adoption before and after review. Confidence: high.
    Reproduction: compare the acceptance rules in archived BRIEF.md with the
    explicit missing TypeScript reducer and fixtures in DISCOVERY.md and the
    package README. Smallest plausible improvement: supply an executable
    production oracle and literal fixtures for every operation and conflict
    boundary before another adoption evaluation.
  2. Fold does not remove causal winner selection or tombstone proofs.
    Category: poor product fit. Severity: Important before and after review.
    Confidence: high. Reproduction: run
    fold_keyed_materialization_is_arrival_ordered_without_a_causal_wrapper in
    tests/fold_comparison.rs. Smallest plausible improvement: keep causality
    application-owned; reconsider only a non-authoritative downstream view.
  3. Failure-aware consumers cannot handle several Fold storage and decode
    boundaries as typed errors.
    Category: API friction. Severity: Important
    before and after review. Confidence: high. Reproduction: inspect
    fold/src/stream/unkeyed.rs, fold/src/stream/mod.rs,
    fold/src/stream/keyed.rs, and
    fold/src/pipeline/terminal/table.rs for the open, commit, checkpoint,
    keyspace, read, and decode unwrap paths, then run the package's Fold
    characterization test. Smallest plausible
    improvement: add documented fallible variants for existing open, keyspace,
    read, decode, commit, and checkpoint boundaries while retaining convenience
    wrappers where useful.
  4. Compaction initially permitted conflicting operation-ID reuse. Category:
    correctness defect. Severity: Critical before repair; none after repair.
    Confidence: high. Reproduction: run
    conflicting_operation_id_reuse_remains_rejected_after_compact_reopen in
    tests/compaction.rs; it proves exact rejection and unchanged visible
    bytes, digest, and counter through compact, publish, and reopen. Smallest
    plausible improvement: none; permanent canonical fingerprints now close the
    prototype defect and are counted in the failing byte gate.
  5. Manifest semantic metadata was initially unauthenticated. Category:
    correctness defect. Severity: Important before repair; none after repair.
    Confidence: high. Reproduction: run
    changed_manifest_metadata_falls_back_or_fails in tests/publication.rs.
    Smallest plausible improvement: none; a separately stored digest now binds
    all canonical manifest bytes before any field is trusted.
  6. The initial publication-fault labels exceeded the evidence. Category:
    documentation gap. Severity: Important before repair; none after repair.
    Confidence: high. Reproduction: run tests/publication.rs and inspect the
    package README's “returned-error test is not a crash test” section. Smallest
    plausible improvement: none; the archive now says exactly that it models an
    ordinary in-process returned error and only parses crash-schedule JSON.
  7. The retained-history design breaks the scale envelope. Category:
    performance problem. Severity: Critical before and after review. Confidence:
    high. Reproduction: run the release workload 500000 command and
    compaction_counts_permanent_identity_metadata_in_retained_byte_gate.
    Smallest plausible improvement: stream operations and externalize retained
    history/fingerprints, then measure the full five-million-operation recipe
    against the real oracle before reconsidering adoption.

Decision audit

The baseline remains authoritative because only it has the actual production
merge semantics. Fold would add a second durable store without replacing
causal winner selection, safe collection, bounded ingress, or generation
publication. The prototype's repaired identity guarantee also causes the
retained-byte gate to fail. ESE and ANNy are exact workload no-fits. The next
responsible step is shadow replay against recorded histories and the real
TypeScript oracle, not a production read-path change.

Trial 2 — Transparency-checkpoint verifier

  • Persona: certificate-transparency monitor maintainer with eight years of Go
    and security operations and four months of Rust
  • Existing system: a trusted Go verifier and SQLite transaction that advance a
    signed checkpoint, raw proof, archive cursor, and decision together
  • Problem: make restart, proof diagnostics, and adversarial recovery easier to
    test without weakening fail-closed cryptographic behavior
  • Outcome: no_fit
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-16--transparency-checkpoint-verifier

Evidence

  • Quality commands:
    • cargo fmt --manifest-path developer-simulation/Cargo.toml --package transparency-checkpoint-verifier -- --check
    • cargo test --offline --locked --manifest-path developer-simulation/Cargo.toml --package transparency-checkpoint-verifier --all-targets
    • cargo clippy --offline --locked --manifest-path developer-simulation/Cargo.toml --package transparency-checkpoint-verifier --all-targets -- -D warnings -D clippy::all -D clippy::pedantic
  • Tests: 27 all-target tests pass. Coverage includes bounded parsing, RFC 8032
    Ed25519, RFC 6962-style Merkle known answers and bit flips, cursor rollback,
    deterministic pending resolution, duplicate/equivocation handling, corrupt
    generation reuse, checked snapshot resume, and generic returned errors after
    nine completed publication stages.
  • Independent review: the reviewer also checked all 2,080 non-empty old/new
    size pairs through 64 leaves against an independent Merkle oracle.
  • Synthetic release observation: 8,000,000 generated leaf hashes completed in
    895 ms with root 6a0bc348...be0c and 1,818,624-byte peak RSS. This does no
    decompression, production archive parsing, signature corpus, or SQLite
    recovery and is not comparable with the baseline's 45-minute restart.
  • Reduced verifier path: 10,000 parse/signature/proof/advance iterations had
    21-microsecond p95 after warm-up. Durable publication is excluded.
  • Checked reopen: eight empty logs reopened in a reported 0 ms. The production
    4,000-checkpoint corpus and Go/SQLite state were unavailable.

Friction trail

The developer identified exact auditability, recoverable storage errors, and
cursor/proof atomicity as the decisive boundaries. Fold's transactions were
plausible, but several public convenience paths panic on storage or decode
failures and the component does not expose the fault surface required by this
security evaluation. ESE and ANNy were unrelated to cryptographic proof.

The first stable state machine passed its supplied crypto and publication
tests, but the reviewer found five lifecycle gaps: cursor rollback was accepted;
a corrupt pre-existing generation could be republished as success; stale
same-base candidates could remain pending and make bytes delivery-order
dependent; reopened state could not resume verification; and three named
failure modes were one generic returned-error model. Test-first repairs close
all five while preserving the no-fit decision and explicit production gaps.

Findings

  1. Production Go, SQLite, wire-format, key, and corpus parity is absent.
    Category: poor product fit. Severity: Critical for adoption before and after
    review. Confidence: high. Reproduction: compare archived BRIEF.md with the
    missing-artifact inventory in DISCOVERY.md and the package README's
    evidence boundaries. Smallest plausible improvement: supply read-only
    production fixtures and a Go decision-ledger export, then run shadow parity.
  2. Fold's fallible storage and decode boundary is missing for this consumer.
    Category: API friction. Severity: Important before and after review.
    Confidence: high. Reproduction: inspect fold/src/stream/unkeyed.rs,
    fold/src/stream/mod.rs, fold/src/stream/keyed.rs, and
    fold/src/pipeline/terminal/table.rs; their open, keyspace, read/decode,
    commit, and checkpoint convenience paths use unwrap rather than typed
    errors. Smallest plausible
    improvement: add documented fallible variants for those existing boundaries
    without creating a new security storage subsystem.
  3. Archive-cursor rollback initially advanced visible state. Category:
    correctness defect. Severity: Critical before repair; none after repair.
    Confidence: high. Reproduction: run
    archive_cursor_rollback_is_rejected_without_mutation in tests/engine.rs;
    it proves the full snapshot unchanged and a later valid advance succeeds.
    Smallest plausible improvement: none; the prototype now rejects rollback
    before mutation, while cursor-to-leaf correspondence remains unimplemented.
  4. Republishing initially trusted a corrupt existing generation by name.
    Category: correctness defect. Severity: Critical before repair; none after
    repair. Confidence: high. Reproduction: run
    republish_rejects_corrupt_existing_generation_and_preserves_current in
    tests/store.rs. Smallest plausible improvement: none; reuse now validates
    the exact expected files, digests, and payload bytes before pointer mutation.
  5. Pending candidates were initially stale and order-dependent. Category:
    correctness defect. Severity: Important before repair; none after repair.
    Confidence: high. Reproduction: run
    same_base_candidates_are_all_decided_after_advance and
    equal_key_pending_delivery_orders_have_identical_bytes in
    tests/engine.rs. Smallest plausible improvement: none; canonical envelope
    identity now completes the order and every stale candidate is decided.
  6. Reopened state initially could not resume verification. Category:
    missing capability. Severity: Important before repair; none after repair.
    Confidence: high. Reproduction: run
    reopened_engine_continues_and_drains_pending_without_divergence in
    tests/store.rs. Smallest plausible improvement: none; checked snapshot
    reconstruction now validates retained state and matches uninterrupted
    continuation, with historical-proof limits disclosed.
  7. Named process-exit, disk-full, and permission claims were one model.
    Category: documentation gap. Severity: Important before repair; none after
    repair. Confidence: high. Reproduction: inspect the package README's generic
    completed-stage returned-error section and run the five store tests.
    Smallest plausible improvement: none; the archive now excludes child kill,
    partial write, real OS errors, and power loss instead of naming them as
    tested.

Decision audit

The Go/SQLite baseline wins by default because the actual implementation and
corpus were absent. The Rust package is a useful characterization of bounded
parsing, Ed25519, small-tree consistency, deterministic decisions, and checked
publication, but it cannot establish production format, key, archive, recovery,
or migration parity. Fold's panic-oriented convenience boundary is unsuitable
for the required fail-closed error handling; ESE and ANNy do not address exact
cryptographic verification. No component should own advancement from this
evidence.

Skeptical review

  • Claims reproduced: all 49 retained trial tests, both strict lint/format
    gates, both release builds, both reduced demos, the 1,000-schedule scalar
    oracle, the 2,080-pair small-tree oracle, and all eight permanent fix-round
    regressions.
  • Claims rejected or softened: production semantic/parity claims, the full
    five-million-operation and production-corpus gates, process-crash and named
    I/O-failure claims, and comparisons between reduced synthetic timings and
    either production baseline.
  • Initial findings: three Critical and five Important prototype/evidence
    defects. All were repaired by the original simulators and closed by the same
    reviewer. Remaining findings: zero Critical, zero Important, zero Minor.
  • Unnecessary dependencies: neither trial retained ESE or ANNy; Trial 1 keeps
    Fold only as a development-time characterization, and Trial 2 uses no BogKit
    dependency.
  • Remaining uncertainty: actual production reducers, formats, keys, corpora,
    operating systems, process exits, I/O failures, power loss, and migration
    paths remain untested.

Cross-run synthesis

  • New evidence: both trials independently found that failure-aware consumers
    cannot handle several Fold storage and decode failures as typed errors. This
    promotes fold-fallible-storage-and-decode-apis at two sources.
  • Recurring evidence: both trials retain an external authority and require
    application-owned causal, cryptographic, and publication semantics. The
    storage/concurrency boundary theme rises from 32 to 34 sources.
  • Confirmed defects: unchanged. Caught Fold write-transaction panic poisoning
    remains at two sources; the persisted-value codec panic remains at one.
  • Candidate improvement: add documented fallible variants for existing Fold
    open, keyspace, read/decode, commit, and checkpoint boundaries. Keep this
    separate from the one-source codec correctness defect.
  • Observations not promoted: the trials requested fault injection at different
    layers, so no shared fault-injection feature is inferred. They do not justify
    a generation manager, canvas API, cryptographic subsystem, archive codec, or
    SQLite replacement.
  • Positioning signal: BogKit remains strongest where embedded materialization,
    text embeddings, or approximate search remove meaningful application work.
    It is a poor fit when the application must retain an external authority and
    still own exact causal or cryptographic semantics plus publication recovery.

Validation

  • Trial-specific tests: 22 causal-canvas tests and 27 transparency-verifier
    tests passed from the normalized nested workspace.
  • Strict lint and formatting: both new packages passed formatting and strict
    pedantic Clippy; the full nested workspace passed warnings-denied Clippy.
    A stronger full-workspace pedantic pass is not claimed because Rust 1.95
    reports pre-existing pedantic lint drift in older archived packages.
  • Runnable demonstrations: both reduced release demos reproduced the retained
    deterministic outputs and honestly scoped measurements.
  • BogKit root workspace: all 45 unit and documentation tests passed against
    merged main commit 98875446c6b9f6ae242666978e8699751935d64c.
  • Archive checks: one nested lockfile, unique package names, no child workspace
    or lockfile, all changed paths under developer-simulation/, and no retained
    secrets, databases, build output, runtime state, symlinks, or large binaries.

@samepant samepant closed this Aug 16, 2026
@urcades urcades reopened this Aug 17, 2026

urcades commented Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-17

Two blind developers worked from separate sanitized copies of current main.
One evaluated corruption recovery for a binary instrument stream; the other
evaluated mergeable approximate reach rollups. Both established their existing
baseline first, considered Fold, ESE, and ANNy, used none, and reached
evidence-backed no_fit decisions.

A separate skeptical reviewer rejected both initial handoffs with five
Important prototype or report findings. The original developers repaired or
honestly narrowed all five test-first. The first repair to Trial 1 introduced
one further Important correctness defect; the same reviewer reproduced it, the
developer repaired it, and a second scoped review closed it. Both final
handoffs were approved with zero Critical, Important, or Minor reviewer
findings remaining. No BogKit core or existing example changed.

Trial 1 — Corruption-resynchronizing instrument stream decoder

  • Persona: laboratory gateway developer with nine years of C and C++, the Rust
    Book, and two small Rust command-line tools
  • Existing system: one append buffer per connection, then a scan for framed
    records or a connection-wide reset after corruption
  • Problem: decode exact marker/length/CRC frames from arbitrary chunks, keep
    memory bounded, and recover valid followers without leaking payload bytes
  • Outcome: no_fit
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-17--corruption-resynchronizing-instrument-stream-decoder

Evidence

  • Quality commands:
    • cargo fmt --manifest-path developer-simulation/Cargo.toml --package instrument-decoder-trial -- --check
    • cargo test --locked --manifest-path developer-simulation/Cargo.toml --package instrument-decoder-trial --all-targets
    • cargo clippy --locked --manifest-path developer-simulation/Cargo.toml --package instrument-decoder-trial --all-targets --all-features -- -D warnings -D clippy::all -D clippy::pedantic
  • Tests: 19 retained tests pass. They cover the CRC known answer, exact wire
    layout, all single-byte split points, bounded over-limit rejection, marker
    and CRC corruption, the modeled baseline, deterministic schedules,
    arbitrary-byte stress, and both sides of the ambiguous nested-frame prefix.
  • Full clean run: whole-frame, one-byte, and irregular schedules each emitted
    1,000,000/1,000,000 frames over 387,200,000 bytes with tuple digest
    0x227BAFFB59534865, trace digest 0xED1C4875E1055725, and zero diagnostics.
  • Decisive damaged run: all schedules processed 25,000 corruptions, emitted no
    damaged frame, and recorded only 48,942/50,000 followers by their own frame
    end. This is an explicit failure of the brief's immediate-recovery gate, not
    an independently counted total of followers that never appear later.
  • Boundedness and performance: the 100,000-case arbitrary-byte run retained at
    most 8,175 bytes and buffered at most 8,192; fresh peak RSS was 6,144,000
    bytes. In the fixed-chunk parser comparison, the slower baseline was 126.388
    MiB/s and the candidate 135.002 MiB/s. These host observations do not offset
    the failed correctness requirement.

Friction trail

The developer began with the root README and public examples. Fold's durable
stream and transaction model was unrelated to a synchronous no-disk framing
path; ESE and ANNy were also exact workload no-fits. A standalone parser first
passed clean and boundedness checks but waited behind a plausible, incomplete
header. The first repair scanned inside that candidate and emitted a later
CRC-valid sequence immediately. Skeptical review then supplied a clean outer
frame whose opaque payload contained an entire valid-looking inner frame: the
one-byte schedule emitted the inner sequence and lost the outer frame.

The final conservative implementation never scans inside an incomplete
accepted-length candidate. It preserves the clean outer frame, but the exact
02 00 13 false-header case cannot emit its first valid follower immediately.
The two byte prefixes are indistinguishable until later bytes arrive, so the
prototype and all three BogKit components are correctly classified no_fit
against the complete brief.

Findings

  1. The wire format cannot guarantee both opaque payloads and unconditional
    immediate recovery.
    Category: missing capability. Severity: Important
    before and after review. Confidence: high. Reproduction: run
    candidate_never_emits_valid_nested_payload_before_outer_is_proved and
    candidate_defers_nested_followers_until_plausible_header_fails, then read
    the incompatible prefixes in archived TRIAL_REPORT.md. Smallest plausible
    improvement: add an unambiguous boundary such as escaping or trusted record
    segmentation, or relax recovery until the containing candidate resolves.
  2. Opportunistic nested emission initially corrupted a clean opaque frame.
    Category: correctness defect. Severity: Important before repair; none after
    repair. Confidence: high. Reproduction: run
    candidate_never_emits_valid_nested_payload_before_outer_is_proved; whole
    and one-byte schedules now emit only outer sequence 42. Smallest plausible
    improvement: none; the decoder now waits for the containing frame and keeps
    the permanent regression.
  3. The modeled connection reset loses already-buffered valid data. Category:
    correctness defect. Severity: Important before and after review. Confidence:
    high. Reproduction: run the release demo described in README.md; the
    modeled baseline clears a damaged frame and its valid follower. Smallest
    plausible improvement: replace connection-wide reset with incremental
    validation and an explicit conservative recovery boundary.
  4. BogKit does not simplify synchronous exact framing. Category: poor
    product fit. Severity: Important before and after review. Confidence: high.
    Reproduction: compare the brief's no-disk byte path with Fold's persistent
    stream, ESE's text embeddings, and ANNy's approximate vector search in
    DISCOVERY.md. Smallest plausible improvement: keep framing standalone and
    consider Fold only for an optional downstream stream of validated,
    privacy-safe diagnostics.
  5. The root onboarding leaves the valid use-none path implicit. Category:
    documentation gap. Severity: Minor before and after review. Confidence:
    high. Reproduction: start from the root README with the archived binary
    framing brief and compare the component descriptions with DISCOVERY.md.
    Smallest plausible improvement: extend the existing capability-matrix
    candidate with explicit exact synchronous protocol and no-fit examples.

Decision audit

The conservative parser was chosen over opportunistic resynchronization because
clean opaque payload semantics are provable while early nested emission is not.
The append/search reset remains only a modeled baseline; production gateway
code and instrument captures were unavailable. The full synthetic and property
runs establish the stated generated cases and memory bounds, not production
readiness. The prototype is retained as an executable incompatibility and
recovery reproducer, not as an adoption candidate.

Trial 2 — Mergeable unique-installation reach rollups

  • Persona: analytics-infrastructure engineer at a business-software company
    with eight years of Kotlin and SQL and four months of Rust
  • Existing system: exact per-bucket hash sets merged centrally for an immutable
    daily report
  • Problem: estimate unique installations across eight shards with bounded,
    portable state and measurable accuracy, determinism, memory, and runtime
  • Outcome: no_fit
  • BogKit components considered: Fold, ESE, and ANNy
  • BogKit components used: none
  • Archive:
    runs/2026-08-17--mergeable-unique-installation-reach-rollups

Evidence

  • Quality commands:
    • cargo fmt --manifest-path developer-simulation/Cargo.toml --package reach-rollup-lab -- --check
    • cargo test --locked --manifest-path developer-simulation/Cargo.toml --package reach-rollup-lab --release --all-targets --all-features
    • cargo clippy --locked --manifest-path developer-simulation/Cargo.toml --package reach-rollup-lab --all-targets --all-features -- -D warnings -D clippy::all -D clippy::pedantic
  • Tests: 21 tests pass and one benchmark wrapper remains intentionally ignored
    in the ordinary suite. Coverage includes exact-oracle comparison, 50 merge
    orders, duplicates and self-merge, complete state-file integrity, malformed
    input, safe report publication, and deterministic output.
  • Accuracy: the disclosed tuning matrix recorded 0.858% median, 2.671% p95,
    and 5.626% worst absolute error. A fixed held-out seed not used for further
    tuning recorded 0.858%, 2.708%, and 6.124%.
  • Full load: 12,000,000 records produced 10,800,000 exact unique
    bucket-occurrences and 5,760 canonical rows. Direct aggregation, eight-shard
    aggregation, 50 merge orders, duplicate replay, and repeated state/report
    bytes agree; corrected state digest is ecda69b5ba1467d1.
  • Normalized-workspace host comparison: after one earlier warm-up, three exact
    runs took 1.03, 1.09, and 1.06 seconds, while three candidate runs took 0.54,
    0.53, and 0.53 seconds. A separate current-profile measurement recorded
    49,594,368-byte candidate peak RSS versus 645,136,384 bytes for exact, more
    than 13 times lower. Both paths were single-threaded, but the host had 14
    cores and the 128 MiB ceiling was measured rather than container-enforced.

Friction trail

The exact hash-set baseline established correctness first. A 4,096-register,
4,124-byte HyperLogLog-style state then passed merge, state-size, and
determinism tests. The requested tuning matrix narrowly failed p95 before the
estimator transition was changed; that matrix is therefore labeled tuning
evidence, and a separate fixed seed is retained as held-out evidence.

The first state-file format checksummed each nested sketch but did not bind the
outer tenant/hour key. Skeptical review changed one valid tenant or hour into
another and the decoder accepted the reassigned state. Version 2 now checksums
the complete header, count, every key, and every nested state before
constructing a rollup. The reviewer also corrected the component analysis:
Fold's positive-only aggregate can express local HLL updates, but the public
surface does not demonstrate the required portable external partial-state merge
and adds persistence and transactions that this immutable batch does not need.

Findings

  1. The first state format did not bind keys to states. Category:
    correctness defect. Severity: Important before repair; none after repair.
    Confidence: high. Reproduction: run the valid-window tenant and hour
    mutation regressions in tests/state_file.rs; version 2 rejects both before
    returning a rollup. Smallest plausible improvement: none; the versioned
    complete-file checksum and permanent regressions close the prototype defect.
  2. BogKit does not demonstrate the required portable external shard merge.
    Category: poor product fit. Severity: Important before and after review.
    Confidence: high. Reproduction: run the eight-shard and 50-order tests, then
    compare their portable register-wise merge boundary with the public Fold
    entry points described in DISCOVERY.md. Smallest plausible improvement:
    keep this proof standalone; do not infer a new merge subsystem from one
    trial.
  3. Public onboarding does not state external partial-state merge support.
    Category: documentation gap. Severity: Important before and after review.
    Confidence: high. Reproduction: start from the root README and public
    starter/timeseries examples, then compare them with the exact external merge
    exercised by tests/load_contract.rs. Smallest plausible improvement:
    extend the existing capability matrix to state append-only aggregation,
    retraction, portable serialization, external merge, and state-bound support.

Decision audit

The standalone sketch is acceptable only for the brief's advisory immutable
daily report; it is not exact and must not be used for billing, access, fraud,
quotas, privacy, or contracts. Fold was rejected because portable shard-state
exchange remains application-owned and the batch proof needs no durable store.
The held-out run uses a different fixed seed from the same generator family,
not a production distribution or independently preregistered study. OS-level
power loss, real filesystem errors, skewed production identifiers, and the
target four-core container remain untested.

Skeptical review

  • Claims reproduced: all 19 Trial 1 tests, 21 Trial 2 tests, two independent
    parser probes, valid-range state-key mutation probes, both strict lint and
    formatting gates, full clean/damaged decoder evidence, the 12-million-record
    load, accuracy files, deterministic merge orders, host RSS, and both
    comparisons.
  • Claims rejected or softened: universal immediate follower recovery,
    opportunistic scanning inside opaque payload, the initial unbound state-file
    checksum, the claim that Fold cannot express positive-only HLL updates, and
    outcome/finding labels that exceeded the policy.
  • Initial findings: zero Critical, five Important, and zero Minor. The first
    Trial 1 repair introduced one further Important correctness defect. All six
    were repaired or explicitly converted into the honest no-fit boundary and
    closed by the same reviewer. Remaining reviewer findings: zero Critical,
    zero Important, zero Minor.
  • Unnecessary dependencies: neither trial retains Fold, ESE, ANNy, or any
    third-party dependency.
  • Remaining uncertainty: real instrument traffic, production gateway behavior,
    independent statistical distributions, target containers, enforced memory
    caps, and power-loss durability remain untested.

Cross-run synthesis

  • New evidence: no new BogKit correctness defect or candidate crossed the
    promotion threshold.
  • Recurring evidence: both trials are exact bounded transformations whose key
    semantics remain outside durable materialization. The existing
    storage/concurrency-boundary theme rises from 34 to 36 sources and now
    includes exact binary framing and externally mergeable immutable rollups.
  • Candidate improvements: the existing capability-matrix candidate should
    state protocol hot-path, portable accumulator, and explicit no-fit
    boundaries. No new decoder, HLL, external-merge, Fold-batching, publisher,
    CRC, or generic use-none subsystem is justified.
  • Observations not promoted: a positive-only Fold aggregate can represent the
    local HLL update, but external state exchange and transaction sizing are
    one-source observations. The decoder's wire ambiguity is protocol-specific.
  • Positioning signal: BogKit is useful when embedded materialization, text
    embedding, or approximate search removes meaningful application work. It is
    a poor fit for exact synchronous parsing or a small immutable batch whose
    decisive portable merge stays application-owned.

Validation

  • Trial-specific tests: 19 decoder tests and 21 rollup tests passed from the
    normalized nested workspace; the rollup benchmark wrapper remains explicitly
    ignored in the ordinary suite and was run separately.
  • Strict lint and formatting: both new packages passed formatting and strict
    pedantic Clippy. The full nested workspace passed warnings-denied Clippy.
  • Runnable demonstrations: the decoder demo/property/benchmark and the rollup
    demo/load/comparison reproduced their retained deterministic evidence.
  • BogKit root workspace: all 45 unit and documentation tests passed against
    merged current-main commit 20f2ca50d5d06f51edfe8b8570c0fb48caf9eb81.
  • Archive checks: one nested lockfile, 42 unique package names, no child
    workspace or lockfile, all changed paths under developer-simulation/, and
    no retained secrets, databases, build output, runtime state, symlinks, or
    large binaries.

urcades commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Developer simulation — 2026-08-18

Two blind developers evaluated exact, immutable batch boundaries from separate sanitized copies of current main. Both considered Fold, ESE, and ANNy, used none, and reached reviewed no_fit conclusions. No BogKit core or existing example changed.

Trial 1 — Exact laboratory-unit conversion admission gate

Persona: healthcare-interface developer with eight years of Java and SQL and four months of Rust.
Outcome: no_fit.
Archive: laboratory-unit-conversion-gate

The prototype validates immutable reference tables, parses exact decimals, applies rational affine conversions with one round-half-to-even step, rejects unsafe rows, and publishes a canonical advisory report.

Evidence:

  • 20 retained tests passed, including 20,000 comparisons with an independent slow arithmetic reference, bounded input, duplicate and alias identities, deterministic shuffles, and publication lifecycle checks.
  • Five full million-row shuffles and three repeats each produced 975,000 conversions, 25,000 rejections, a 93,065,000-byte report, and digest b9c8adf790c0717818cdaaeb83d5bbb612aa3780eb60e4eba3d97732c1446eb5.
  • The skeptic's fresh macOS run took 1.17 seconds and 290,521,088 bytes maximum resident memory. That is 22,085,632 bytes above the 256 MiB gate, so the resource gate fails on this host.
  • A normalized 10,000-row demo reproduced 9,750 conversions, 250 rejections, and digest 256be30a4b0845bb13d4e999587042cdd60f9c6512f2ffc1e8da8a54379b71a0.
  • The machine report now separates completed execution from acceptance and never reports an acceptance pass without the unavailable signed fixtures, Java comparison, and declared Linux environment.

Decision audit: keep the converter standalone. Fold's durable incremental state does not remove exact decimal grammar, checked arithmetic, reference validation, canonical sorting, or file publication; ESE and ANNy are unrelated. The retained in-memory proof also fails its memory gate, strengthening the no-fit result.

Trial 2 — Deterministic freight-capacity batch clearing

Persona: freight-marketplace backend developer with six years of Ruby and PostgreSQL and five months of Rust.
Outcome: no_fit.
Archive: freight-capacity-batch-clearing

The standalone matcher validates an immutable snapshot, applies exact seller-ask price-time priority, conserves integer quantities and checked gross value, ignores input order, and publishes one canonical advisory proposal.

Evidence:

  • 33 retained integration tests passed, including 40 literal golden markets, 10,000 fixed-seed markets checked fill-for-fill against an independent scanner, bounded input, integrity fields, alias identities, publication states, and the real CLI.
  • Ten full 650,000-order shuffles and three repeats produced 498,521 fills, gross value 312220517981477594, an 80,960,196-byte proposal, and digest e7ce11984ca3e04197df9dd43013361b123e5ecf4b877f3a36f0a08dd64ebcad.
  • The skeptic's fresh macOS run took 0.97 seconds and 481,558,528 bytes maximum resident memory, below 512 MiB on this host but not a declared two-core Linux or Ruby comparison.
  • A normalized 6,500-order demo produced 4,967 fills, gross value 3075163243940483, an 806,647-byte proposal, and digest d24a6f1898c9ea72b069ff4a3bf2e6a0d3d8de290449609d6e05377db1740cd3.

Decision audit: keep the matcher standalone. Fold would add persistent lifecycle to a one-shot exact batch while leaving validation, sorting, matching, checked arithmetic, and publication custom; ESE and ANNy are unrelated.

Skeptical review and synthesis

The initial reviewer rejected each handoff with four Important and one Minor prototype or report finding. The original developers repaired all ten findings test-first:

  • both tools now reject direct, resolved, hard-link, and relevant temporary output aliases before publication;
  • relative output publication retains the parent directory handle and distinguishes pre-rename nonpublication from post-rename durability uncertainty;
  • Trial 1 no longer false-passes failed or absent acceptance evidence and retains one meaningful negative fixture instead of sixty duplicates;
  • Trial 2 bounds overlong-line consumption at 4,097 bytes and requires both manifest integrity fields.

Scoped re-review approved both repaired snapshots. A separate normalized staged-archive audit reported zero remaining Critical, Important, or Minor discrepancies.

No new BogKit defect or candidate met the promotion threshold. These two no-fit trials raise the existing storage/concurrency/authority-boundary theme from 36 to 38 independent sources; no other recurring count changes.

Full report: developer-simulation/reports/2026-08-18.md

Validation: 20 + 33 trial tests passed; both packages passed formatting and strict pedantic lint; the complete 44-package nested workspace and warnings-denied lint passed; all 45 BogKit unit/documentation tests passed against current-main commit 20f2ca50d5d06f51edfe8b8570c0fb48caf9eb81. The archive contains one nested lockfile and no generated build output, databases, runtime state, symlinks, secrets, or large binaries.

@urcades urcades closed this Aug 18, 2026
@urcades
urcades deleted the ed/developer-simulation branch August 18, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants