docs(engine): state and pin the concurrent-use guarantee - #332
Open
OmarAlJarrah wants to merge 3 commits into
Open
docs(engine): state and pin the concurrent-use guarantee#332OmarAlJarrah wants to merge 3 commits into
OmarAlJarrah wants to merge 3 commits into
Conversation
…cy-guarantee # Conflicts: # compilers/compilers.go # engine/engine.go
A worker that stopped partway through left zero values in its results, and the comparison reported those as documents compiled differently — which sends a reader looking for a corrupted lowering when nothing lowered at all. Planting an early break made it print that for 74 of 77 specs. The loop now names the case and says so. The length check above it stays: it cannot fail while runCorpus returns either an error or a full-length slice, and it is what keeps the indexing below from panicking should that ever change. Its message no longer claims to be about finishing the corpus, which is the check that was just added. The Engine doc said the race property is pinned "only under -race" and left the reader to wonder whether anything runs it. #344 landed -race in the coverage step, so it does.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
One
engine.Enginecan serveRuncalls from several goroutines at once, but nothing said soand nothing pinned it. An embedder had no way to tell, so the safe move was to over-construct —
an engine per request — and a later change that parked a cache or a scratch buffer on the
Enginewould have broken the property with a green build.Rather than take the "pure, reentrant stages" rule on trust, the guarantee was checked before
being written down. Every package-level
varreachable fromRun— 27 of them across the 22packages in
go list -deps ./engine— is a declaration-only table: keyword sets,reflect.Typevalues, sentinel errors. None is assigned, indexed into, or appended to anywhere after package
init.
The shared value matters as much as the shared package.
engine.Newbuilds one*openapi.Compilerand every goroutine dispatches through it, so a field on that struct wouldbreak the guarantee just as surely as a package var.
type Compiler struct{}— there is nothingon it to race on.
The codebase does use package-level function variables as test seams —
newEngine,createOutput,renameOutput,harness.compile,reserializeJSON,encodeYAML— and thoseare genuinely mutable package state; a test swapping one while another goroutine compiled would
race. But all of them live in
cmd/morphic,cmd/morphic-harness,internal/harnessandir/irtest, none of which appears ingo list -deps ./engine. They do not narrow the guarantee.So the guarantee is stated as broadly as it is true, and the one place it is genuinely narrower
is called out rather than papered over:
compilers.Registry.Lookupis safe concurrently onlyonce registration has finished, because
Registerwrites an unsynchronized map.engine.NewWithsatisfies that by construction — it fills a fresh registry and only then wraps it — but an
embedder registering a compiler after starting goroutines would not.
Doc comments on
engine.Engine,engine.Engine.Run,compilers.Compiler,compilers.Registryand
pass.Validate, plus one new test. No behaviour change: strip the comment lines and the threesource files are identical to what they replace.
Test plan
engine/concurrency_test.godrives the whole 77-spec conformance corpus through a single shared*engine.Enginefrom 8 goroutines, and requires every document to be byte-identical to a baselinecompiled with a fresh
Engineper spec.Two choices in it are load-bearing. It does not build an
Engineper goroutine — that sharesnothing and proves nothing. And it does not stop at "no race reported": a data race is only one
way concurrency corrupts output, so the documents are compared, which also re-checks the
determinism invariant. The baseline uses unshared engines on purpose; a baseline drawn from the
shared engine would carry the same corruption as the concurrent runs and cancel it out.
Clean run:
Whole repo, race detector on:
go test ./... -race -count=1exits 0 with zeroWARNING: DATA RACEoccurrences. The gate runs that build —make coveragepasses-race— sothe first of the two properties is checked on every CI run rather than only by hand. The slowest
package under the detector is
internal/harnessat 8.6 s, against the coverage script's-timeout 300s.Proof the test bites
Four deliberate defects, each planted, observed, and reverted.
1. Per-instance state on the
Engine— alastFormatfield assigned inRun, the shape amemoizing
Enginewould take. The new test fails under-race:Every other test in the package passes with this planted, because each builds its own
Engine;sharing one is what catches it. The package is also
okwithout-race, which is what makesthe detector step load-bearing rather than decorative.
2. A race-free wrong answer — a mutex-guarded memo on the
Engine, keyed on source formatinstead of on the spec, handing back a deep copy so no pointer is shared. Zero
DATA RACEreports, and the test still fails, on the comparison:
This is the case a race-only test would wave through, and the reason the baseline is built from
unshared engines.
3. A degenerate baseline — every baseline entry compiled from the same spec, to confirm the
anti-vacuity guard is not decorative:
4. A worker that stops partway through the corpus. This one reported the wrong thing: the
specs it never reached hold zero values, which the diff called "compiled differently". The
comparison loop now names that case for what it is, and says so:
The audit
The package-var claim is the load-bearing one, so it was made by walking the AST rather than by
grepping: index, selector and star writes are resolved to their root identifier, and locals that
shadow a package var are excluded. It was then held to the standard it exists to enforce.
Planting a map-element write (
formatTable["x"] = "y") and a sentinel reassignment(
errOptionChannels = nil) each produce exactly one finding, and a local shadowing a package varproduces none. A grep-based version of the same audit reported the same clean result and missed
both plants, which is why it is not the one the claim rests on.
Gate
make gateexits 0: gofmt, vet, lint, both nolint checks, build, the coverage counter'sverifier, coverage at 100%, fuzz and benchmarks.
Closes #82