Skip to content

Source-derived dialect census with anti-drift guards (Epic 19 / R2a function-library audit) - #249

Open
fupelaqu wants to merge 1 commit into
mainfrom
feature/19.1
Open

Source-derived dialect census with anti-drift guards (Epic 19 / R2a function-library audit)#249
fupelaqu wants to merge 1 commit into
mainfrom
feature/19.1

Conversation

@fupelaqu

Copy link
Copy Markdown
Contributor

Summary

Story 19.1 of the R2a function-library audit: one machine-generated, drift-proof inventory of every construct the elasticsql SQL dialect accepts, so Epic 22's jOOQ-render estimate rests on a counted surface that cannot rot between the audit and the implementation that consumes it.

Three test-tree sources under sql/src/test/scala/app/softnetwork/elastic/sql/census/ (no production or published code changes; test artifacts are not published):

  • CensusModel.scala — row model (CensusEntry, Kind, EsConstruct, Standard with shape-checked Rule-R evidence gates) + emitters (RFC-4180 CSV with every cell quoted, provenance-stamped JSON object {"provenance", "rows"}, GFM markdown with backslash-first | escaping).
  • DialectCensus.scala — 259 hand-authored SYNTAX-FORM rows: 171 function rows covering all 95 SQLKeywords.functionTokens and all 149 accepted spellings (54 aliases, 22 computed at runtime by TimeField.words — 9 exist in no source literal, which is why this census RUNS instead of scraping), 29 operators, 36 clauses, 6 literals, 17 units. Every ansi_adjacent citation fetched from PostgreSQL 16 / MySQL 8.4 / DuckDB docs; every ansi row cites a Feature id grounded on PostgreSQL 16's SQL-conformance table.
  • DialectCensusSpec.scala — the generator/guard: wrap-tolerant anchors resolved to file:line at emit time (exactly-one-hit or the build fails naming the anchor), every example parsed through the real Parser, ES constructs derived from the parsed AST (es_construct_source = derived 170 / pinned 89), two-way registry coverage diffs, the 95/95/149 pins, per-token minimum-form gates, and a phantom-help corpus walk (13 phantoms, 3 of them wired into json/_index.json). Nothing is assume-guarded — a wrong working directory fails, it never skips. Runs on every sbt sql/test.

Findings recorded in the census (19.2/19.3 inputs)

  • || (Pipe) and bare E are dead tokens — declared and registered, but no production consumes them; jOOQ renders ||, a guaranteed round-trip gap.
  • Bare CROSS JOIN is grammar-accepted but rejected by StandardJoin.validate (the CrossJoin exemption in Join.validate is unreachable).
  • Silent-semantics traps: LOG is the NATURAL log (PG/DuckDB log = base-10); STRPOS reverses PG/DuckDB argument order; DAYOFWEEK numbering differs across all three engines; canonical DATE_TRUNC(date, part) reverses PG/DuckDB.

Verification

  • sbt "sql/testOnly *DialectCensusSpec" — 16/16 green; emits sql/target/epic-19/dialect-census.{csv,json,md}.
  • sbt "+ sql/test" — 548/548 on both Scala 2.12.20 and 2.13.16.
  • sbt scalafmtAll clean; bare sbt headerCheck green; SQLKeywordsSpec untouched and green.

The census tables consumed by stories 19.2/19.3/19.4 are local _bmad-output/planning-artifacts/epic-19-function-census.{md,csv,json,md-table} deliverables (gitignored by project convention); the reproducibility guarantee is this tracked spec, which regenerates them byte-for-byte.

Closes #248

🤖 Generated with Claude Code

…ards (Epic 19 / R2a audit)

One machine-generated, drift-proof inventory of every construct the SQL
dialect accepts: 259 SYNTAX-FORM rows (171 function rows covering all 95
SQLKeywords.functionTokens and all 149 accepted spellings, 29 operators,
36 clauses, 6 literals, 17 units), each anchored to a verbatim source
substring resolved to file:line at emit time (exactly-one-hit,
wrap-tolerant), each example proven parseable by the real Parser, ES
constructs derived from the parsed AST with an es_construct_source
column, Rule-R shape-checked citations (every ansi_adjacent URL fetched),
two-way registry coverage diffs, the 95/95/149 pins, and a phantom-help
corpus walk (13 phantoms, 3 of them wired). Emits
sql/target/epic-19/dialect-census.{csv,json,md} on every run; nothing is
assume-guarded. Green on Scala 2.12.20 + 2.13.16; SQLKeywordsSpec
untouched.

Story 19.1 (dialect-census), Epic 19.

Closed Issue #248

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@fupelaqu
fupelaqu marked this pull request as ready for review September 1, 2026 05:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Authoritative source-derived dialect census with anti-drift guards (R2a function-library audit)

1 participant