Skip to content

Rewrite the lineage readme, and make CI read prose - #53

Closed
sqlparser wants to merge 1 commit into
unblock-bump-pr-buildfrom
stale-docs-check
Closed

Rewrite the lineage readme, and make CI read prose#53
sqlparser wants to merge 1 commit into
unblock-bump-pr-buildfrom
stale-docs-check

Conversation

@sqlparser

Copy link
Copy Markdown
Owner

Stacked on #52 — merge that first, and GitHub will retarget this to master.

From the first-time evaluation of a fresh clone (2026-08-24). The root README path works end to end; what fails an evaluator is the folder readmes.

The lineage readme told visitors the demo was broken

src/main/java/gudusoft/gsqlparser/demos/dlineage/readme.md — the folder most people open first — said, and I verified each of these against the current tree:

the readme said reality
DataFlowAnalyzer is excluded from the root build pom.xml excludes nothing at all since 2026-07-28; the uber jar carries the class
…because the trial parser lacks the metadata layer it builds and runs fine on trial 4.2.6
"That build currently fails", use mvn -f pom_dlineage.xml package that POM was deleted 2026-07-28
java -jar gudusoft.dlineage.jar /t oracle /f ../sample/demo.sql run verbatim: Error: Unable to access jarfile. Wrong jar name, wrong sample path

13 dead references in one file, and every build stayed green.

Rewritten against measured behaviour. The option list is now reproduced verbatim from the tool’s own output instead of hand-maintained — the old copy had already drifted, missing /csv-simple, /showER and the gaussdb dialect. Three things it never mentioned are now documented, each verified:

  • /fromdb is dead. SqlflowIngester.export(...) is commented out and the class deleted, so it emits an empty <dlineage/> and writes no metadata.json — while the readme spent two sections teaching it, with four vendor examples.
  • mvn exec:java cannot run this demo on JDK 21 — JAXB against exec:java’s child classloader throws a loader constraint violation before any output. The uber jar is the supported route, which is exactly why CI smokes the jar.
  • The trial 10,000-byte cap rejects 16 of the 89 .sql files under samples/, all of them schema dumps under samples/dlineageBasic/, returning a <dlineage> whose only content is an <error> — easy to read as "no lineage found".

demos/readme.md gets the same treatment: its Maven section told the reader to comment out a private parent POM, repoint system-scope dependencies at external_lib/, and add a vendored gudusoft.gsqlparser-3.0.1.5.jar. All removed in the 2026-07/08 cleanups.

The general fix

.github/scripts/check-stale-docs.sh fails if any tracked .md outside README.md and docs/maintenance-notes.md names pom_dlineage.xml, gudusoft.dlineage.jar, or the old demos package root. build.yml runs it on every push and PR.

It runs --self-test first, and that is not ceremony — a fixed-string scan that matches nothing is indistinguishable from a clean repository, the same fail-open shape as the pre-commit hook this repo already tests in CI.

$ .github/scripts/check-stale-docs.sh --self-test
  ok    a clean document passes
  ok    "pom_dlineage.xml" is caught
  ok    "gudusoft.dlineage.jar" is caught
  ok    "src/main/java/demos/" is caught
  ok    "src\main\java\demos\" is caught
self-test: all 5 cases pass

$ .github/scripts/check-stale-docs.sh
checking 61 markdown files for references to deleted things
ok: no documentation points at anything that has been removed

It was also confirmed to fail on an injected regression in a real readme before being wired up.

Not in this PR

The evaluation raised three product questions this cannot answer: shipping T*SQLDataSource in the trial jar (or taking connector/ off the trial path), restoring or deleting /fromdb, and adding a sub-10 KB schema sample so "lineage on a real schema" is trial-evaluable.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UK3dBXFXqiDRYJxz1YxWCr

A first-time evaluation of a fresh clone on 2026-08-24 found the root README's
path works end to end -- 156 tests, 79 demos start, 24/24 demo cases -- and then
found that the lineage demo's own readme, the folder most visitors open first,
still told them the demo was broken:

  - "DataFlowAnalyzer is **excluded from the root build**". It is not, and has
    not been since 2026-07-28; pom.xml excludes nothing at all, and the uber jar
    carries the class. The stated reason -- that the trial parser lacks the
    metadata layer -- was wrong too. It builds and runs on trial 4.2.6.
  - "That build currently fails", followed by `mvn -f pom_dlineage.xml package`
    for a POM deleted on 2026-07-28.
  - `java -jar gudusoft.dlineage.jar /t oracle /f ../sample/demo.sql` -- wrong
    jar name and wrong sample path. Run verbatim it gives "Unable to access
    jarfile". The working command produces 24 relationships.

13 dead references in one file, and nothing was red.

The readme is rewritten against measured behaviour. The option list is now
reproduced verbatim from the tool's own output rather than hand-maintained: the
old copy had drifted, missing /csv-simple, /showER and the gaussdb dialect.
Three things it never said are now documented, each verified here:

  - `/fromdb` is dead. SqlflowIngester.export(...) is commented out and that
    class has been deleted, so it prints an empty <dlineage/> and writes no
    metadata.json, while the readme devoted two sections to teaching it.
  - `mvn exec:java` cannot run this demo on JDK 21 -- JAXB against exec:java's
    child classloader gives a loader constraint violation before any output.
    The uber jar is the supported route, which is why CI smokes the jar.
  - The trial parser's 10,000-byte cap rejects 16 of the 89 .sql files under
    samples/, every one of them a schema dump under samples/dlineageBasic/, and
    returns a <dlineage> containing only an <error> -- easy to read as "no
    lineage found".

demos/readme.md gets the same treatment: its Maven section told the reader to
comment out a private parent POM, repoint a list of system-scope dependencies at
external_lib/ and add a vendored gudusoft.gsqlparser-3.0.1.5.jar. All of that
was removed in the 2026-07 and 2026-08 cleanups.

Then the general fix, since prose does not compile and neither of these was
visible to a green build: .github/scripts/check-stale-docs.sh fails if any
tracked .md outside README.md and docs/maintenance-notes.md names
pom_dlineage.xml, gudusoft.dlineage.jar or the old demos package root. build.yml
runs it on every push and pull request.

It runs --self-test first, and that is not ceremony: a fixed-string scan that
matches nothing is indistinguishable from a clean repository, which is the same
way the hook this repo tests in CI fails open. The self-test asserts each dead
string is caught and that a clean document passes; the check was also confirmed
to fail on an injected regression in a real readme before being wired up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UK3dBXFXqiDRYJxz1YxWCr
@sqlparser
sqlparser deleted the branch unblock-bump-pr-build August 24, 2026 14:11
@sqlparser sqlparser closed this Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant