feat(schema): repository-artifact layer — artifacts, dependencies, config keys - #103
Open
rahlk wants to merge 22 commits into
Open
feat(schema): repository-artifact layer — artifacts, dependencies, config keys#103rahlk wants to merge 22 commits into
rahlk wants to merge 22 commits into
Conversation
BREAKING: the released --call-graph-provider and --tsc-only flags are removed (commander now rejects them); the tsc resolver is the one base call graph, per the defuse-linker design (docs/design/specs/defuse-linker-call-graph.md). Gone with them: the union/jelly providers and selector, the __jelly multi-call binary mode and CANTS_SELF_JELLY re-exec, the @cs-au-dk/jelly dependency, its patch, and the --external @babel/preset-typescript bundling workaround (bundle: 998 → 455 modules). mergeCallGraphs stays — the defuse linker overlays the tsc base through it. python-sdk's tsc_only kwarg keeps passing --tsc-only until its tracked follow-up lands (spec, release plan). Full suite + typecheck green; binary builds.
…#98) walkBody only visited the body's CHILDREN unless the body was itself a callable boundary, so `u => u.describe()` never recorded u.describe() as a call site — an L1 gap the Jelly leg papered over with its own approximated edge. Visiting the body node itself closes it: the site lands in the anon's call_sites, the tsc resolver types it precisely (users: User[] ⇒ u: User), and the body{} call node appears at L1. Additive wire change (a real call site that was missing).
…base (#98) The local pass that backfills what the resolver missed (docs/design/specs/defuse-linker-call-graph.md; python's Jedi+defuse architecture). Per-callable and bounded-round only, sorted iteration — deterministic by construction, no whole-program fixpoint: T1 local value chase (alias chains via bounded symbol hops) T2 decorator invocations (method/accessor/parameter owners; edge-only) T3 external-callback rule (function value passed to an external or unresolved callee ⇒ enclosing→lambda; edge-only — the documented divergence from python) T4 bounded interprocedural votes (param-invoking sites resolve to the functions passed by resolved-internal callers, two rounds; factory returns through a unique returned function) T5 CHA-by-name fallback (bounded fan, skip-not-truncate over the cap) Edges carry prov ["defuse"] and overlay the tsc base via mergeCallGraphs; body-node resolutions return out-of-band and are applied by backfillCallees — never persisted into callee_signature (cache provenance rule). Jelly-recovery on the fixtures: 6/6 jelly-only edges (3 decorator, 1 callback via T3, 2 upgraded to typed tsc resolutions by the concise-arrow fix); edge counts equal the old union exactly (57 / 22 / 2). Full suite + typecheck green.
…ual 0 (#98) The reference-validation harness (scripts/joern: cpg call dump + the signature-mapping comparator) and the audited exception ledger (docs/design/specs/defuse-linker-joern-ledger.md). Corpus: the three repo fixtures + nestjs-realworld-example-app @ c1c2cc4. Every Joern real pair is covered; exclusions are classified per family (their lowering synthetics, parameters-as-callees stubs, super-to-interface fabrications, decorator-attribution variants) — python's ledger discipline. A/B paired runs byte-identical on all four apps.
`inline\`url(...)\`` never recorded a call site — walkBody, the callee resolver, and the call-expression index all matched Call/New only, while the L3 exception model already treated tagged templates as calls. Found by the vscode Joern ledger (432-strong same-file residual family). Additive wire change: real call nodes that were missing.
…+ regression test (#98)
…lable (#98) `createCSSRule(sel, style = getSharedStyleSheet())` executes the default in the callee's activation, but walkBody only covered getBody() — the call (and any nested arrow) was invisible. Found by the vscode ledger (domStylesheets family). Additive wire change + regression test.
…ied (#98) Corpus gate holds at residual 0 (74/74). vscode @ a3c9dc6 (8,735 files, 1.15M LOC): 54,703/54,947 single-candidate Joern real pairs covered after four ledger-driven fix rounds; the 244 residual is classified (escaped closure-locals — python #150's staged tier, vendored marked, the static/instance signature-collision finding, accessor tails). Scale table: cants L2 5m52s/24.4GB for 1.02M edges vs Joern parse 9m06s/30.3GB. Comparator gained fan-row and misresolution-variant rules (their multi-candidate rows are enumeration, not resolution).
…ain (#98) Two residual families from the vscode ledger: - .js/.jsx/.mjs/.cjs sources were never DISCOVERED (SOURCE_EXTS was ts-only) — vscode's vendored marked.js was analyzed through its bodiless .d.ts, so every internal edge was missing. JS files are now first-class, with a compiled-sibling guard (a .js next to its same- prefix .ts is build output, skipped — the compiler's own allowJs duplicate rule). - T4c: `this.field(...)` where the field's value arrives through the constructor (parameter property or this.f = param) resolves to the function values passed at the class's `new` sites — with ONE bounded parameter hop (register(key, cb) → new Migration(key, cb)) and module-scope resolved calls now feeding the vote rounds. The registered callbacks' own param-invoking sites then resolve in the existing T4 rounds: the full migrateOptions chain (apply → callback → apply.write) lands, no fixpoint anywhere.
#98) The compiled-sibling guard registered marked.d.ts's prefix and skipped marked.js — inverted for the hand-written-declarations pattern. Sibling rules are now two-way: .js beside a REAL .ts source is build output (skipped); .d.ts beside an analyzed .js is its declaration file (skipped as a module; the checker still reads it from disk).
…sidual 189 classified (#98)
…#100) - Class property initializers execute in the constructor: their call sites now attribute to the ctor (explicit or implicit), and an initializer ARROW materializes as a class-scoped positional anon — signature ↔ containment stays aligned, and the property-arrow known-gap closes. Instance-prop sites leave the module sweeps (vscode's Registry.as-in-field-initializer family). - T4a property form: 'template.onChange(...)' where the receiver is a parameter votes over object-literal properties (incl. method shorthand) passed at that position. - T4b chained return summaries: 'return makeInner()' follows one resolved-internal level, memoized and cycle-guarded. - Joern comparator: methods' parameter tables ride the dump; a residual whose target leaf-name is a parameter of the Joern caller is their parameters-as-callees fabrication wearing a real name — proven by their own table, classified, never gated. Corpus gate: 74/74, residual 0. vscode re-audit deferred until the L4 benchmark run frees the box.
… by their tables (#100)
…OM ceiling removed (#100)
…, config keys (#101) Python-parity port of codeanalyzer-python 51ee29e: - application.artifacts{}: every non-source file as a first-class node (can://…/@artifact/<path>, dotfiles keep their dot), classified by the shipped rules table (closed artifact_kind enum, catch-all other), content_hash + size always, text under the capture policy (--no-artifact-text / --artifact-text-max-bytes, 256 KiB default — parsing is independent of capture). - Contained TSDependency children from every package.json (workspace members included): npm sections map to the shared scope vocabulary plus the coined additive 'peer' token; the JSON lockfile family (package-lock / npm-shrinkwrap / bun.lock JSONC) backfills resolved_version on the OWNING manifest's declared records only — lockfiles never create records, yarn/pnpm are inventory-only. - Contained TSConfigKey children: .env-family flat keys (namespace env, placeholder refs) and JSON dotted keys. - assignIds stamps all three families per run; level-free and identical at every -a (parity gates taught the three containment families). - The wire strip is now STRUCTURAL (structuredClone + targeted deletes): artifact content_hash is wire payload where the module trio is internal — and the old stringify-roundtrip clone OOM'd at vscode-L4 scale (measured on the L4 benchmark; this removes that ceiling). - Neo4j contract 2.2.0: :TSArtifact/:TSDependency/:TSConfigKey + TS_HAS_ARTIFACT/TS_DECLARES_DEPENDENCY/TS_DEFINES_CONFIG, constraints derived, schema.neo4j.json regenerated; text stays off the graph. - SDK note: python-sdk models are extra=forbid — they must gain these families before the SDK's analyzer pin moves (recorded in spec+issue). Fixture artifacts-app + 15-test gate suite; full suite 151 green.
rahlk
force-pushed
the
feat/issue-101-artifacts
branch
from
August 27, 2026 11:00
5ab3dfc to
e954070
Compare
rahlk
force-pushed
the
feat/issue-100-linker-propagation
branch
from
August 27, 2026 13:27
fbda332 to
b520db3
Compare
The first cut anchored on 51ee29e — an UNMERGED python branch. The ratified contract (python PR #160, spec PR #158) differs materially; this rebuilds the layer to it: - Artifact ids are LANGUAGE-NEUTRAL (can://artifact/<app>/<path>): the first can:// segment is a namespace, so sibling analyzers over one repo emit the same id for the same file. - Artifacts are flat nodes: format, roles[] (union across rule matches), sha256, size_bytes, verbatim UNBOUNDED source (python's decision), extraction status. Capture is rules-matched only (shipped glob table + shebang scripts); text-cap flags removed. config_keys dropped — python's unit 4 owns config extraction. - Dependencies are FLAT evidence rows on the application (no node ids): kind runtime|dev|optional|build + the coined additive 'peer', declared_in (artifact id, re-stamped per run), provides_imports (@types/x also provides x, DT scope-mangling unmangled), prov declared/lockfile; JSON-lock family backfills locked_version on the sibling manifest's records. - unresolved_imports: every non-relative non-builtin specifier root; a VALUE import needs the runtime package, import type is satisfiable by @types alone; --resolve-installed (opt-in) probes node_modules metadata (prov installed-metadata). - Neo4j 2.2.0: NEUTRAL :Artifact/:Package (purl pkg:npm/..., scoped %40-encoded) — the sanctioned prefix-gate exception so siblings MERGE onto the same nodes — HAS_ARTIFACT / DECLARES_DEPENDENCY(_k=kind) / LOCKS (coarse fan from every lock artifact) + the analyzer's own TS_PROVIDES / TS_UNRESOLVED_IMPORT into minted module-level :TSExternal ghosts. Count-parity gates taught the neutral rows. Fixture + 13-test suite rebuilt to the new contract; full suite 149 green; spec + SCHEMA_DECISIONS record the anchor correction.
Contributor
Author
|
Recalibrated to python PR #160 (the ratified contract) — the original anchor (51ee29e) turned out to be an unmerged branch. Delta and decisions recorded in the spec; branch tip is the reviewable state. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #101. Stacked on #102. Spec: docs/design/specs/artifacts-and-dependencies.md.
What (python PR #160 parity)
application.artifacts{}— every RULES-matched non-code file as a flat node with a language-neutral id (can://artifact/<app>/<path>— sibling analyzers over one repo emit the SAME id for the same file):format,roles[](dependency-manifest / tool-config / container-image / service-topology / ci / env / packaging / legal / docs / script / unknown),sha256,size_bytes, verbatim unboundedsource,extractionstatus. Shebang scripts captured; unmatched files are not artifacts.application.dependencies[]— flat, evidence-tagged records from everypackage.json(workspaces included): kindsruntime|dev|optional|buildplus the coined additivepeer;declared_in= the manifest's artifact id;provides_imports(@types/xalso providesx);provchainsdeclared→+lockfilewhen the sibling JSON-family lock (package-lock/npm-shrinkwrap/bun.lockJSONC) pinslocked_version. Locks never create records;yarn.lock/pnpm-lock.yamlare inventory-only.application.unresolved_imports[]— the dependency-hygiene signal: every non-relative, non-builtin specifier root with no accounting declaration. A VALUE import needs the runtime package;import typeis satisfiable by@types/xalone; only-@types-for-a-value-import → partially bound (prov: ["heuristic"]).--resolve-installed(opt-in, default off) probesnode_modulesmetadata (prov: ["installed-metadata"]); default runs read only repo files, byte-identical.:Artifact/:Package(purl ids, scopedpkg:npm/%40scope/name): the deliberate, now-sanctioned exception to TS-prefixing so sibling analyzers MERGE onto the same nodes (the conformance gate allowlists exactly these).HAS_ARTIFACT/DECLARES_DEPENDENCY(_k=kind) /LOCKS(coarse fan from every lock artifact, python's documented rule) + the analyzer's own claimsTS_PROVIDES/TS_UNRESOLVED_IMPORTinto minted module-level:TSExternalghosts — dependencies join the call graph rather than sit beside it.structuredClone) — it also removed the measured vscode-L4 stringify OOM (see the ledger).Gates
Rebuilt fixture (root+workspace manifests, both JSON locks, yarn.lock inventory-only,
.env, tsconfig, Dockerfile, CI workflow, LICENSE, an undeclared VALUE import, animport typesatisfied by @types) + 13-test suite: neutral id grammar, every kind token incl.peer, prov chains, scoped purls, ghost edges,--resolve-installed, level-invariance-a 1≡-a 4, two-run byte-identical determinism. Count-parity gates taught the neutral rows + minted ghosts. Full suite 149 green;schema.neo4j.jsonregenerated at 2.2.0.Release constraints
extra="forbid"— the SDK gains the three families before its analyzer pin moves (python's PR #160 carries the same obligation).--app-nameagreement (spec §2 precondition).config_keysdeliberately absent — python's unit 4 owns config extraction; the earlier TS parser is parked in the spec.