Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
b618ba2
docs(design): spec the tsc + defuse linker call graph, Jelly removal …
rahlk Aug 27, 2026
506564f
feat(callgraph)!: remove Jelly wholesale — one resolver code path (#98)
rahlk Aug 27, 2026
30f95ac
fix(syntactic): record the call when a concise arrow body IS the call…
rahlk Aug 27, 2026
902fae3
feat(callgraph): the defuse linker — per-callable tiers over the tsc …
rahlk Aug 27, 2026
2981d0d
test(ledger): Joern jssrc2cpg superset gate — 74/74 real pairs, resid…
rahlk Aug 27, 2026
26aa4e2
docs: call-graph sections for the tsc + defuse linker (#98)
rahlk Aug 27, 2026
671c454
fix(syntactic): tagged template expressions are call sites (#98)
rahlk Aug 27, 2026
45a0d0d
fix(syntactic): guard tagged-template call sites (no arguments list) …
rahlk Aug 27, 2026
5991953
fix(callgraph): tagged-template callee access in the linker sweeps (#98)
rahlk Aug 27, 2026
6d921a9
fix(syntactic): parameter-default initializer calls belong to the cal…
rahlk Aug 27, 2026
61725e2
test(ledger): vscode-scale Joern audit — 99.56%, residual 244 classif…
rahlk Aug 27, 2026
55d3818
feat(callgraph): JS source discovery + the T4c ctor-field callback ch…
rahlk Aug 27, 2026
54f9bd7
fix(syntactic): a .d.ts beside its .js is declarations, not the modul…
rahlk Aug 27, 2026
93771a5
test(ledger): rounds 5-8 — JS discovery, T4c chain; vscode 99.66%, re…
rahlk Aug 27, 2026
cb52efd
docs(ledger): residual heading count (#98)
rahlk Aug 27, 2026
756a710
feat(callgraph): propagation tiers + property-initializer attribution…
rahlk Aug 27, 2026
f2fdd72
test(ledger): rounds 9-10 — 99.72%, residual 135; param-shadow proven…
rahlk Aug 27, 2026
fbda332
test(ledger): vscode L4 end-to-end — 10m42s / 28.6GB, SDG at scale, O…
rahlk Aug 27, 2026
905abbd
docs(design): spec the repository-artifact layer, python parity (#101)
rahlk Aug 27, 2026
8cbf665
feat(schema): the repository-artifact layer — artifacts, dependencies…
rahlk Aug 27, 2026
e954070
docs(design): record the artifact-layer spec as implemented (#101)
rahlk Aug 27, 2026
b2710f2
feat(schema)!: recalibrate the artifact layer to python PR #160 (#101)
rahlk Aug 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 18 additions & 8 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,12 @@ while cache round-trips tree), `l1Body` (`call_sites` → `body{}`), `heritage`,
(`src/schema/emit.ts`) runs them + assembles envelope + strips INTERNAL fields
(`call_sites`, `abs_path`, cache trio).

Call graph defaults to **union** of two backends: TS compiler resolver
and embedded [Jelly](https://github.com/cs-au-dk/jelly) flow analyzer (recovers
higher-order/callback edges resolver misses). Merged edges keep
`provenance` tag (`tsc` / `jelly`); `--tsc-only` or `--call-graph-provider jelly`
picks one alone.
Call graph = tsc resolver + **defuse linker** (#98): deterministic per-callable
pass over resolver leftovers — alias chains, decorator edges, library-callback
edges, bounded interprocedural votes, CHA-by-name fallback. No whole-program
fixpoint, no backend flag, one code path. Module-scope calls attributed to
MODULE (python #131 parity). prov tags: `tsc` / `defuse` / `import`. Joern
superset ledger: `docs/design/specs/defuse-linker-joern-ledger.md`.

## Architecture — follow the pipeline

Expand All @@ -65,8 +66,9 @@ it first; everything else is stage it calls, in order:
2. **buildSymbolTable** (`src/syntactic_analysis`) — modules, classes, interfaces,
enums, type aliases, namespaces, functions, methods, variables, decorators,
JSDoc, with precise source spans.
3. **call graph** (`src/semantic_analysis`) — `selectProvider()` picks tsc / jelly /
union; each provider returns edges + external (phantom) symbols.
3. **call graph** (`src/semantic_analysis`) — tsc resolver (`callGraph.ts`, incl.
module-scope sweep + RTA + phantoms) then `defuseLinker.ts` tiers T1–T5;
merged with provenance union.
4. **program graphs** (`src/dataflow`) — levels 3–4 (`-a 3`/`-a 4`): CFG → post-dominance/CDG →
access-path def-use → PDG → SCC-condensed bottom-up summaries → SDG. This is *compute*
(IR in `src/schema/graphs.ts`); `src/dataflow/attach.ts` writes it **onto tree**
Expand Down Expand Up @@ -96,13 +98,21 @@ test — treat both as contracts, keep in lockstep with JSON.
| `src/core.ts` | `analyze()` orchestrator — the spine |
| `src/options` | Parsed CLI options / `AnalysisOptions` |
| `src/syntactic_analysis` | Symbol table (ts-morph traversal) |
| `src/semantic_analysis` | Call-graph providers (tsc, jelly, union), phantoms |
| `src/semantic_analysis` | Call graph: tsc resolver + defuse linker (T1–T5), phantoms |
| `src/dataflow` | L3/L4 program-graph **compute** (CFG, dominance/CDG, def-use, summaries, SDG) + `attach.ts` (IR → tree) |
| `src/schema` | **the native v2 model** (`schema.ts`) + per-run passes (`assignIds`/`l1Body`/`heritage`/`homing`/`l2Callees`) + `emit.ts` (`finalizeAnalysis`) + `signatureOf` + graphs IR |
| `src/build` | Dep materialization; `build/neo4j` = the v2 graph projection (project/rows/cypher/bolt/schema) |
| `src/utils` | fs, caching, logging, serialization (`serialize.ts` writes the envelope), version |
| `test` | Bun tests + `fixtures/sample-app` + `fixtures/dataflow-app`; `schema-v2.test.ts` = the L1–L4 gates |

**Repository-artifact layer** (#101, python PR #160 parity): `application.artifacts{}`
(rules-matched non-code files, LANGUAGE-NEUTRAL `can://artifact/<app>/<path>` ids,
roles[], verbatim unbounded `source`) + flat `dependencies[]` (npm kinds incl.
coined `peer`, prov-tagged, lock backfill) + `unresolved_imports[]` (@types
type-only rule; `--resolve-installed` opt-in probe). `src/artifacts/`. Neo4j
contract 2.2.0: NEUTRAL :Artifact/:Package (purl) — sanctioned prefix exception —
plus TS_PROVIDES/TS_UNRESOLVED_IMPORT into :TSExternal ghosts.

## Commands

- `bun run start -- --input /path/to/project` — run analyzer from source.
Expand Down
36 changes: 14 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ structure into a **Neo4j property graph**. It is the TypeScript backend behind
[Python](https://github.com/codellm-devkit/codeanalyzer-python) and
[Java](https://github.com/codellm-devkit/codeanalyzer-java) siblings.

By default the call graph is the **union** of two backends: the TypeScript compiler's resolver and
[Jelly](https://github.com/cs-au-dk/jelly) — a flow-based analyzer that resolves higher-order and
callback edges the resolver misses, embedded in the `cants` binary (no extra install). Merged edges
keep a `provenance` tag (`tsc` / `jelly`), so you can still tell the two apart. Pass `--tsc-only` to
drop Jelly and run the resolver alone, or `--call-graph-provider jelly` for Jelly alone.
The call graph is the TypeScript compiler's resolver plus a **defuse linker** — a deterministic,
per-callable pass that backfills the edges the resolver misses (alias chains, decorator
invocations, callbacks handed to library calls, parameter-flow calls) with no whole-program
fixpoint. Edges keep a `provenance` tag (`tsc` / `defuse` / `import`), so you can tell the layers
apart, and the output is byte-identical across runs.

## Table of Contents

Expand All @@ -55,9 +55,9 @@ drop Jelly and run the resolver alone, or `--call-graph-provider jelly` for Jell
methods, variables, decorators, and JSDoc, with precise source spans.
- **Call graph** — the TypeScript compiler's resolver plus Rapid Type Analysis (RTA), with
**phantom (external) nodes** for calls into imported libraries and Node builtins.
- **Pluggable call-graph backend** — the `union` of the `tsc` resolver and the embedded
[Jelly](https://github.com/cs-au-dk/jelly) flow analyzer by default (`--tsc-only` for the resolver
alone, `--call-graph-provider jelly` for Jelly alone).
- **Defuse linker** — a deterministic per-callable pass over the resolver's leftovers: alias
chains, decorator invocations, library-callback edges, and bounded interprocedural votes —
validated as a strict superset of Joern's real call pairs on the reference corpus.
- **Neo4j output** — project the analysis into a labeled property graph: a self-contained
`graph.cypher` snapshot, or an **incremental** push to a live database over Bolt.
- **Versioned schema** — a machine-readable, version-stamped Neo4j schema contract
Expand Down Expand Up @@ -182,11 +182,8 @@ Options:
node_modules)
--no-phantoms disable phantom (external) nodes for
imported/required library calls
--call-graph-provider <name> call-graph backend: union (default, tsc ∪
jelly) | tsc | jelly | both (deprecated alias
of union) (default: "union")
--tsc-only use the tsc resolver only — opt out of Jelly
edges (overrides --call-graph-provider)
--resolve-installed probe node_modules metadata for import→package
binding (default: repo files only)
-c, --cache-dir <dir> cache/intermediate directory
-v, --verbose increase verbosity (repeatable)
-h, --help display help for command
Expand Down Expand Up @@ -214,17 +211,12 @@ Options:
cants --input ./my-ts-project --target-files src/a.ts src/b.ts
```

4. **Resolver-only call graph (opt out of Jelly):**
```sh
cants --input ./my-ts-project --tsc-only
```

5. **Force a clean rebuild with a custom cache directory:**
4. **Force a clean rebuild with a custom cache directory:**
```sh
cants --input ./my-ts-project --eager --cache-dir /path/to/custom-cache
```

6. **Program graphs (level 3): CFG/PDG/SDG in `analysis.json`:**
5. **Program graphs (level 3): CFG/PDG/SDG in `analysis.json`:**
```sh
cants --input ./my-ts-project -a 3 # full program_graphs section
cants --input ./my-ts-project -a 3 --graphs cfg,pdg # scope the emitted graphs
Expand Down Expand Up @@ -282,8 +274,8 @@ nodes all join.

**Substrate (locked in [issue #2](https://github.com/codellm-devkit/codeanalyzer-typescript/issues/2)):**
the CFG and reaching-definitions are hand-built from the ts-morph AST; the call-graph oracle is
the existing provenance-merged tsc ∪ Jelly graph; aliasing is a flow-insensitive copy-alias MVP
(Jelly points-to-backed propagation is a staged upgrade). Function summaries are composed
the provenance-merged tsc + defuse graph; aliasing is a flow-insensitive copy-alias MVP
(points-to-backed propagation is a staged upgrade). Function summaries are composed
bottom-up over the SCC condensation of the call graph, with k-limited access paths; module
globals ride the SDG as extra parameters. The analysis is deliberately sound-leaning and
over-approximate; known unsoundness (dynamic `eval`, reflection/monkey-patching, npm-internal
Expand Down
96 changes: 96 additions & 0 deletions docs/design/specs/artifacts-and-dependencies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Artifacts and dependencies — the repository-artifact layer for TypeScript

- **Status:** implemented (branch `feat/issue-101-artifacts`); **recalibrated 2026-08-27** to the
ratified python contract after the first cut anchored on an orphaned branch
- **Scope:** `codeanalyzer-typescript`; schema v2 **additive** (no level, id-tier, or existing-field movement)
- **Parity anchor:** codeanalyzer-python **PR #160** (implementation of the approved spec
`2026-08-27-artifacts-and-dependencies-design.md`, PR #158). NOT the `51ee29e`
`feat/configuration-files` branch — that shape (language-namespaced `@artifact/` ids, contained
dependency/config-key children, `artifact_kind` enum, text-capture caps) was never merged; this
spec's first revision mirrored it and has been rebuilt.
- **Tracking:** one work item (#101), one PR (#103), branch stacked on `feat/issue-100-linker-propagation`

## Contract-impact triage

| Question | Answer |
| --- | --- |
| Schema v2 shape | **additive**: `application.artifacts{}` (flat nodes), `application.dependencies[]` (flat evidence rows), `application.unresolved_imports[]` |
| Identity | artifact ids are **language-NEUTRAL**: `can://artifact/<app>/<path>` — the first `can://` segment is a namespace (a language for code, the literal `artifact` for files), so sibling analyzers over one repo emit the SAME id for the same file. `<app>` agreement is the precondition for cross-analyzer joins |
| Levels / monotonicity | ungated, identical at `-a 1..4`; monotonicity holds trivially |
| schema_version | unchanged; Neo4j contract 2.1.0 → **2.2.0** (additive) |
| Repos | `codeanalyzer-typescript` now; **python-sdk** must gain the three families before its analyzer pin moves (its models are `extra="forbid"` — verified; python's own PR #160 carries the same obligation); repo docs |
| Shared vocabulary movement | ONE additive token: dependency `kind: "peer"` (npm's contract-with-host) against the ratified enum `runtime\|dev\|optional\|build`. Recorded like the `"reaching-defs"` precedent |

## The mirrored model (python PR #160 shapes)

```
application.artifacts: Record<repoRelPath, TSArtifact>
TSArtifact id = can://artifact/<app>/<path> (per-run), kind "artifact", path,
format (json|jsonc|yaml|toml|ini|dockerfile|yarnlock|env|text),
roles[] (dependency-manifest|tool-config|container-image|service-topology|
ci|env|packaging|legal|docs|script|unknown),
size_bytes, sha256, source (verbatim, UNBOUNDED by decision — spec §3),
extraction (none|partial|full)

application.dependencies: TSDependency[] # flat, no node ids — :Package (purl) is the node
TSDependency name (@scope kept), spec, kind (runtime|dev|optional|peer|build), extras[] (npm: []),
declared_in (artifact id), locked_version?, provides_imports[],
prov[] (declared|lockfile|installed-metadata|heuristic)

application.unresolved_imports: TSImportBinding[]
TSImportBinding module (specifier root), bound_to?, prov[]
```

## Locked TS decisions (recalibration session 2026-08-27)

1. **config_keys dropped** — python's unit 4 owns config extraction; config-role artifacts get
node + roles + source only. The earlier TS config-key parser is parked (this file is its record).
2. **`peer` kind coined** (additive). npm mapping: `dependencies→runtime`,
`devDependencies→dev`, `optionalDependencies→optional`, `peerDependencies→peer`.
3. **Capture is rules-matched only** (python's posture): the shipped table in
`src/artifacts/rules.ts` (glob → format/roles, roles union across matches; basename patterns
match at any depth, `/`-patterns anchor at root) + extensionless shebang files as `script`.
An unmatched file is NOT an artifact. `source` is verbatim and unbounded (python's decision;
revisit only with measured payload numbers); binary probe failures carry `source: ""`.
4. **Dependencies are declared-only and flat**: every `package.json` (workspace members included)
emits records with `declared_in` = its artifact id; the JSON lock family
(`package-lock.json`/`npm-shrinkwrap.json`/`bun.lock` JSONC) backfills `locked_version` on the
OWNING (sibling) manifest's records and appends `"lockfile"` to `prov`; locks never create
records; `yarn.lock`/`pnpm-lock.yaml` are inventory-only artifacts.
5. **provides_imports**: the package name itself; `@types/x` also provides `x`
(DefinitelyTyped `scope__pkg` unmangled to `@scope/pkg`).
6. **Import binding / unresolved_imports**: every non-relative, non-builtin specifier ROOT from
the symbol table's imports. A VALUE import needs the runtime package; an `import type` is
satisfiable by `@types/x` alone. Only-@types-for-a-value-import → partially bound
(`bound_to: "@types/x"`, `prov: ["heuristic"]`). `--resolve-installed` (opt-in, default off)
probes `node_modules/<name>/package.json` (`prov: ["installed-metadata"]`); default runs read
only repo files and stay byte-identical.
7. **Neo4j (contract 2.2.0)**: language-NEUTRAL `:Artifact` and `:Package` (purl ids,
`pkg:npm/<name>`, scoped `pkg:npm/%40scope/<name>`) — the deliberate, sanctioned exception to
TS-prefixing so sibling analyzers MERGE onto the same nodes (the conformance gate allowlists
exactly these). Edges: `HAS_ARTIFACT`, `DECLARES_DEPENDENCY` (props spec/kind/extras/prov,
`_k` = kind), `LOCKS` (version; fans from every lock artifact present — python's documented
coarse fan), and the analyzer's own claims `TS_PROVIDES` (Package→minted module-level
`:TSExternal` ghost) and `TS_UNRESOLVED_IMPORT` (application→ghost, prov). `source` stays off
the graph.
8. **Pipeline**: `src/artifacts/` (rules, deps, binding, index) runs in `analyze()` after the
symbol table (binding needs module imports), level-ungated, not cached; `assignIds` stamps
artifact ids and re-stamps `declared_in` per run (`--app-name` rule).

## Definition of done

- Three sections emitted identically at every `-a`; monotonicity + conformance + count-parity
gates green (parity gate counts neutral Artifact/Package rows + minted ghosts explicitly).
- Fixture app: root+workspace manifests, both JSON locks, `yarn.lock` inventory-only, `.env`,
tsconfig, Dockerfile, CI workflow, LICENSE, an undeclared VALUE import, an `import type`
satisfied by `@types` — every kind token incl. `peer`, prov chains, purl ids (scoped included),
`--resolve-installed` exercised.
- Determinism: two consecutive default runs byte-identical.
- `schema.neo4j.json` regenerated at 2.2.0; CLAUDE.md + SCHEMA_DECISIONS + README/--help updated.

## Release plan

Ships in the minor after the linker train (#97 → #99 → #102 → #103); schema_version unmoved;
Neo4j 2.2.0 in release notes. **SDK lockstep required** (`extra="forbid"`): python-sdk gains the
three families before its pin moves. Cross-analyzer id joins additionally require pinned
`--app-name` agreement between analyzers (spec §2 precondition).
Loading