Repository-artifact layer for python — artifact / dependency / config_key nodes

[georgesafta]

Plan

kind-conjuring-dawn.md (local plan) — implements the application-anchored slice of the committed spec docs/design/specs/repository-artifact-layer.md (codellm-devkit/.github), roadmap candidates 15–17.

Problem

codeanalyzer-python emits a source-only view: application.symbol_table plus git provenance (app.repository, core.py:625). It has no model of the repository's non-source files, its declared dependencies, or its configuration keys — the producer-side evidence a cross-service / whole-application analysis needs. This issue adds an application-anchored artifact layer (three new node families) as the first of the group-D units, mirroring the existing repository field's ungated, all-levels shape.

Scope boundary

• Adds artifact / dependency / config_key nodes only — the three application-anchored families.
• Does NOT add config_use (the PY_USES_CONFIG body-node edge, candidate 18) or the PyCallArgument.value extension it needs — that is a follow-up PR. (.env config keys ARE emitted now, namespace env, so the follow-up edge has definitions to resolve against.)
• Does NOT touch the source tree, call graph, or dataflow output; existing symbol_table/call_graph/cfg/cdg/ddg emission is unchanged.
• typescript / java analyzers and the python-sdk mirror are separate trains (epic Epic: repository-artifact layer — artifacts, dependencies, config, config-uses (schema + 3 analyzers + SDK) .github#45), out of scope here.

Goals

• PyArtifact / PyDependency / PyConfigKey Pydantic models (schema/py_schema.py), all fields optional-with-default so exclude_none emits no null leaves; artifacts: Dict[str, PyArtifact] map on PyApplication.
• can://…/@artifact/… id helpers in schema/ids.py (artifact_id / dependency_id / config_key_id), reusing child_id + path normalization.
• Artifact inventory builder (discovery walk reusing _SKIP_DIRS; sha256 hash; text-capture policy — decodable→text, binary→none, over-cap→text_truncated), attached at core.py:625 alongside repository_info (ungated, all levels). Classify the Python-ecosystem file set explicitly (.env/.env.*/.flaskenv, manifests, lockfiles, Dockerfile/compose, *.tf, CI); unknown → other, never dropped.
• Dependency parse — extend the pyproject.toml/requirements.txt reader in entrypoints/detect.py (share span/strip helpers; add version_spec/scope/ecosystem/direct) with the §5 scope mapping. Also parse Pipfile, setup.cfg deps, and poetry.lock/uv.lock/Pipfile.lock for resolved_version (lockfile-only transitive deps → direct=False).
• Config-key parse (overlay — parse failure never drops the artifact node) for .env/.env.*/.flaskenv (KEY=value, namespace env), yaml/json/properties/ini/toml: dotted keys, value, recognized references[].
• --artifact-text/--no-artifact-text (default True) and --artifact-text-max-bytes (default 262144), skip_tests wiring — disabling text drops only the payload, not the inventory.
• Neo4j: PyArtifact/PyDependency/PyConfigKey NodeLabels + PY_HAS_ARTIFACT/PY_DECLARES_DEPENDENCY/PY_DEFINES_CONFIG RelTypes in neo4j/schema.py; matching _*_props + b.node/b.edge walk in neo4j/project.py; SCHEMA_VERSION MINOR bump (2.0.0→2.1.0).
• .claude/SCHEMA_DECISIONS.md entry (first-class-nodes decision, ungated placement, id grammar, closed enums, deferred config_use).

Caveats and known risks

• Secret capture. .env and config files often hold secrets, and text capture is on by default (spec's uniform raw-text policy — the file is already on disk). --no-artifact-text is the off-switch; --artifact-text-max-bytes bounds size. This is a documented, deliberate default, not an oversight.
• TOML config parsing needs tomllib (3.11+ stdlib). Verify requires-python before relying on it; if the floor is below 3.11, skip config-key extraction for .toml while still inventorying the artifact (config parsing is a pure overlay — never blocks the node). yaml is already a dependency.
• Neo4j props and the DDL catalog are two hand-maintained lists (project.py builds props manually; it imports only SCHEMA_VERSION from schema.py). They must be kept in agreement by hand — the acceptance check is --emit schema reflecting the new families.
• Monotonicity: the artifact layer is ungated/all-levels, so its section must be byte-identical across -a 1|2|3|4. test_v2_superset.py must cover it.
• Lockfile format drift. poetry.lock/uv.lock/Pipfile.lock have distinct schemas; parse defensively and treat an unparseable lockfile as "no resolved_version" (deps still emitted from the manifest), never a hard failure.

Definition of done

• A fixture project (with pyproject.toml incl. optional-dependencies, requirements.txt, a lockfile, a .env, a .yml + .properties config, a binary file, an over-cap text file) produces an artifacts map where: artifact_kind classification is correct (unknown→other, never dropped); dependencies carry the hand-verified scope/version_spec/ecosystem, with resolved_version present where the lockfile pins it; config keys are the exact dotted-key set with references[], and .env keys carry namespace env; the binary file is inventoried with no text; the over-cap file has text_truncated=True.
• assert_conformant + _assert_no_nulls pass on the fixture output.
• The artifacts section is byte-identical across -a 1|2|3|4 (monotonicity gate green).
• --no-artifact-text run: the inventory (paths/kinds/hashes/deps/config keys) is identical to the default run; only text/text_encoding/text_truncated are absent. --artifact-text-max-bytes set below a fixture file's size yields text_truncated=True with a prefix of exactly that many bytes.
• --emit schema lists the three node labels + three rel types and SCHEMA_VERSION 2.1.0; a projection run emits PY_HAS_ARTIFACT/PY_DECLARES_DEPENDENCY/PY_DEFINES_CONFIG with artifact text present in the graph.
• uv run pytest test/ green; a fixture diff shows symbol_table/call_graph/cfg output unchanged.

[rahlk]

PR #160 (spec in #158, follow-on #161) landed an overlapping foundation for this issue's.

I am filing the reconciliation here rather than closing anything, since this issue plus epic codellm-devkit/.github#45 describe a superset of what #160 ships.

@georgesafta let me know what you think about these...

1. Artifact id grammar — can://artifact/<app>/<path> (language-neutral), not can://python/<app>/@artifact/….
I found that the same artifact file frequently serves two or more languages in one repo (e.g., a docker-compose.yml orchestrates the Python service and the TS frontend); one Dockerfile builds both. Under a language-namespaced id, every language analyzer mints its own node for the same file and a merged graph carries duplicates that no consumer can safely collapse. So I was thinking that a neutral namespace makes the artifact node the join point: analyzers over the same repo MERGE onto one node for the common file. (assuming we have the --app-name agreement)

2. Node labels neutral (Artifact, Package with purl ids); PY_ prefix reserved for language-specific atifacts.

Same rationale as above. Shall we split nodes such that when they represent language-independent things (a file, a PyPI/npm/Maven package) carry no language prefix, so sibling analyzers share them and purl ids (pkg:pypi/<name>) give the cross-ecosystem SBOM join for free.

Edges that are this analyzer's claim stay Py-prefixed exactly as this issue proposes: PY_PROVIDES ("the Python import graph is fed by this package") and PY_UNRESOLVED_IMPORT are Python-specific assertions and are namespaced accordingly. So the prefix convention survives — it just attaches to the language-specific statements, not to the shared nouns.

3. Lockfile-only transitive deps with direct=False — this issue's design is better, and #160 will adopt it.

Adopting PyDependency.direct: bool (default true), lock-only pins emitted with direct=false, prov: ["lockfile"], declared_in = the lock artifact makes more sense, so I Will update that as a follow-up commit on PR #160's barnch.

4. Neo4j SCHEMA_VERSION stays 2.0.0 for now.
There are no consumers of 2.0.0 yet and the artifact-layer contract is still settling. When the epic's vocabulary settles across analyzers, one bump can cover the settled contract.

** What #160 doesn't cover from this issue (lmk if you want me to take them):**

• The config_key family (PyConfigKey, PY_DEFINES_CONFIG, dotted keys, namespace env, references[]) and the follow-up config_use edge. feat: schema v2 artifacts, dependencies, and unresolved imports #160 deliberately captured config files as artifact nodes with verbatim source and no key extraction — so the definitions substrate exists, and key extraction is a pure additive overlay on those nodes.

• Never-drop inventory (unknown → other): feat: schema v2 artifacts, dependencies, and unresolved imports #160 captures rule-matched files (+ shebang scripts) only.

• Text-capture controls (--no-artifact-text, --artifact-text-max-bytes, text_truncated, binary files inventoried without text) and the secret-capture caveat: feat: schema v2 artifacts, dependencies, and unresolved imports #160 currently emits unbounded source and skips binaries entirely.

Do we wanna rebase this issue's scope onto #160's foundation? We keep the above three gaps above as this issue's remaining goals, adopt the settled vocabulary from points 1–2, and land direct=False (point 3) on #160 directly. What do you think?