Skip to content

Correct README claims contradicted by the code - #4

Merged
dmccoystephenson merged 1 commit into
mainfrom
docs/accuracy-audit-readme-claims
Sep 2, 2026
Merged

Correct README claims contradicted by the code#4
dmccoystephenson merged 1 commit into
mainfrom
docs/accuracy-audit-readme-claims

Conversation

@dmccoystephenson

Copy link
Copy Markdown
Contributor

Three documentation claims were checked against the code and found to be contradicted by it. Only the documentation was changed; no source file was touched.

The audit was carried out by Claude. Both suites were run locally to ground the findings — 65 vitest tests and 31 unittest tests, plus npm run typecheck and npm run build, all green — and every TypeScript example in the two READMEs was extracted and type-checked against js/src/index.ts under --strict. All examples compile as written, so none was changed.

# Claim Evidence it is wrong Fix
1 js/README.md, Configuration: repo is "Validated when the client is built — the only place this package throws" requirePath in js/src/client.ts throws github-docs: … is not a published document and is called by both webUrl and rawUrl. Confirmed against the built package: with documents: ['handbook/rules.md', 'handbook/getting-started.md'], docs.webUrl('nope') and docs.rawUrl('nope') both throw, while docs.fetchMarkdown('nope') returns {status: 'not-listed'}. The same over-claim appears in the requireRepoSlug docstring in js/src/repo.ts; that is source, so it was left alone. The cell now says the value throws there if unusable, and names webUrl/rawUrl as the other throw sites, contrasted with fetchMarkdown's not-listed.
2 python/README.md, Development: python3 -m unittest discover -s tests -v Run from python/ in a fresh clone this collects one error rather than 31 tests, because nothing puts python/src on sys.path. CI installs the package before discovering — .github/workflows/ci.yml: python -m pip install . then python -m unittest discover -s tests -v. PYTHONPATH=src prepended, with a comment naming pip install -e . as the alternative. Verified: 31 tests, OK.
3 Root README, Development: cd js && … followed by cd python && … Both lines are written to be run from the repo root, but in one shell the second cd python resolves to js/python and fails. The Python line also carries the sys.path problem above. Each line is now a subshell, and the Python one sets PYTHONPATH=src. Both were run from the repo root exactly as written and both pass.

Claims that were checked and found accurate, and therefore left alone: the package names against js/package.json name and python/pyproject.toml name; Node 18+ and ESM against engines.node, "type": "module" and the CI matrix (18, 20, 22); Python 3.9+ against requires-python and the CI matrix (3.93.13); every documented npm script against package.json scripts; every name in the js README's API listing against the re-exports in js/src/index.ts (all 28 present); the whole DocsClientConfig defaults table against createDocsClient (ref HEAD, transport api-with-token / raw-without, timeoutMs 5000, maxDocumentBytes 1048576, apiBase, fetchImpl, and the unset-vs-empty documents distinction); the MarkdownFetch union against client.ts; the resolveLink worked example, which was executed and returns exactly the URL the README prints; the whole GitHubDocsConfig table against the dataclass defaults in python/src/github_docs/client.py, including all three message templates verbatim; every method in the Python API listing against the class; the four numbered steps of "What a save actually does" against save_file; the docs-edit/<slugified-path> branch name against branch_name_for and slugify_path; and the "standard library only" claim, dependencies being empty in pyproject.toml and the only HTTP import being urllib.


drafted by Claude on behalf of Daniel Stephenson

🤖 Generated with Claude Code

https://claude.ai/code/session_019KfUfNYTjPUrGcJnS5Ha23

- js/README.md: "Validated when the client is built - the only place this
  package throws" is wrong. js/src/client.ts's `requirePath` throws
  `github-docs: ... is not a published document` from `webUrl` and `rawUrl`
  when the target is not in the catalogue. Verified against the built package:
  `createDocsClient({repo:'acme-guild/handbook', documents:['handbook/rules.md']})
  .webUrl('nope')` throws, while `.fetchMarkdown('nope')` returns
  `{status:'not-listed'}`.

- python/README.md and the root README: `python3 -m unittest discover -s tests -v`
  could not work as written from a fresh clone -- nothing puts python/src on
  sys.path, so discovery collects one error instead of 31 tests. CI installs the
  package first (.github/workflows/ci.yml: `python -m pip install .`). Added
  PYTHONPATH=src; 31 tests pass.

- Root README: both development lines were written to run from the repo root,
  but `cd python` after `cd js` resolves to js/python. Wrapped each in a
  subshell; both verified from the repo root.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019KfUfNYTjPUrGcJnS5Ha23
@dmccoystephenson
dmccoystephenson merged commit ef9041c into main Sep 2, 2026
16 checks passed
@dmccoystephenson
dmccoystephenson deleted the docs/accuracy-audit-readme-claims branch September 2, 2026 05:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant