Skip to content

feat(cli): Verified BibTeX generation via papi show -l bib #8

Description

@hummat

Problem or Motivation

No easy way to get correct BibTeX for papers in the database. Users manually construct citations or copy from external sources, risking stale arXiv entries (paper since published at a venue), wrong DOIs, missing pages/venue fields, or author mismatches.

Complements existing papi add --from-file library.bib (import) to complete the import/export story — but with verification against authoritative metadata APIs.

Inspired by audit_bib.py which verifies .bib files against Crossref, OpenAlex, and Semantic Scholar.

Proposed Solution

CLI Surface

Integrate as a new show level (consistent with existing meta/summary/equations/tex/tldr):

# Print verified BibTeX to stdout (generates + caches on first access)
papi show paper1 --level bib
papi show paper1 paper2 -l bib

# Force re-fetch from APIs (ignore cached ref.bib)
papi show paper1 -l bib --refresh

Storage

Cache verified BibTeX as ref.bib per paper:

~/.paperpipe/papers/{name}/
├── meta.json
├── paper.pdf
├── ref.bib          # ← NEW: verified BibTeX entry
├── summary.md
└── ...

Generated lazily on first papi show -l bib. Subsequent calls return the cached file.

API Verification Pipeline

Query three sources (in order of reliability) to build the most accurate BibTeX:

Source Lookup method Checks Rate limits
Crossref DOI → metadata Title, authors, year, venue, pages Generous (with mailto header)
OpenAlex DOI or title search Title, authors, year, venue, arXiv staleness 100k/day
Semantic Scholar DOI or title search Title, authors, year 1 RPS (fallback only)

Pipeline per paper:

  1. Read meta.json for arxiv_id, doi, title, authors
  2. If DOI exists → query Crossref for canonical metadata
  3. Query OpenAlex (by DOI or title) for venue, year, arXiv staleness detection
  4. If Crossref + OpenAlex miss → fall back to Semantic Scholar
  5. arXiv auto-upgrade: If paper is arXiv but OpenAlex/Crossref finds a published venue + DOI, generate BibTeX for the published version (note in output: % upgraded from arXiv to CVPR 2024)
  6. Format as BibTeX, write to ref.bib

Fallback

If all APIs fail (no network, paper too new), generate BibTeX from local meta.json only — still useful, just unverified. Mark with % unverified: generated from local metadata only.

Architecture

New module paperpipe/bib.py containing:

  • API query logic (Crossref, OpenAlex, S2) — urllib only, no new deps
  • Title matching / venue alias resolution (ported from audit_bib.py)
  • BibTeX formatting
  • Entry type selection (@inproceedings / @article / @misc)

Example Output

% verified via Crossref (doi:10.48550/arXiv.2106.09685)
% upgraded from arXiv to ICLR 2022
@inproceedings{lora,
  author    = {Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},
  title     = {{LoRA}: Low-Rank Adaptation of Large Language Models},
  booktitle = {International Conference on Learning Representations},
  year      = {2022},
  doi       = {10.48550/arXiv.2106.09685},
  url       = {https://openreview.net/forum?id=nZeVKeeFYf9}
}

Alternatives Considered

  • Separate papi bib subcommand: More room for flags but adds CLI surface area. show -l bib is consistent with existing levels.
  • Always fetch on-the-fly: Slow, wasteful. Caching as ref.bib is better.
  • Store in meta.json: Clutters the metadata file; .bib is its own format.
  • Format from local metadata only (no API): Original scope of this issue. Works for basic export but misses the key value: correctness, venue detection, arXiv upgrade.

Area

CLI commands

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions