Skip to content

Fix Zotero silently disappearing as a citation source - #1120

Open
juliasilge wants to merge 3 commits into
mainfrom
zotero-restore-retry
Open

Fix Zotero silently disappearing as a citation source#1120
juliasilge wants to merge 3 commits into
mainfrom
zotero-restore-retry

Conversation

@juliasilge

@juliasilge juliasilge commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Fixes #1101

Zotero could disappear as a citation source in the visual editor, without any error message. This affected the local library, and in one case, newly added items. Four separate bugs caused this, all of which are addressed here:

  1. Config push raced LSP startup. The initial Zotero configuration push to the language server could race the server's own startup. This was a regression in 1.136.0, when LSP startup became lazy. A failed push now retries. The Quarto output channel logs each failure. If the sync still fails, Quarto shows a warning message with a retry option.
  2. Config changes did not retry on failure. A change to Quarto > Zotero > Library, the data directory, or the API key pushed the new config one time only. On failure, the extension discarded the result. The LSP then kept the old settings for the rest of the session, with no warning. This now uses the same retry path as the initial sync.
  3. Concurrent lookups raced on the same cached database copy. The citation picker sends getCollections and getActiveCollectionSpecs at the same time. Both requests copy the local Zotero database to the same cache path. A failed open in one request could delete the copy that the other request still needed. As a result, both requests failed with unable to open database file. Access to the cached copy is now one request at a time, per data directory.
  4. The local database reader could not open a WAL-mode library. Current Zotero versions (9 and 10) use WAL mode by default. node-sqlite3-wasm cannot open a WAL-mode database. We now clear the WAL flag in the file header of the throwaway copy before we open it.

Changes I decided NOT to make here

These are remaining problems, but I think too risky to try to address now to fix the 1.136.0 regression:

  • Recreate the LSP client after a startup failure. An automated review found a problem inside the vscode-languageclient library itself. Its LanguageClient.start() method saves its internal start promise and never clears it after a StartFailed state. Our retry logic cannot recover from this specific failure for that reason. A correct fix needs a new LanguageClient instance. That change touches shared LSP startup code for every LSP feature, not only Zotero. Let's leave this alone for now as it is a bigger and more risky change than the rest of this PR.
  • Register LSP custom methods earlier during server startup. The client reports the state "Running" as soon as the LSP handshake completes, but the server does not register its custom RPC methods, including the Zotero config push, until later in its own onInitialized handler. An earlier registration can shrink this race, or maybe remove it, at the source. This also touches shared LSP startup code so let's leave it for now. The retry from fix 1 already covers the common case.
  • Read Zotero's -wal journal file. Zotero writes new and changed items to a -wal file first. Zotero moves this data into the main zotero.sqlite file only at a checkpoint. Our fix makes the main file open again, but an item added or edited in the Zotero desktop app still does not appear in the citation picker until Zotero checkpoints on its own. A correct fix needs either the sqlite3 command-line tool to force a checkpoint, or a different SQLite library that supports WAL mode natively. Both options are bigger changes than this PR should take on; see the separate issue Local Zotero library: new or edited items do not appear until Zotero checkpoints its WAL file #1119.

Manual validation

I tested this against a real local Zotero 10 library:

Screenshot 2026-09-09 at 12 49 52 PM
  1. Insert Citation shows the local library as a source. The inserted citation shows the correct author, year, and title.
  2. Open the citation picker many times and insert several citations in a row. No unable to open database file error appears in the Quarto output channel.
  3. Change Quarto > Zotero > Library between none and local while the extension was running. Each change syncs the config correctly.
  4. A full reload of the window, a cold start, still activates Zotero correctly.
  5. Known limitation, not fixed in this PR, is that an item added or edited in the Zotero desktop app does not appear in the citation picker until Zotero checkpoints its WAL file on its own.

@posit-snyk-bot

posit-snyk-bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@juliasilge
juliasilge marked this pull request as ready for review September 9, 2026 20:03
@juliasilge
juliasilge requested a review from jmcphers September 9, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Zotero <-> Visual Editor link ?broken in 1.136.0

2 participants