Skip to content

Docs: document native Google Cloud Storage for the Litestream replica - #25

Open
distronode-com wants to merge 2 commits into
Calnode:mainfrom
distronode-com:docs/litestream-gcs
Open

Docs: document native Google Cloud Storage for the Litestream replica#25
distronode-com wants to merge 2 commits into
Calnode:mainfrom
distronode-com:docs/litestream-gcs

Conversation

@distronode-com

Copy link
Copy Markdown

DEPLOY.md documents only s3:// for the backup replica

The backup section covers R2, B2, MinIO, Spaces, Wasabi and AWS, and describes
LITESTREAM_REPLICA_URL as s3://<bucket>/calnode. A GCP deployer therefore has no reason
to know the bundled Litestream speaks GCS natively — which is the simpler option there,
because credentials come from the instance metadata server and there is no access key to
create, store or rotate.

Verified against the litestream binary in the published image, not inferred from its docs:

  • The scheme is gcs://, not gs://. gs:// — the scheme every other Google tool uses
    — is rejected with unknown replica type in config: "" and exit 1, before any network
    call
    , so it presents as neither a credentials nor a bucket problem. That is the detail
    most likely to cost someone an hour, so it leads the section.
  • The bundled /etc/litestream.yml needs no edit. Its endpoint: and region: lines
    expand to empty strings and a gcs replica ignores both.
  • No LITESTREAM_ACCESS_KEY_ID / _SECRET_ACCESS_KEY. One variable is enough.

The trade, stated

That last point has a consequence a deployer can only find by reading the code, so the
section says it: recordingStorage() in internal/handler/livekit_recording.go reuses the
backup bucket for meeting recordings over the S3 API and requires both credential
variables. With a native GCS replica, recording storage is unavailable. It degrades cleanly
— Settings → Storage reports recordings_storage_ready: false and the room's Record button
stays hidden — but it is a choice, so the text points back at the S3-compatible route for
anyone who wants recording.

Scope

Added beside the S3 documentation, which is unchanged. The replica-URL table row gains a
pointer, and the two "Enabling it" steps that assumed five variables name the GCS case in
one clause each. go test ./... green, gofmt -l . empty, go vet ./... clean (docs only).

DEPLOY.md described LITESTREAM_REPLICA_URL as `s3://<bucket>/calnode` and the
backup section covered only S3-compatible providers, so a GCP deployer had no
reason to know the bundled Litestream speaks GCS natively — which is the simpler
option there, because credentials come from the instance metadata server and there
is no access key to create, store or rotate.

Verified against the litestream binary in the published image rather than inferred
from its docs:

- the scheme is `gcs://`. `gs://` — the scheme every other Google tool uses — is
  rejected with `unknown replica type in config: ""` and exit 1, before any
  network call, so it presents as neither a credentials nor a bucket problem.
  That is the one thing most likely to cost someone an hour, so it leads.
- the bundled /etc/litestream.yml needs no edit: its `endpoint:` and `region:`
  lines expand to empty strings and a gcs replica ignores both.
- no LITESTREAM_ACCESS_KEY_ID / _SECRET_ACCESS_KEY are needed.

That last point is also a trade worth stating, and it is the part a deployer can
only discover by looking at the code: recordingStorage() in
internal/handler/livekit_recording.go reuses the backup bucket for meeting
recordings over the S3 API and requires both credential variables, so a native GCS
replica leaves recording storage unavailable. It degrades cleanly — Settings →
Storage reports recordings_storage_ready: false and the Record button stays hidden
— but it is a choice, so the section says so and points back at the S3 route for
anyone who wants recording.

The S3 documentation is unchanged; this is added beside it. The two enabling steps
that assumed five variables now name the GCS case in one clause each.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — two polish items on the recording trade-off.

Reviewed changes

Docs-only addition to DEPLOY.md §6: native GCS Litestream replica (gcs://) for the bundled binary, plus the recording/S3 credential trade-off.

  • Native GCS replica — Documents LITESTREAM_REPLICA_URL=gcs://…, ADC/metadata credentials, and that empty endpoint/region in /etc/litestream.yml are fine.
  • Scheme trap — Calls out gcs:// vs gs:// and the unknown replica type in config: "" failure mode.
  • Recording trade — Explains that recordingStorage() still needs S3 keys, so native GCS leaves recordings unavailable with clean UI degrade.

Technical claims check out against Litestream 0.3.13 (pinned in Dockerfile) and recordingStorage / Settings / room gating. Current litestream.io main docs use gs:// for v0.5.x — this PR correctly tracks the shipped version, not upstream latest.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Grok𝕏

Comment thread DEPLOY.md Outdated
The native gcs:// section ended by telling anyone who wanted built-in recording
to "choose the S3-compatible route above", but the provider table above has no
GCS row, so that sentence dead-ends exactly the reader it is addressed to.

Spell out the two options instead: move the replica to an S3-API bucket, or
reach the same GCS bucket through Google's S3 interoperability API with HMAC
keys and an s3:// URL. The second is the ordinary S3-compatible route with GCS
as the provider rather than anything new, and it is marked as untested by us
rather than presented as a verified configuration, because we run native gcs://
with recordings off.

Also points at docs/VIDEO.md section 2, which already enumerates what recording
needs from either route.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Incremental delta since the prior pullfrog review (490e7ad417c7aa): the recording-trade dead-end for GCP deployers.

  • Two concrete options — Replaces “choose the S3-compatible route above” with (1) an S3-API bucket among the documented providers, or (2) the same GCS bucket via Google’s S3 interoperability API (HMAC keys, LITESTREAM_ENDPOINT=https://storage.googleapis.com, s3:// replica URL).
  • Honest verification bound — Marks GCS interop + recording as unexercised rather than implying a verified matrix row.
  • Cross-link — Points at docs/VIDEO.md §2 for the three env vars recording actually needs.

Prior inline feedback is addressed. Claims match recordingStorage() and VIDEO.md. No new concerns on this delta.

Pullfrog  | View workflow run | Using Grok𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant