Skip to content

refactor(storage): one add_job builder for every enqueue shape; drop duplicated S3 resolvers - #1785

Merged
pyramation merged 2 commits into
mainfrom
feat/storage-enqueue-residue
Aug 31, 2026
Merged

refactor(storage): one add_job builder for every enqueue shape; drop duplicated S3 resolvers#1785
pyramation merged 2 commits into
mainfrom
feat/storage-enqueue-residue

Conversation

@pyramation

Copy link
Copy Markdown
Contributor

Summary

Storage bucket-creation residue from #1901: three code paths that each did the same thing their own way, collapsed to one. No behavior change intended — the one-door invariant (plugins enqueue or lazily request provisioning; only the reconciler/policy mints and records a physical name) is unchanged, this just stops it being re-implemented per scope.

graphile-bucket-provisioner-plugin had three inline app_jobs.add_job SQL strings — global, database, owner scope — that drifted in whitespace, argument order and max_attempts. They now share one builder, so a scope branch supplies only its payload and identity arguments:

function addJobStatement(payloadEntries: string, identityArgs: string): string {
  return `SELECT (app_jobs.add_job(
      identifier => 'storage:provision_bucket',
      payload => json_build_object(${payloadEntries}),
      queue_name => 'bucket:' || $1::text,
      max_attempts => 25,
      priority => 0${identityArgs}
    )).id AS id`;
}

// database scope
text = addJobStatement(
  `'database_id', $2::uuid, 'id', $1::uuid, 'scope', $3::text`,
  `, db_id => $2, entity_id => $2, organization_id => NULL, entity_type => $3`,
);

graphile-presigned-url-plugin/src/download-url-field.ts carried its own copies of resolveS3 and resolveS3ForDatabase — including a second one whose parameters were in the opposite order — and now imports the canonical pair from ./physical-bucket. Record-once bookkeeping (UPDATE ... SET physical_name = $1 WHERE id = $2 AND physical_name IS NULL) exists once, as recordPhysicalName in graphile-storage-registry, and remains the only writer of that column.

Verification (MinIO, minio/minio:edge-cicd — the CI image)

All three storage doors exercised against a live MinIO:

  • lazy first upload — uploads integration, 2 suites / 51 tests
  • explicit provisionBucket mutation + eager insert-trigger provisioning — graphile-bucket-provisioner-plugin, 8 suites / 123 tests
  • download/presign against the reconciled physical name — graphile-presigned-url-plugin, 7 suites / 86 tests

Plus pnpm build:dev across the workspace and eslint clean on both touched packages.

Paired with constructive-io/constructive-db#3561, which fixes the migration-time partition race the enqueue path lands on (#1918).

Link to Devin session: https://app.devin.ai/sessions/b1df45674ae742bca5cc743c7d41adc2
Open in Devin Desktop: https://app.devin.ai/desktop/session/b1df45674ae742bca5cc743c7d41adc2?variant=devin
Requested by: @pyramation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@blacksmith-sh

This comment has been minimized.

The pgpm-boilerplates template repo gained safegres.config.js and safegres-perf-baseline.json per scaffolded module (constructive-io/pgpm-boilerplates#42, merged after main's last CI run), so every init snapshot in this repo now sees two extra files.
@pyramation
pyramation merged commit 22c0100 into main Aug 31, 2026
20 checks passed
@pyramation
pyramation deleted the feat/storage-enqueue-residue branch August 31, 2026 04:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant