Skip to content

feat: list pending data migrations, run one by version, and mark repeatable ones - #6

Merged
tw00 merged 3 commits into
mainfrom
tw/pending-and-run-by-version
Sep 25, 2026
Merged

tw00 merged 3 commits into
mainfrom
tw/pending-and-run-by-version

Conversation

@tw00

@tw00 tw00 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Moves what tv-labs/platform#10396 built inside Sauron into the library, per review there, and adds the one-shot/repeatable annotation.

Summary

  • DataMigration.pending(repo, paths) lists the data migrations that have not run, oldest first, as {version, name}. It reads schema_migrations without the migration lock and without creating the table.
  • DataMigration.run(repo, version, paths, opts \\ []) runs that one data migration and no other. It runs nothing and returns {:error, :not_found} for a version no file has, {:error, :duplicate_version} for one two files share, and {:error, :already_applied} for a one-shot data migration that has run. opts go to Ecto.Migrator.up/4.
  • use DataMigration, repeatable: true in place of use Ecto.Migration marks a data migration run/4 may run again. Anything else, including a plain use Ecto.Migration, is one-shot. Either kind is pending until it has run once. A rerun deletes the version's schema_migrations row (the same delete Ecto.Migrator.down/4 makes, without running down/0), then runs up/0. If that raises, the row stays deleted and the migration shows as pending.
  • Test harness: test_helper.exs puts the sandbox in manual mode, and ConnCase stops its owner. In the default auto mode, a process a test starts (a LiveView, the migrator's task) took a connection of its own, and its writes committed. The Postgres, MySQL and MSSQL test repos now set migration_lock: false, as the SQLite one already did: in manual mode the lock holds the test's connection while the migrator's task waits for it. The run/4 tests pass skip_table_creation: true, since MySQL commits the open transaction on the migrator's CREATE TABLE IF NOT EXISTS.
  • Version 0.2.0, CHANGELOG under unreleased, README section. Not published: ./bin/release 0.1.2 0.2.0 after merge fills in the date.

The dashboard's "Migrate up" still runs a data migration once. Rerunning a repeatable one from the page can come later.

Evidence

  • mix test --warnings-as-errors (Elixir 1.18.4, OTP 27): 32 tests, 0 failures (22 before, 10 new) on SQLite and on Postgres (a local server on another port), each on several seeds. CI runs all four adapters and dialyzer: green on 727aba7.
  • The leak: after one mix test on main, the local test.db keeps the page test's 99999999999999 row in schema_migrations. On this branch it has 0 rows after a run, on SQLite and Postgres.
  • mix compile --warnings-as-errors, mix format --check-formatted and mix dialyzer (0 errors) are clean. On Elixir 1.20 the only warnings are the existing HEEx ones in page.ex:86 and page.ex:220.
  • Sauron (tv-labs/platform#10396) compiled against this lib/data_migration.ex with its three repeatable shims marked: its data migration, Release and admin overview tests pass, 248/248.

🤖 Generated with Claude Code

tw00 and others added 2 commits September 24, 2026 22:37
…le ones

DataMigration.pending/2 lists the data migrations that have not run,
oldest first. DataMigration.run/4 runs one by version and no other,
returning an error and running nothing for a version no file has, a
version two files share, or a one-shot data migration that has run.
`use DataMigration, repeatable: true` marks one that run/4 may run
again; everything else is one-shot.

The test sandbox now runs in manual mode and stops each owner, so a
migration a test runs no longer commits its schema_migrations row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…st repos

With the sandbox in manual mode, the lock held the test's connection in
the process that called the migrator, and the migrator's task timed out
waiting for it. The SQLite repo already had it off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MySQL commits the open transaction on DDL, so the migrator's CREATE
TABLE IF NOT EXISTS ended the test's sandbox and its rows committed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread mix.exs
defmodule DataMigration.MixProject do
use Mix.Project
@version "0.1.2"
@version "0.2.0"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or 0.1.3?

@tw00
tw00 merged commit 2483a4b into main Sep 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants