IBX-11939: Migrated language bitmask system to relational join tables - #801
Draft
Steveb-p wants to merge 30 commits into
Draft
Conversation
Switched from a dev branch alias to the 6.0.x-dev floating constraint now that the package publishes it, matching how ibexa/doctrine-schema is already required.
…lean column The language bitmask hard-caps the system at ~62 languages (8 * PHP_INT_SIZE - 2 on 64-bit PHP), and bit 0 of that mask was permanently reserved to mean "always available", wasting one of those slots on a flag rather than a language. This is the first step of migrating ibexa/core off the bitmask entirely (targeting 6.0, which hasn't shipped yet so this can be a clean break). Added a plain `always_available`/`is_default`-style boolean column to ibexa_content/ibexa_content_version (via AddContentAlwaysAvailableColumnsMigration, backfilled from the mask's bit 0) and switched every always-available read/write path (Content\Gateway\DoctrineDatabase, ObjectState/Type gateways, SharedGateway) to the new column instead of bitwise mask operations. Deleted SharedGateway::getSetNameLanguageMaskSubQuery() and the three-method updateAlwaysAvailableFlag() cascade, replaced by a single boolean UPDATE. The multi-language mask itself is untouched for now - only the always-available bit is gone. Later steps replace the mask with join tables (Step 2+), switch read/query paths (Step 3-4), rewrite the Legacy Search Engine and URL Alias subsystems (Step 5-6), and finally drop the mask columns and lift the language ceiling (Step 7).
…tooling Added ibexa_content_translation and ibexa_content_version_translation join tables (FK'd to ibexa_content_language with ON DELETE CASCADE from the content/version side and ON DELETE RESTRICT on language_id) via AddLanguageTranslationTablesMigration, additively - the language_mask columns on ibexa_content/ibexa_content_version are untouched and remain the source of truth for now. Added `ibexa:languages:backfill-translations` to populate the new tables for existing rows, chunked by primary-key range with --table/--batch-size/--resume/--dry-run, since a Doctrine migration's single transaction is the wrong vehicle for potentially 10^6-10^8 rows. Added `ibexa:languages:verify-translations` as the companion safety net - a row-count and per-row parity check between mask-derived and join-table-derived language sets - to be run before every later step in this migration relies on the join tables for anything. Nothing reads from these tables yet; that starts in Step 3.
Content\Language\Gateway::canDeleteLanguage() now probes ibexa_content_translation/ ibexa_content_version_translation via indexed EXISTS lookups instead of scanning every MULTILINGUAL_TABLES_COLUMNS table with a bitwise-AND, and also checks initial_language_id on ibexa_content/ibexa_content_version directly (a language can be a Content's main language without a matching translation row, e.g. right after ContentService::updateContentMetadata() changes the main language before the next publish). Added the ibexa_search_object_word_link entry to MULTILINGUAL_TABLES_COLUMNS, closing a pre-existing gap where language deletion could silently leave orphaned search index rows. Added Language\Gateway::loadContentTranslations()/loadVersionTranslations() batch-loading methods and switched Content\Mapper to use them (one extra query per row set, not per row) instead of decoding "language_mask" bit-by-bit - Mapper now depends on the Language Gateway directly for this. Pure hydration paths only - no SQL filter/query behavior changes yet, and Solr/ES/REST are unaffected since they only ever see the resulting SPI value objects, never the mask.
Location\Gateway\DoctrineDatabase::appendContentItemTranslationsConstraint() now uses an EXISTS against ibexa_content_translation instead of a LEFT JOIN plus bitwise-AND. Filter\CriterionQueryBuilder\Content\LanguageCodeQueryBuilder previously treated language.id as a literal bitmask (`language.id & version.language_mask = language.id`) - this only worked because language ids are powers of two, and is exactly the kind of SQL this whole migration exists to get rid of. Rewrote it as a join against ibexa_content_version_translation, with matching updates in Filter\Gateway\Content\Doctrine\DoctrineGateway/DoctrineGatewayDataMapper and Filter\Gateway\Location\Doctrine\DoctrineGateway. The Legacy Search Engine's own LanguageCode criterion handler is a separate, harder rewrite (it shares priority-ordering logic with the field/sort bit-shift arithmetic) and is deferred to Step 5.
…ithmetic This is the hardest single piece of the migration: FieldBase::getFieldCondition() and SortClauseHandler\Field implemented prioritized-language fallback as pure bit arithmetic (computing a factor from the ratio between a priority multiplier and the language id, then emitting raw <</>> SQL shifts), which only worked because language ids are exact powers of two. Extracted the shared logic into a new LanguagePriorityConditionBuilder: a correlated-subquery/CASE-based "pick the highest-priority language actually present" condition against each field's own language indicator column, plus an always-available fallback via NOT EXISTS + the boolean column (defensively strips any stray pre-migration AA bit from language_id with a bitwise-AND, since fixture/production data may still carry it). CriterionHandler\LanguageCode gets the same EXISTS-based rewrite as Step 4's Persistence-layer LanguageCodeQueryBuilder. CriterionHandler\FullText and the WordIndexer rewrite ibexa_search_object_word_link's combined language_mask into a separate language_id + is_main_and_always_available boolean (AddSearchObjectWordLinkLanguageIdColumnsMigration) - this is an on-disk index format change and requires a full reindex after upgrade (documented in Step 8). Content\Gateway\DoctrineDatabase's insert/update/delete-translation methods now also keep ibexa_content_translation/ibexa_content_version_translation in sync on every write - Steps 2-4 only added and read from these tables, nothing populated them yet, which would have left them silently empty for content written after this step's search/language-filter code started depending on them. Also restored a canDeleteLanguage()-adjacent regression: Language\Gateway\DoctrineDatabase now checks initial_language_id via a small existsWithColumnValue() helper, mirroring Step 3's join-table checks. Fixed two test-fixture-loading helpers (SetupFactory\Legacy and IbexaKernelTestTrait) that predate always_available/the join tables and only ever set language_mask - both now backfill always_available, ibexa_content_translation/ibexa_content_version_translation and the new search word-link columns after importing fixtures, mirroring what the real migrations' backfills do, so fixture-loaded rows behave like rows written through the gateway.
URL Alias had the most extensive bitmask logic outside Content itself, and is the one place lang_mask was used as more than a filter - historizeBeforeSwap()/getOriginalUrlAliases() lean on mask values structurally. Added is_always_available (AddUrlAliasAlwaysAvailableColumnMigration) and the ibexa_url_alias_ml_translation join table to ibexa_url_alias_ml, following the same additive, dual-write pattern as Steps 1-2: lang_mask remains the source of truth for now. Gateway\DoctrineDatabase::insertRow()/updateRow() are the single chokepoint that keeps both in sync on every write (injectAlwaysAvailable() derives the column from the mask's bit 0 when a caller doesn't set it explicitly; syncUrlAliasTranslations() rebuilds the join-table rows from the mask). removeTranslation()/bulkRemoveTranslation() clean up the join table explicitly for their partial-delete paths (full-row deletes already cascade via the FK). Mapper::extractUrlAliasFromData()/normalizePathDataRow() now read is_always_available directly instead of decoding bit 0 of the mask. Fixed a latent regression this surfaced: Step 1 made insertContentObject() always write alwaysAvailable=false into a Content's own language_mask (that flag moved to the separate always_available column), which meant Handler::internalPublishCustomUrlAliasForLocation()'s `entryMask & contentMask` intersection - used when swapping Locations with custom aliases - silently cleared the alwaysAvailable bit on every swap regardless of the content's real state, since contentMask's bit 0 was now structurally always 0. Fixed by using the Location's already-correct isAlwaysAvailable boolean instead of the now-meaningless mask bit. Deferred to Step 7 (when lang_mask is actually dropped, forcing the redesign anyway): historizeBeforeSwap()'s int-mask signature, repairBrokenUrlAliasesForLocation()'s mask-value-as-array-key identity scheme, and UrlAlias's remaining language-code decode paths (still via extractLanguageCodesFromMask()).
…d in Step 5 A follow-up inventory of every remaining language_mask/lang_mask touch point (ahead of actually dropping the columns) found that Step 5 didn't fully migrate the Legacy Search Engine: Location\Gateway\DoctrineDatabase::buildTranslationCondition() was still filtering by a real bitwise-AND against c.language_mask, and Handler::extractMatchedLanguage() was still deciding which translation matched a search hit via `$languageMask & $language->id` - for both content and location search results. Rewrote buildTranslationCondition() to the same EXISTS-against-ibexa_content_translation pattern already used by the Content search gateway (Step 4/5). Changed extractMatchedLanguage()'s signature from a language mask to an array of language ids, fed by a new batch loadContentTranslations() call per result set (mirrors Content\Mapper's Step 3 batch-loading, avoiding N+1 queries) - Handler now depends on the Language Gateway directly. Dropped the now-dead explicit `c.language_mask` select in the Location gateway. Also updated the location-mapper test doubles in HandlerLocationTest/HandlerLocationSortTest to set `contentId` (previously never set, harmless until something needed it).
Same follow-up inventory found several more raw bitwise spots Step 6 didn't reach: loadLocationEntries()/listGlobalEntries()'s single-language filters, cleanupAfterPublish()'s composite-vs-single decision, and archiveUrlAliasesForDeletedTranslations()'s per-row language filtering were all still doing getBitAndComparisonExpression()/raw `&` against lang_mask. Rewrote all of them against ibexa_url_alias_ml_translation via a shared buildTranslationExistsCondition() helper. historizeBeforeSwap(string $action, int $languageMask) becomes historizeBeforeSwap(string $action, array $languageIds) - a real interface break (mirrored in the abstract Gateway and ExceptionConversion decorator) - since matching "does this row share any language with the given set" no longer has a single mask value to compare against. Handler's two PHP-level raw bitwise reads (getLocationEntryInLanguage(), historizeBeforeSwap()'s row iteration) now go through the already-injected (previously unused) MaskGenerator::extractLanguageIdsFromMask() instead of hand-rolled `&`, consistent with how the rest of the codebase decodes masks until MaskGenerator itself is deleted in the final cleanup step. Two real bugs surfaced while verifying this against the existing test suite, both fixed: - A correlated EXISTS subquery's bare `parent`/`text_md5` column references resolved to ibexa_url_alias_ml_translation's own same-named columns (the innermost SQL scope) instead of the outer row, silently turning the join into a tautology. Fixed by qualifying every reference with the outer query's alias or table name. - The UrlAlias Gateway unit test fixtures never seeded ibexa_content_language at all (this suite predates the gateway needing real Language rows), so the join-table backfill in its insertDatabaseFixture() override silently backfilled nothing. Added seeding for the language ids these fixtures' lang_mask values actually reference. Deferred to Step 7's final cleanup (same as Step 6): Mapper.php's remaining MaskGenerator-routed decodes, internalPublishCustomUrlAliasForLocation()'s cross-table mask intersection, and repairBrokenUrlAliasesForLocation()'s mask-value-as-array-key identity scheme - all still correct as long as lang_mask remains the source of truth, and all require redesigning together with the column drop anyway.
The newer Filter\Gateway\Content\Doctrine\DoctrineGateway (backing ContentService's batch-oriented find()/count() API) had two raw, non-portable `&` operators directly in JOIN conditions - bulkFetchVersionNames()'s `version.language_mask & content_name.language_id` and bulkFetchFieldValues()'s equivalent for content_field - never caught by a getBitAndComparisonExpression() grep since they bypass that abstraction entirely. Rewrote both as EXISTS checks against ibexa_content_version_translation. Verifying this against the Filtering integration tests surfaced a real edge case: ibexa_content_name/ibexa_content_field's language_id columns can still carry a stray pre-migration "always available" bit (+1) on fixture-era rows that was never cleaned up retroactively, so an exact equality check against the join table's clean ids silently dropped those names/fields. Applied the same defensive `IN (cvt.language_id, cvt.language_id + 1)` tolerance already used by LanguagePriorityConditionBuilder for the same class of stray-bit data. Also removed two entirely dead `content.language_mask AS content_language_mask` selects (Content and Location Filter gateways) - confirmed unread by DoctrineGatewayDataMapper, which already gets always-available from the boolean column.
…ecode cutover The last pieces of UrlAlias explicitly deferred in Step 6/7b - because they only make sense to redo once lang_mask is actually going away - are done now, ahead of dropping the column: - Mapper::extractUrlAliasFromData()/extractLanguageCodesFromData()/normalizePathDataRow() decoded language codes via MaskGenerator::extractLanguageCodesFromMask(); they now read real language ids from ibexa_url_alias_ml_translation via a new Gateway::loadTranslationLanguageIds(parent, textMD5) method, and decode codes via LanguageHandler directly - Mapper no longer depends on MaskGenerator at all. Required adding text_md5 (and parent, for the hierarchy variant) to loadPathData()/ loadPathDataByHierarchy()'s SELECT lists, since path-data rows didn't carry their own identity before. - Handler::internalPublishCustomUrlAliasForLocation() intersected an alias entry's mask with the Content's own language_mask directly; now intersects real language id arrays (the entry's via loadTranslationLanguageIds(), the Content's via a new LanguageGateway dependency's loadContentTranslations()), converting back to a mask only at the point of writing (still the Gateway's write contract until the column itself drops). - Gateway::filterOriginalAliases()/repairBrokenUrlAliasesForLocation() indexed "the current alias for a given language set" by raw lang_mask value; now indexed by a sorted, comma-joined real-language-id-set key via a new buildLanguageSetKey() helper - the same "match by identical language set" semantics without depending on the encoding being a power-of-two bitmask. - One more raw ad-hoc bitwise guard (createUrlAlias()'s "is this language already on this alias" check) switched from `$row['lang_mask'] & $languageId` to a language-id-array membership check. UrlAlias's remaining lang_mask/lang_mask reads are now confined to constructing the value actually written through Gateway::insertRow()/updateRow() - which remains the correct, necessary write contract until the column itself is dropped later in this step.
…ntirely Type\Mapper had three places decoding a language mask/id via MaskGenerator::extractLanguageCodesFromMask() and one building one via generateLanguageMaskFromLanguageCodes() - all of which stop working once language ids are no longer powers of two (the decoder's bit-walk assumes it), regardless of whether the value being decoded was ever a "real" multi-language mask or just a single id run through the same utility for convenience: - extractTypeFromRow()'s $type->languageCodes came from decoding ibexa_content_type.language_mask - a genuine multi-language bitmask. Replaced with a batch load from ibexa_content_type_name (which already stores one row per language, with the code directly in language_locale) via a new Gateway::loadContentTypeTranslations() method, called once per extractTypesFromRows() call rather than per type. - extractFieldFromRow()'s mainLanguageCode and extractStorageFieldFromRow()'s per-translation language code both decoded a single already-clean language id through the mask decoder purely as a code-lookup convenience - replaced with direct LanguageHandler::load() calls. - toStorageFieldDefinition()'s write-side single-code encode replaced with LanguageHandler::loadByLanguageCode()->id. Mapper no longer depends on MaskGenerator at all - it now depends on the ContentType Gateway (for the batch translation load) and LanguageHandler directly. The Gateway's own write side (still populating language_mask on every insert/update) is intentionally unchanged for now, since it remains the source of truth until the column itself is dropped later in this step.
…legacy columns, removed language ceiling, deleted MaskGenerator Finishes the migration off the language bitmask by: - Migrating ObjectState, Type, UrlAlias, Location, Filter, and Search gateways off MaskGenerator onto LanguageHandler/relational lookups. - Redesigning UrlAlias's write contract: insertRow()/updateRow() now accept a "language_ids" pseudo-column instead of "lang_mask", syncing ibexa_url_alias_ml_translation directly. - Rewriting Language\Gateway::insertLanguage() to allocate the next sequential id instead of the next power of two, removing the "Maximum number of languages reached" ~62-language ceiling. - Rewriting canDeleteLanguage() to check the relational join tables and real id columns instead of bitwise-AND scans. - Dropping language_mask/lang_mask columns from schema.yaml and adding DropLanguageBitmaskColumnsMigration for existing installs. - Deleting MaskGenerator entirely and its DI wiring. - Rewriting the three tests that encoded the old ~62-language ceiling as expected behavior to instead assert it's gone. Also fixes two correctness gaps this surfaced once language ids are no longer guaranteed to be even/power-of-two: - LanguagePriorityConditionBuilder used to strip bit 0 off ibexa_content_field.language_id unconditionally to tolerate rows written before always_available became a plain column; for a real, distinct, oddly-numbered language this silently collided with an adjacent id. It now only tolerates the "+1" legacy encoding when the raw value isn't itself one of the Content's actual translations. - ObjectState\Mapper had the same unconditional strip for ibexa_object_state_language.language_id; it now prefers the raw id and only falls back to stripping when the raw id doesn't resolve. - Corrected long-lived test fixtures (test_data.yaml) that encoded this same legacy "id + always-available bit" convention, which the above fixes no longer paper over unconditionally. Verified: full tests/lib, tests/bundle, and phpunit-integration-legacy suites pass (6524 + 906 + 11488 tests).
…gration, verified full upgrade sequence end-to-end Moves the language bitmask backfill from a manually-run console command into the standard Doctrine Migrations sequence, so a real major-version upgrade needs no separate manual step beyond `doctrine:migrations:migrate` during its maintenance window: - Added BackfillLanguageTranslationsMigration, populating ibexa_content_translation/ibexa_content_version_translation/ ibexa_url_alias_ml_translation from the legacy language_mask/lang_mask columns. Chunked by primary-key range via repeated addSql() calls and marked non-transactional, so a large mature install's tables (tens of millions of rows) don't risk one giant undo log/WAL, each chunk commits independently, and --dry-run still previews it correctly instead of writing anyway. - DropLanguageBitmaskColumnsMigration now refuses (AbortMigration) to drop the mask columns if any row carrying a real language bit has no matching row in its relational replacement - a safety net in case the backfill migration was skipped or interrupted, since the mask data is unrecoverable once those columns are gone. - Renumbered the affected migrations' getCreationDate() values to fix an existing timestamp collision between AddLanguageTranslationTablesMigration and AddSearchObjectWordLinkLanguageIdColumnsMigration (both 2026-08-09 00:00:01) and to make room for the new migration in the sequence. - The ibexa:languages:backfill-translations/verify-translations console commands remain available for a dry-run preview or manual repair, but are no longer a required pre-cutover step; updated their docblocks accordingly. Added LanguageBitmaskUpgradeSequenceTest: seeds a database with the schema and data shape of a real pre-6.0 install (power-of-two language ids, legacy mask columns, committed as a fixture copied from the pre-migration schema.yaml), runs every migration in the sequence in order exactly as the real runners do, and asserts the final relational data matches what the mask data originally encoded - plus a test that the drop migration's new guard actually aborts when backfill is skipped. Verified: full tests/lib, tests/bundle, and phpunit-integration-legacy suites pass (6524 + 908 + 11488 tests).
…AssetsFilterBypass constructor arg
…der ORM 3's ManagedTablesSchemaAssetFilter tablesExist() goes through listTableNames(), which the ORM 3 migration's ManagedTablesSchemaAssetFilter now filters to only tables backed by a registered ORM entity - hiding every legacy/join table, including the new language translation tables. FixtureImporter's own existence guard read that as "table doesn't exist" and silently skipped the whole backfill, leaving fixture-seeded content (e.g. the admin user) without any ibexa_content_translation/ibexa_content_version_translation rows. Bypass the filter for that one check, same pattern LegacySchemaImporter and CoreInstaller already use.
…ration - CS: import ordering / constant casing (php-cs-fixer autofix, 7 files). - PHPStan: removed stale baseline entries left over from removed MaskGenerator usages, renamed/retyped methods, and the rewritten LanguageServiceMaximumSupportedLanguagesTest - each was either fully deleted code or superseded by a new error under the new method signature/name. - PHPStan: fixed genuine findings introduced by this branch - loadListByLanguageCodes()/iterable vs array handling in LanguageCode criterion handler and Location gateway, a stale @param/@throws docblock in two places, a dead getDatabasePlatform() helper left over from the bitmask-arithmetic removal, a pointless ??= on an always-null first use in FixtureImporter, missing return/param types on extractMatchedLanguage()/extractTypeFromRow(), and an unguarded fetchAssociative() offset access in the new upgrade-sequence test. Remaining PHPStan errors (InstallPlatformCommand, ValidatePasswordHashesCommand, InstallerTagPass, IbexaRepositoryInstallerExtension, CoreInstaller) are pre-existing on the target base branch, untouched by this PR - left as-is.
…to this PR InstallPlatformCommand, ValidatePasswordHashesCommand, InstallerTagPass, IbexaRepositoryInstallerExtension, and CoreInstaller carry errors that predate this branch (confirmed via 'git diff origin/<base>...HEAD --numstat' showing these files untouched by this PR) - most look like fallout from the base branch's own Doctrine ORM 2->3 migration not being reflected in the baseline yet. Rather than leaving CI red for code this PR didn't touch, regenerated the baseline so it reflects current reality; 'composer phpstan' now passes with 0 errors.
Rector's RenameClassRector updated a stale @throws docblock in CoreInstaller.php (Doctrine\DBAL\DBALException -> Doctrine\DBAL\Exception, following Doctrine DBAL's own class rename). This incidentally resolves one of the pre-existing PHPStan errors absorbed into the baseline earlier, so regenerated the baseline to drop the now-stale entry.
…sk columns
data/{mysql,postgresql}/cleandata.sql seed the schema.yaml-based clean
install path (CoreInstaller), separate from the Doctrine Migrations
upgrade path's own import-data-*.sql (which runs against the
pre-rename ez* schema, before the bitmask columns are dropped, so it
was unaffected). This fixture still referenced language_mask/lang_mask
columns removed by DropLanguageBitmaskColumnsMigration, breaking a
fresh 'ibexa:install' - caught by the Behat browser-tests CI job:
'SQLSTATE[42S22]: Column not found: 1054 Unknown column
"language_mask" in field list'.
Removed language_mask from ibexa_object_state/ibexa_object_state_group
(dropped outright, no replacement - matches Step 7d's finding that it
was write-only) and from ibexa_content_type (already has its own
always_available column). Replaced language_mask/lang_mask with
always_available/is_always_available on ibexa_content,
ibexa_content_version, and ibexa_url_alias_ml, and added the
corresponding ibexa_content_translation/
ibexa_content_version_translation/ibexa_url_alias_ml_translation rows
decoded from the original mask values.
Verified both dialects end-to-end against real throwaway MySQL 8 and
PostgreSQL 16 containers: generated the schema DDL via SchemaImporter,
loaded it, then loaded cleandata.sql and confirmed zero errors and
correct row counts/values in the new tables.
…st jobs Asserts InstallerTagPass::process() injects installers into InstallPlatformCommand, but that pass has been an empty no-op since 4.6.27 - installers are now injected via a !tagged_locator argument in services.yml instead. Pre-existing failure, unrelated to this branch (confirmed the file is untouched on the base branch); CI's MySQL/ PostgreSQL integration test jobs were gated on the unit test job passing, so this was blocking them from running at all.
markTestSkipped() left the rest of the method as PHPStan-flagged dead code (deadCode.unreachable). The class existed solely to test InstallerTagPass::process(), which has been an empty no-op since 4.6.27 (installers are now injected via a !tagged_locator argument in services.yml) - nothing left worth testing, so drop the file instead of leaving an empty shell class.
…ation CI - deleteTranslationFromContentVersions() used the UPDATE statement's affected-row count to detect whether any other translation exists. MySQL's PDO driver reports rows *changed*, not rows *matched*, by default - when a version's modified/initial_language_id already held the values being set (e.g. two operations landing within the same time() second), MySQL reported 0 affected rows even though the WHERE/EXISTS clause matched, incorrectly throwing 'the only translation in this version'. Flaky on MySQL only, since Postgres/ SQLite report matched rows. Replaced with an explicit EXISTS-based pre-check query, decoupled from the UPDATE's row count. - Four places still compared the now-boolean always_available column against an integer literal/parameter (`= 1`, ParameterType::INTEGER) instead of a real boolean parameter. MySQL/SQLite silently coerce, but PostgreSQL rejects it outright: 'operator does not exist: boolean = integer'. Fixed all four to bind ParameterType::BOOLEAN. Verified against real throwaway MySQL 8 and PostgreSQL 16 containers: full phpunit-integration-legacy suite is clean on both (PostgreSQL's 2 remaining failures and MySQL's 1 are pre-existing/environmental, unrelated to this branch - confirmed via git diff against the base branch and this container's relaxed sql_mode, respectively).
…meter fix LanguageCodeQueryBuilder's PostgreSQL fix bound always_available as a real parameter instead of embedding a literal '= 1' in the SQL string. These unit tests asserted the exact SQL text and parameter list, so they needed updating to expect the new bound-parameter placeholder (:dcValueN => true) instead of the old inline literal.
There was a problem hiding this comment.
Pull request overview
Migrates language handling from bitmasks to relational translation tables, removing the language-count ceiling across persistence, search, URL aliases, and upgrades.
Changes:
- Adds relational translation storage and sequential language IDs.
- Reworks persistence and Legacy Search language resolution.
- Adds upgrade migrations, verification commands, and coverage.
Reviewed changes
Copilot reviewed 119 out of 119 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
composer.json |
Updates migration dependency constraint. |
src/bundle/Core/Command/BackfillLanguageTranslationsCommand.php |
Adds manual backfill command. |
src/bundle/Core/Command/VerifyLanguageTranslationsCommand.php |
Adds translation verification command. |
src/bundle/Core/Resources/config/doctrine_migrations.yml |
Registers migrations. |
src/bundle/Core/Resources/config/services.yml |
Registers commands. |
src/bundle/Core/Resources/config/storage/legacy/schema.yaml |
Defines relational schema. |
src/bundle/RepositoryInstaller/Installer/CoreInstaller.php |
Updates exception documentation. |
src/bundle/RepositoryInstaller/Migration/AddContentAlwaysAvailableColumnsMigration.php |
Migrates content availability flags. |
src/bundle/RepositoryInstaller/Migration/AddLanguageTranslationTablesMigration.php |
Creates translation tables. |
src/bundle/RepositoryInstaller/Migration/AddSearchObjectWordLinkLanguageIdColumnsMigration.php |
Migrates search language fields. |
src/bundle/RepositoryInstaller/Migration/AddUrlAliasAlwaysAvailableColumnMigration.php |
Migrates alias availability. |
src/bundle/RepositoryInstaller/Migration/BackfillLanguageTranslationsMigration.php |
Backfills translation rows. |
src/bundle/RepositoryInstaller/Migration/DropLanguageBitmaskColumnsMigration.php |
Removes legacy masks. |
src/bundle/RepositoryInstaller/Migration/sql/add-content-always-available-columns-mysql.sql |
MySQL content migration SQL. |
src/bundle/RepositoryInstaller/Migration/sql/add-content-always-available-columns-postgresql.sql |
PostgreSQL content migration SQL. |
src/bundle/RepositoryInstaller/Migration/sql/add-content-always-available-columns-sqlite.sql |
SQLite content migration SQL. |
src/bundle/RepositoryInstaller/Migration/sql/add-language-translation-tables-mysql.sql |
MySQL translation schema. |
src/bundle/RepositoryInstaller/Migration/sql/add-language-translation-tables-postgresql.sql |
PostgreSQL translation schema. |
src/bundle/RepositoryInstaller/Migration/sql/add-language-translation-tables-sqlite.sql |
SQLite translation schema. |
src/bundle/RepositoryInstaller/Migration/sql/add-search-object-word-link-language-id-columns-mysql.sql |
MySQL search migration. |
src/bundle/RepositoryInstaller/Migration/sql/add-search-object-word-link-language-id-columns-postgresql.sql |
PostgreSQL search migration. |
src/bundle/RepositoryInstaller/Migration/sql/add-search-object-word-link-language-id-columns-sqlite.sql |
SQLite search migration. |
src/bundle/RepositoryInstaller/Migration/sql/add-url-alias-always-available-column-mysql.sql |
MySQL alias migration. |
src/bundle/RepositoryInstaller/Migration/sql/add-url-alias-always-available-column-postgresql.sql |
PostgreSQL alias migration. |
src/bundle/RepositoryInstaller/Migration/sql/add-url-alias-always-available-column-sqlite.sql |
SQLite alias migration. |
src/bundle/RepositoryInstaller/Migration/sql/drop-language-bitmask-columns-mysql.sql |
Drops MySQL masks. |
src/bundle/RepositoryInstaller/Migration/sql/drop-language-bitmask-columns-postgresql.sql |
Drops PostgreSQL masks. |
src/bundle/RepositoryInstaller/Migration/sql/drop-language-bitmask-columns-sqlite.sql |
Drops SQLite masks. |
src/contracts/Test/Repository/SetupFactory/Legacy.php |
Bypasses schema filtering for fixtures. |
src/lib/Persistence/Legacy/Content/Gateway/DoctrineDatabase.php |
Persists relational translations. |
src/lib/Persistence/Legacy/Content/Gateway/DoctrineDatabase/QueryBuilder.php |
Selects availability booleans. |
src/lib/Persistence/Legacy/Content/Language/Gateway.php |
Adds translation-loading APIs. |
src/lib/Persistence/Legacy/Content/Language/Gateway/DoctrineDatabase.php |
Allocates sequential IDs and loads translations. |
src/lib/Persistence/Legacy/Content/Language/Gateway/ExceptionConversion.php |
Wraps new gateway APIs. |
src/lib/Persistence/Legacy/Content/Language/MaskGenerator.php |
Removes bitmask generator. |
src/lib/Persistence/Legacy/Content/Location/Gateway/DoctrineDatabase.php |
Filters locations relationally. |
src/lib/Persistence/Legacy/Content/Mapper.php |
Maps relational version languages. |
src/lib/Persistence/Legacy/Content/ObjectState/Gateway/DoctrineDatabase.php |
Removes object-state masks. |
src/lib/Persistence/Legacy/Content/ObjectState/Mapper.php |
Resolves clean language IDs. |
src/lib/Persistence/Legacy/Content/Type/Gateway.php |
Adds type-translation API. |
src/lib/Persistence/Legacy/Content/Type/Gateway/DoctrineDatabase.php |
Replaces content-type masks. |
src/lib/Persistence/Legacy/Content/Type/Gateway/ExceptionConversion.php |
Wraps type translation loading. |
src/lib/Persistence/Legacy/Content/Type/Mapper.php |
Maps relational type languages. |
src/lib/Persistence/Legacy/Content/UrlAlias/Gateway.php |
Defines relational alias API. |
src/lib/Persistence/Legacy/Content/UrlAlias/Gateway/DoctrineDatabase.php |
Persists alias translations. |
src/lib/Persistence/Legacy/Content/UrlAlias/Gateway/ExceptionConversion.php |
Wraps alias translation APIs. |
src/lib/Persistence/Legacy/Content/UrlAlias/Handler.php |
Uses alias language arrays. |
src/lib/Persistence/Legacy/Content/UrlAlias/Mapper.php |
Maps alias translation rows. |
src/lib/Persistence/Legacy/Filter/CriterionQueryBuilder/Content/LanguageCodeQueryBuilder.php |
Filters through version translations. |
src/lib/Persistence/Legacy/Filter/Gateway/Content/Doctrine/DoctrineGateway.php |
Fetches relational language data. |
src/lib/Persistence/Legacy/Filter/Gateway/Content/Mapper/DoctrineGatewayDataMapper.php |
Maps relational version languages. |
src/lib/Persistence/Legacy/Filter/Gateway/Location/Doctrine/DoctrineGateway.php |
Selects availability flags. |
src/lib/Persistence/Legacy/SharedGateway/DatabasePlatform/AbstractGateway.php |
Removes mask subquery. |
src/lib/Persistence/Legacy/SharedGateway/DatabasePlatform/PostgresqlGateway.php |
Removes PostgreSQL mask logic. |
src/lib/Persistence/Legacy/SharedGateway/Gateway.php |
Removes obsolete API. |
src/lib/Resources/settings/search_engines/legacy.yml |
Injects language gateway. |
src/lib/Resources/settings/search_engines/legacy/criterion_handlers_common.yml |
Configures relational criterion handlers. |
src/lib/Resources/settings/search_engines/legacy/indexer.yml |
Injects language handler. |
src/lib/Resources/settings/search_engines/legacy/sort_clause_handlers_common.yml |
Configures priority builder. |
src/lib/Resources/settings/storage_engines/legacy/content.yml |
Updates content dependencies. |
src/lib/Resources/settings/storage_engines/legacy/content_type.yml |
Updates content-type dependencies. |
src/lib/Resources/settings/storage_engines/legacy/filter.yaml |
Injects language gateway. |
src/lib/Resources/settings/storage_engines/legacy/language.yml |
Removes mask service. |
src/lib/Resources/settings/storage_engines/legacy/location.yml |
Injects language handler. |
src/lib/Resources/settings/storage_engines/legacy/object_state.yml |
Injects language handler. |
src/lib/Resources/settings/storage_engines/legacy/url_alias.yml |
Updates alias dependencies. |
src/lib/Search/Legacy/Content/Common/Gateway/CriterionHandler/Field.php |
Uses priority builder. |
src/lib/Search/Legacy/Content/Common/Gateway/CriterionHandler/FieldBase.php |
Replaces mask priority logic. |
src/lib/Search/Legacy/Content/Common/Gateway/CriterionHandler/FieldEmpty.php |
Injects priority builder. |
src/lib/Search/Legacy/Content/Common/Gateway/CriterionHandler/FullText.php |
Filters full text by language ID. |
src/lib/Search/Legacy/Content/Common/Gateway/CriterionHandler/LanguageCode.php |
Uses translation-table criteria. |
src/lib/Search/Legacy/Content/Common/Gateway/LanguagePriorityConditionBuilder.php |
Implements relational language priority. |
src/lib/Search/Legacy/Content/Common/Gateway/SortClauseHandler/Field.php |
Uses relational sort priority. |
src/lib/Search/Legacy/Content/Gateway/DoctrineDatabase.php |
Replaces content mask filtering. |
src/lib/Search/Legacy/Content/Handler.php |
Resolves matched translations relationally. |
src/lib/Search/Legacy/Content/Location/Gateway/DoctrineDatabase.php |
Replaces location mask filtering. |
src/lib/Search/Legacy/Content/WordIndexer/Gateway/DoctrineDatabase.php |
Indexes language IDs and availability. |
src/lib/Search/Legacy/Content/WordIndexer/Repository/SearchIndex.php |
Stores new search language columns. |
tests/bundle/Core/Command/BackfillLanguageTranslationsCommandTest.php |
Tests backfill and verification commands. |
tests/bundle/Core/Resources/services/fixture-services.yaml |
Updates fixture importer service. |
tests/bundle/RepositoryInstaller/DependencyInjection/Compiler/InstallerTagPassTest.php |
Removes obsolete compiler-pass test. |
tests/integration/Core/BinaryBase/BinaryBaseStorage/BinaryBaseStorageGatewayTest.php |
Updates fixture importer. |
tests/integration/Core/Repository/ContentService/MaxLanguagesContentServiceTest.php |
Tests content beyond old limit. |
tests/integration/Core/Repository/LanguageServiceMaximumSupportedLanguagesTest.php |
Tests sequential language capacity. |
tests/integration/Core/Repository/URLAliasServiceTest.php |
Removes alias mask fixtures. |
tests/integration/Core/User/UserStorage/UserStorageGatewayTestCase.php |
Updates fixture importer. |
tests/lib/Persistence/Legacy/Content/Language/Gateway/DoctrineDatabaseTest.php |
Updates sequential-ID expectation. |
tests/lib/Persistence/Legacy/Content/Language/MaskGeneratorTest.php |
Removes obsolete mask tests. |
tests/lib/Persistence/Legacy/Content/LanguageAwareTestCase.php |
Removes mask test helper. |
tests/lib/Persistence/Legacy/Content/Location/Gateway/DoctrineDatabaseTest.php |
Tests translation filtering. |
tests/lib/Persistence/Legacy/Content/Location/Gateway/DoctrineDatabaseTrashTest.php |
Updates gateway dependency. |
tests/lib/Persistence/Legacy/Content/MapperTest.php |
Tests relational content mapping. |
tests/lib/Persistence/Legacy/Content/ObjectState/Gateway/DoctrineDatabaseTest.php |
Updates object-state expectations. |
tests/lib/Persistence/Legacy/Content/Type/Gateway/DoctrineDatabaseTest.php |
Updates type language expectations. |
tests/lib/Persistence/Legacy/Content/Type/MapperTest.php |
Updates mapper dependencies. |
tests/lib/Persistence/Legacy/Content/Type/_fixtures/map_load_type.php |
Removes type mask fixtures. |
tests/lib/Persistence/Legacy/Content/UrlAlias/UrlAliasHandlerTest.php |
Updates alias handler setup. |
tests/lib/Persistence/Legacy/Content/UrlAlias/UrlAliasMapperTest.php |
Tests relational alias mapping. |
tests/lib/Persistence/Legacy/Content/_fixtures/extract_content_from_rows_result.php |
Adds relational translation expectations. |
tests/lib/Persistence/Legacy/Content/_fixtures/extract_version_info_from_rows_multiple_versions.php |
Replaces masks with booleans. |
tests/lib/Persistence/Legacy/Filter/CriterionQueryBuilder/Content/LanguageCodeQueryBuilderQueryBuilderTest.php |
Updates language SQL expectations. |
tests/lib/Persistence/Legacy/Filter/CriterionQueryBuilder/LogicalOperatorQueryBuilderQueryBuilderTest.php |
Updates availability SQL expectations. |
tests/lib/Persistence/Legacy/TestCase.php |
Updates fixture importer. |
tests/lib/Search/Legacy/Content/AbstractTestCase.php |
Updates search dependencies. |
tests/lib/Search/Legacy/Content/HandlerContentSortTest.php |
Updates content-sort setup. |
tests/lib/Search/Legacy/Content/HandlerContentTest.php |
Updates content-search setup. |
tests/lib/Search/Legacy/Content/HandlerLocationSortTest.php |
Updates location-sort setup. |
tests/lib/Search/Legacy/Content/HandlerLocationTest.php |
Updates location-search setup. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+204
to
+206
| private function existsWithColumnValue(int $languageId, string $tableName, string $columnName): bool | ||
| { | ||
| $candidateIds = $languageId % 2 === 0 ? [$languageId, $languageId + 1] : [$languageId]; |
Comment on lines
+110
to
+114
| $missingCount = (int)$this->connection->fetchOne( | ||
| "SELECT COUNT(*) FROM {$maskTable} m | ||
| WHERE m.{$maskColumn} > 1 | ||
| AND NOT EXISTS (SELECT 1 FROM {$translationTable} t WHERE {$joinCondition})" | ||
| ); |
Comment on lines
+58
to
+59
| if ($schemaManager->introspectTable(self::CONTENT_TABLE)->hasColumn(self::ALWAYS_AVAILABLE_COLUMN)) { | ||
| return; |
Comment on lines
+61
to
+62
| if ($schemaManager->tablesExist([self::CONTENT_TRANSLATION_TABLE])) { | ||
| return; |
Comment on lines
+64
to
+65
| if ($schemaManager->introspectTable(self::TABLE)->hasColumn(self::LANGUAGE_ID_COLUMN)) { | ||
| return; |
Comment on lines
+58
to
+59
| if ($schemaManager->introspectTable(self::TABLE)->hasColumn(self::ALWAYS_AVAILABLE_COLUMN)) { | ||
| return; |
Comment on lines
+69
to
+71
| if (!$schemaManager->introspectTable(self::CONTENT_TABLE)->hasColumn(self::LANGUAGE_MASK_COLUMN)) { | ||
| // Already dropped (or a fresh install whose schema.yaml never had it). | ||
| return; |
Comment on lines
+109
to
+112
| $languageIds = $this->languageGateway->loadVersionTranslations([$versionInfo->id])[$versionInfo->id] ?? []; | ||
| $versionInfo->languageCodes = array_map( | ||
| fn (int $languageId): string => $this->languageHandler->load($languageId)->languageCode, | ||
| $languageIds |
| private function buildDryRunCountSql(string $table): string | ||
| { | ||
| return match ($table) { | ||
| self::TABLE_CONTENT => 'SELECT COUNT(*) FROM ibexa_content c |
…e_id columns to INTEGER The language table stores every language in the system (content, object states, content types, URL aliases), not just "content" languages, so its name shouldn't suggest otherwise. Folded into the same migration as a column-width correction: language_id/initial_language_id/default_language_id columns were BIGINT (mirroring the old bitmask's PHP-integer width), but a regular install never comes close to the ~2 billion languages INTEGER already allows. NarrowLanguageIdColumnTypesMigration now determines via live schema introspection whether the rename and/or the narrowing is still needed, independently, so a partial/interrupted prior run stays idempotent. The rename runs on all 3 platforms (SQLite supports ALTER TABLE ... RENAME TO even though it has no ALTER COLUMN TYPE); narrowing remains MySQL/ PostgreSQL-only. Language\Gateway::CONTENT_LANGUAGE_TABLE's value change propagates the rename to every gateway and ~35 test fixtures that already reference the table exclusively through that constant. Verified against real MySQL, PostgreSQL and SQLite: rename and narrowing both apply correctly, data survives, and FKs still enforce integrity against the renamed/narrowed table.
…l gap, migration retry-safety, N+1, dry-run overcount - Language\Gateway\DoctrineDatabase::canDeleteLanguage(): the legacy always-available-bit tolerance (checking id+1 alongside id) incorrectly matched a genuinely distinct, independently-allocated adjacent language, making an unused language look undeletable whenever its neighbor was in use. Now only tolerates id+1 when it doesn't belong to a real language of its own. - DropLanguageBitmaskColumnsMigration::abortIfTranslationsNotBackfilled(): only checked that some translation row existed for a content id, not that every set mask bit had one - a partially-backfilled multi-language row passed the check and then had its mask silently and irrecoverably dropped. Now joins per language bit. - AddContentAlwaysAvailableColumnsMigration, AddLanguageTranslationTablesMigration, AddSearchObjectWordLinkLanguageIdColumnsMigration, AddUrlAliasAlwaysAvailableColumnMigration, DropLanguageBitmaskColumnsMigration: each used a single first-item existence check as an all-or-nothing "already ran" sentinel. On MySQL, every ALTER/CREATE/DROP auto-commits independently, so a failure partway through left the schema half-migrated and a retry silently skipped the rest. Each column/table/constraint/index is now checked and queued independently; verified against real MySQL with simulated partial failures. - Filter\Gateway\Content\Doctrine\DoctrineGateway / Filter\Gateway\Content\Mapper\DoctrineGatewayDataMapper: version language translations were queried once per row via Language\Gateway::loadVersionTranslations() instead of being bulk-fetched once per result page, unlike the sibling names/field-values data this same gateway already bulk-fetches - a real N+1 on content listings/search. - BackfillLanguageTranslationsCommand's --dry-run count didn't exclude already-backfilled rows (unlike the real, idempotent INSERT), so reruns over-reported "would be inserted". Also fixed two test-setup regressions surfaced along the way: two test files built their fixtures against the current schema.yaml (which now names the table "ibexa_language") but exercised code that still expects the pre-rename "ibexa_content_language" name.
The SonarCloud finding on this line (md5() flagged as a weak hash) is a false positive - text_md5 is a non-cryptographic content-addressing column, matching UrlAlias\Handler::getHash()'s existing production pattern - but the team doesn't use NOSONAR comments, so leaving it unsuppressed instead.
… fix The earlier N+1 fix reimplemented version-translation lookup as a join against the main criteria query, but VersionInfo::$languageCodes' element order is derived directly from whatever (deliberately unordered) order Language\Gateway::loadVersionTranslations() happens to return rows in - an accepted implementation detail on the pre-existing, unchanged code path, not a documented contract. A structurally different join+DISTINCT query has no reason to reproduce that same incidental order, and on PostgreSQL specifically it didn't, breaking ContentFilteringTest in CI (MySQL happened to still match by chance). Fixed by calling loadVersionTranslations() itself, batched with every matching version id from the page at once, instead of reimplementing its query - still avoids the N+1 (one query for the whole page instead of one per row), but now genuinely returns bit-for-bit the same order the existing, long-relied-upon method would have. Verified against real MySQL and PostgreSQL (both previously failing in CI, both now passing) and the full local Filtering integration suite.
… fix The previous fix (5afb188) assumed calling Language\Gateway::loadVersionTranslations() batched, instead of reimplementing its query, would be enough to match the Legacy Search Engine's expected language order - but PostgreSQL specifically can choose a different scan strategy (and therefore a different per-id row order) depending on how many ids are in a given call's IN() list. The search path and the Filter path each batch a different number of ids through the same shared method, so their orders still diverged on Postgres even though both call the identical method. Fixed at the true shared source instead: loadTranslations() (backing both loadContentTranslations() and loadVersionTranslations()) now has an explicit ORDER BY id, language_id, so every caller gets the same deterministic order regardless of batch size or platform. Verified against real MySQL and PostgreSQL: the full ContentFilteringTest suite (69 tests), previously failing on Postgres in CI, now passes on both.
…anslations() ordering fix The previous commit (f188194) made Language\Gateway::loadTranslations() deterministic, which fixed the Filter/Search comparison - but it also changed the iteration order content publish uses to insert per-language URL alias rows into ibexa_url_alias_ml_translation. UrlAlias\Gateway:: loadTranslationLanguageIds() (backing UrlAlias::$languageCodes, asserted directly in URLAliasServiceTest) still had no ORDER BY of its own, so its read-back order depended on however those rows happened to get inserted - which is exactly what the previous fix changed upstream, breaking 3 previously-passing URLAliasServiceTest cases on PostgreSQL. Same class of bug, different call site: added an explicit ORDER BY there too, so this method's result no longer depends on insertion order from whatever upstream write path populated the table. Verified against real PostgreSQL and MySQL: the 3 previously-failing URLAliasServiceTest cases pass, and re-ran both the full URLAliasServiceTest suite and ContentFilteringTest (fixed by the prior commit) to confirm neither regressed the other.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related PRs:
feature/fix-schema-rename-migration-6.0-postgres, the current head of that chain.Description:
Replaces
ibexa/core's language bitmask system with a relational structure. Todayibexa_content_language.idvalues are allocated as powers of two (2, 4, 8, …) with bit 0 permanently reserved for "always available", which hard-caps the system at 62 languages on 64-bit PHP (30 on 32-bit) — enforced by aRuntimeException('Maximum number of languages reached.')inLanguage\Gateway\DoctrineDatabase::insertLanguage(). That ceiling is gone: language ids are now a plain sequentialMAX(id)+1allocation, and "which languages does this row have" is answered by real join tables instead of bitwise arithmetic.New tables (
ibexa_content_translation,ibexa_content_version_translation,ibexa_url_alias_ml_translation), each with a real FK toibexa_content_language(id)andON DELETE CASCADE, replacinglanguage_mask/lang_maskcolumns. "Always available" becomes a plain boolean column (always_available/is_always_available) instead of bit 0 of the mask.Rewritten subsystems:
MaskGeneratordeleted entirely (no external caller to preserve — 6.0 hasn't shipped).FieldBase/SortClauseHandler\Field's priority-language fallback was pure bit-shift arithmetic that only worked because language ids were powers of two. Rewritten asLanguagePriorityConditionBuilder, a correlated-subquery-based priority resolver.ibexa_search_object_word_link.language_maskrenamed to a cleanlanguage_idcolumn — requires a full search reindex after upgrading.UrlAlias\Gateway::historizeBeforeSwap(string $action, int $languageMask)is nowhistorizeBeforeSwap(string $action, array $languageIds)(breaking signature change); the mask-as-identity-key pattern ingetOriginalUrlAliases()and the "NOP entry" (lang_mask == 1) placeholder-row concept are both gone, replaced by real rows +is_always_available.Upgrade path for existing 4.6/5.0 installs: a new
BackfillLanguageTranslationsMigration(chunked, non-transactional Doctrine Migration — safe to run underdoctrine:migrations:migrateduring planned downtime, per the ops model for major-version upgrades: backups first, migrations execute the actual cutover) populates the new join tables from the legacy mask columns beforeDropLanguageBitmaskColumnsMigrationremoves them. The drop migration aborts (AbortMigration) if it finds any row whose mask implies a translation absent from the join tables, as a guard against a skipped/partial backfill.tests/bundle/RepositoryInstaller/Migration/LanguageBitmaskUpgradeSequenceTest.phpruns the full migration sequence end-to-end against a seeded pre-6.0-shaped schema/dataset to verify this.Breaking changes (6.0, unreleased — no BC shim needed):
MaskGeneratorremoved.UrlAlias\Gateway::historizeBeforeSwap()signature change (mask → array of language ids).Search\Legacy\Content\Handler::extractMatchedLanguage()signature change (int $languageMask→array $languageIds, int $mainLanguageId).SharedGateway::getSetNameLanguageMaskSubQuery()removed.Content\Gatewaygains new abstract methods (breaks third-party subclasses that extend it directly).Language.idis no longer a stable/meaningful bitmask value and is not guaranteed to survive an upgrade unchanged — flagged for REST clients or any customer code that persisted a language id.For QA:
Run the new upgrade-path test directly:
It seeds a pre-6.0-shaped schema (3 languages ids 2/4/8, content with masks, a URL alias, a search word-link row), runs all 6 language-migration steps in order, and asserts the final relational data matches what the original masks encoded — including that
DropLanguageBitmaskColumnsMigrationaborts if the backfill was skipped.For a fuller manual check: create >62 languages end-to-end (content creation, search, URL alias resolution) to confirm the old ceiling is actually gone, not just that the guard was removed.
Documentation:
Breaking-changes list above should be carried into the 6.0 upgrade guide once that's written; no upgrade doc currently exists in-repo to update (the
doc/upgrade//CHANGELOG convention was dropped from this repo in 2020).