CONCAT(K) reintroduced as the folding operation; JOIN(K) redefined as the record product - #2
Merged
Merged
Conversation
… the record product The operation spelled JOIN(K) up to 0.5.x concatenated the records of the provided anchors to the anchor's record (one wide record, the number of records strictly decreasing) — a fold, what pandas.concat(axis=1) does, not a join. It keeps that semantics under its original name CONCAT(K); JOIN(K) now multiplies: every record of the anchor is combined with every provided record (cross product for K = ∅, equi-join on the key positions otherwise, a shared named attribute is a natural-join condition). The provided anchors become joined-away (J): excluded from the recordset, but their records stay available so that several anchors may join the same records irrespective of action order. Working state: rec is multi-valued (a non-empty sequence of records per anchor), the component J is added, allRec() returns the live anchors only; SchemaConstructionStrategy visits (anchor, record, position) triples; record generation emits one record per item-based record. CONCAT(K) no longer deduplicates named attributes: a named attribute shared by the concatenated records (apart from the key) violates a precondition — the action has no effect and a Diagnostic is recorded (TableInterpreter.diagnostics(); withStrictPreconditions(true) raises). Task 098 lists its full group key, CONCAT(0,1,2,3), where the former JOIN(0,1) silently dropped the repeated A/B attributes; expected recordsets of all migrated tasks are unchanged. Grammar: concatOp / CONCAT keyword; serializer emits CONCAT(k1, k2); VS Code grammar highlights CONCAT. Conformance: 12 positive tasks migrated JOIN(K) -> CONCAT(K); semantic cases concat_by_key, join_product, join_equi_key added. Tests: WorkingStateConcatTest, WorkingStateJoinTest, TableInterpreterMultiRecordTest.
examples.md: Examples 2 and 3 on CONCAT(0), CONCAT vs JOIN note, new Example 6 (record join: exploding a delimited key against stacked columns); rtl-reference, model/itm, model/atp, api, architecture, embedded-rtl, index updated; CHANGELOG Unreleased: Changed (breaking) with the migration table; pom 0.6.0-SNAPSHOT.
K is unbounded and positional; it must list every position that repeats across the rows of a group (task 098: CONCAT(0,1,2,3), not (0,1)); a row that differs at a key position is rejected with a diagnostic instead of being folded silently.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
JOIN(K)up to 0.5.x folded records (one wide record,pandas.concat(axis=1)), not a join. It keeps that semantics under its original nameCONCAT(K).JOIN(K)is redefined as the record product: every record of the anchor × every provided record — cross product forK = ∅, equi-join on the key positions otherwise, a shared named attribute acts as a natural-join condition. Provided anchors become joined-away (J): excluded from the recordset, records stay available (several anchors may join the same records irrespective of action order).recis multi-valued, componentJadded,allRec()returns live anchors only; recordsets are multisets (order is a documented default).CONCAT(K)no longer deduplicates named attributes: a shared attribute outsideKis a precondition violation — no effect +Diagnostic(TableInterpreter.diagnostics();withStrictPreconditions(true)raises).JOIN(K)→CONCAT(K)in existing patterns. Corpus tasks 016, 023, 025, 033, 046, 047, 050, 053, 069, 094, 097, 098 migrated; task 098 lists its full group keyCONCAT(0,1,2,3). Expected recordsets unchanged.Motivation: ATBench
multistep_test1(explode × stack) is expressible only with a record product; on 0.5.3 the pattern produced 1089 records with silently dropped values, now 4356 matching the ground truth.Test plan
mvn test— 2141 tests, 0 failures (2117 + 24 new:WorkingStateConcatTest,WorkingStateJoinTest,TableInterpreterMultiRecordTest, semantic casesconcat_by_key,join_product,join_equi_key)git diff src/test/resourcesempty — expected CSVs of migrated tasks untouchedmkdocs build --strictregtab-eval-on-atbench):multistep_test1matches gt and label reference; the other 243 outcomes unchangedVERSIONbumped)