Add BI SQL examples capturing tool-generated metric queries

bi-sql-examples/README.md

@@ -0,0 +1,37 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements.  See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership.  The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied.  See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+BI SQL examples
+===============
+
+Real SQL that BI tools generate when they query, via SQL, metrics defined outside
+the tool. The
+goal is to study these query shapes to ensure that Ossie can fully and safely
+cover the complex cases BI tools produce.
+
+The examples read measures with the `MEASURE()` function, but that is only for
+illustration: any proposal will require the BI tool to have some function for
+querying measures.
+
+Layout
+------
+
+One subfolder per BI tool. Each subfolder holds a `setup.sql` that shows the
+shape of the tables and metrics, one `.sql` file per feature area, and a
+`README.md`.

bi-sql-examples/tableau/01-top-n-filters.sql

@@ -0,0 +1,118 @@
+-- Licensed to the Apache Software Foundation (ASF) under one
+-- or more contributor license agreements.  See the NOTICE file
+-- distributed with this work for additional information
+-- regarding copyright ownership.  The ASF licenses this file
+-- to you under the Apache License, Version 2.0 (the
+-- "License"); you may not use this file except in compliance
+-- with the License.  You may obtain a copy of the License at
+--
+--   http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing,
+-- software distributed under the License is distributed on an
+-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+-- KIND, either express or implied.  See the License for the
+-- specific language governing permissions and limitations
+-- under the License.
+
+-- ============================================================================
+-- Tableau BI SQL examples: Top N filters
+--
+-- Shows how Tableau's low-code filter UI turns into SQL, from a plain aggregate
+-- up to a top-N filter that needs a join and subquery. The last query shows an
+-- optimization Tableau applies when the filter dimension matches the
+-- visualization dimension.
+--
+-- Tableau feature: Filter Data from Your Views (the "Top" tab).
+-- https://help.tableau.com/current/pro/desktop/en-us/filtering.htm
+-- ============================================================================
+
+-- ----------------------------------------------------------------------------
+-- Query 1: simple aggregation.
+-- Visualize Category by the Total Quantity measure. A plain aggregate that
+-- calls the MEASURE function.
+-- ----------------------------------------------------------------------------
+SELECT
+  `orders_metrics`.`category` AS `category`,
+  (MEASURE(`orders_metrics`.`total_quantity`)) AS `total_quantity`
+FROM
+ `orders_metrics` `orders_metrics`
+GROUP BY
+  1;
+
+-- ----------------------------------------------------------------------------
+-- Query 2: simple WHERE filter.
+-- A low-code date-range filter (years 2023 and 2024) becomes a plain WHERE.
+-- Filters expressible as a simple WHERE clause tend to port across BI vendors.
+-- ----------------------------------------------------------------------------
+SELECT
+  `orders_metrics`.`category` AS `category`,
+  (MEASURE(`orders_metrics`.`total_quantity`)) AS `total_quantity`
+FROM
+  `orders_metrics` `orders_metrics`
+WHERE
+  (YEAR(`orders_metrics`.`order_date`) IN (2023, 2024))
+GROUP BY
+  1;
+
+-- ----------------------------------------------------------------------------
+-- Query 3: top-N filter on a different dimension (join + subquery).
+-- Keep the top 2 states by Total Revenue while visualizing Category by Total
+-- Quantity. A subquery ranks states by the revenue measure; the main query
+-- joins to it to apply the filter before aggregating.
+-- ----------------------------------------------------------------------------
+SELECT
+  `orders_metrics`.`category` AS `category`,
+  (MEASURE(`orders_metrics`.`total_quantity`)) AS `total_quantity`
+FROM
+  `orders_metrics` `orders_metrics`
+    JOIN (
+      SELECT
+        `orders_metrics`.`state_id` AS `state_id`,
+        (MEASURE(`orders_metrics`.`total_revenue`)) AS `x__alias__0`
+      FROM
+        `orders_metrics` `orders_metrics`
+      GROUP BY
+        1
+      ORDER BY
+        `x__alias__0` DESC,
+        `state_id` ASC
+      LIMIT 2
+    ) `t0`
+      ON (`orders_metrics`.`state_id` = `t0`.`state_id`)
+GROUP BY
+  1;
+
+-- ----------------------------------------------------------------------------
+-- Query 4: top-N filter on the same dimension (folded).
+-- Keep the top 2 categories by Total Revenue while visualizing Category. Because
+-- the filter dimension equals the visualization dimension, Tableau folds the
+-- filter subquery into the main query - no join needed.
+--
+-- Note: this fold corresponds to Query 3's join-and-subquery technique, not to
+-- its specific result (Query 3 filters top-2 states; this filters top-2
+-- categories). The fold is valid only under the stable-domain assumption: that a
+-- dimension's domain is fixed regardless of the other measures and dimensions in
+-- the query.
+--
+-- Separately, Query 3 joins the top-N subquery with an equality predicate
+-- (state_id = t0.state_id), not IS NOT DISTINCT FROM. That matches a null-safe
+-- join only when state_id has no NULLs. This is a distinct assumption from the
+-- fold's: the fold relies on a stable domain, while this plain-= join relies on
+-- a NULL-free join key.
+--
+-- Both optimizations are unsafe against multi-table models behind opaque
+-- interfaces, where adding or removing a measure can change the dimension domain.
+-- ----------------------------------------------------------------------------
+SELECT
+  `orders_metrics`.`category` AS `category`,
+  (MEASURE(`orders_metrics`.`total_quantity`)) AS `total_quantity`,
+  (MEASURE(`orders_metrics`.`total_revenue`)) AS `x__alias__0`
+FROM
+  `orders_metrics` `orders_metrics`
+GROUP BY
+  1
+ORDER BY
+  `x__alias__0` DESC,
+  `category` ASC
+LIMIT 2;

bi-sql-examples/tableau/02-lod-two-stage-aggregation.sql

@@ -0,0 +1,59 @@
+-- Licensed to the Apache Software Foundation (ASF) under one
+-- or more contributor license agreements.  See the NOTICE file
+-- distributed with this work for additional information
+-- regarding copyright ownership.  The ASF licenses this file
+-- to you under the Apache License, Version 2.0 (the
+-- "License"); you may not use this file except in compliance
+-- with the License.  You may obtain a copy of the License at
+--
+--   http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing,
+-- software distributed under the License is distributed on an
+-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+-- KIND, either express or implied.  See the License for the
+-- specific language governing permissions and limitations
+-- under the License.
+
+-- ============================================================================
+-- Tableau BI SQL examples: LoDs for two-stage aggregation
+--
+-- Measures compose with Tableau's own FIXED level-of-detail (LoD) calculations.
+-- This computes a measure at a per-state grain, then averages that result: for
+-- each category, the average across its states of total revenue.
+--
+-- The generated SQL is two-stage: an inner query computes the per-state measure
+-- (MEASURE(total_revenue) grouped by state_id), and the outer query applies the
+-- second aggregation (AVG) after joining on the state grain. The join uses
+-- IS NOT DISTINCT FROM so NULL state_id values match.
+--
+-- Tableau feature: FIXED Level of Detail Expressions.
+-- https://help.tableau.com/current/pro/desktop/en-us/calculations_calculatedfields_lod_fixed.htm
+-- ============================================================================
+
+SELECT
+  `t0`.`category` AS `category`,
+  AVG(`t1`.`x_measure__1`) AS `average_of_total_revenues_by_state`
+FROM
+  (
+    SELECT
+      `orders_metrics`.`category` AS `category`,
+      `orders_metrics`.`state_id` AS `state_id`
+    FROM
+      `orders_metrics` `orders_metrics`
+    GROUP BY
+      1,
+      2
+  ) `t0`
+    JOIN (
+      SELECT
+        `orders_metrics`.`state_id` AS `state_id`,
+        (MEASURE(`orders_metrics`.`total_revenue`)) AS `x_measure__1`
+      FROM
+        `orders_metrics` `orders_metrics`
+      GROUP BY
+        1
+    ) `t1`
+      ON (`t0`.`state_id` IS NOT DISTINCT FROM `t1`.`state_id`)
+GROUP BY
+  1;

bi-sql-examples/tableau/03-sets.sql

@@ -0,0 +1,83 @@
+-- Licensed to the Apache Software Foundation (ASF) under one
+-- or more contributor license agreements.  See the NOTICE file
+-- distributed with this work for additional information
+-- regarding copyright ownership.  The ASF licenses this file
+-- to you under the Apache License, Version 2.0 (the
+-- "License"); you may not use this file except in compliance
+-- with the License.  You may obtain a copy of the License at
+--
+--   http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing,
+-- software distributed under the License is distributed on an
+-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+-- KIND, either express or implied.  See the License for the
+-- specific language governing permissions and limitations
+-- under the License.
+
+-- ============================================================================
+-- Tableau BI SQL examples: sets (LoD calculation vs. built-in Set feature)
+--
+-- Two different low-code paths to the same result, producing different SQL.
+-- Goal: graph Total Quantity split by high-revenue vs. low-revenue categories,
+-- where a high-revenue category has Total Revenue >= 5000.
+--
+-- Key takeaway: BI tools can generate very different SQL for similar user-facing
+-- capabilities, so a SQL interface to reusable semantics must be robust across
+-- query shapes.
+--
+-- Tableau feature: Create Sets.
+-- https://help.tableau.com/current/pro/desktop/en-us/sortgroup_sets_create.htm
+-- ============================================================================
+
+-- ----------------------------------------------------------------------------
+-- Query A: FIXED LoD calculation used as a dimension.
+-- Tableau computes Total Revenue per category in a subquery, joins it back to
+-- the main table (IS NOT DISTINCT FROM handles NULL categories), and derives
+-- the split dimension by applying the >= 5000 comparison to the measure.
+-- ----------------------------------------------------------------------------
+SELECT
+  (`t0`.`x_measure__0` >= 5000) AS `is_top_category`,
+  (MEASURE(`orders_metrics`.`total_quantity`)) AS `total_quantity`
+FROM
+  `orders_metrics` `orders_metrics`
+    JOIN (
+      SELECT
+        `orders_metrics`.`category` AS `category`,
+        (MEASURE(`orders_metrics`.`total_revenue`)) AS `x_measure__0`
+      FROM
+        `orders_metrics` `orders_metrics`
+      GROUP BY
+        1
+    ) `t0`
+      ON (`orders_metrics`.`category` IS NOT DISTINCT FROM `t0`.`category`)
+GROUP BY
+  1;
+
+-- ----------------------------------------------------------------------------
+-- Query B: the built-in Set feature (in/out membership).
+-- Tableau computes the high-revenue categories in a subquery that filters with
+-- HAVING and emits the category plus a constant flag column. The main table
+-- LEFT JOINs that subquery (the left join keeps all rows) and derives set
+-- membership by testing whether the flag is non-NULL.
+-- ----------------------------------------------------------------------------
+SELECT
+  (NOT (`t0`.`xtemp1_output` IS NULL)) AS `io_high_revenue_categories`,
+  (MEASURE(`orders_metrics`.`total_quantity`)) AS `total_quantity`
+FROM
+  `orders_metrics` `orders_metrics`
+    LEFT OUTER JOIN (
+      SELECT
+        `orders_metrics`.`category` AS `category`,
+        1 AS `xtemp1_output`,
+        (MEASURE(`orders_metrics`.`total_revenue`)) AS `x_measure__0`
+      FROM
+        `orders_metrics` `orders_metrics`
+      GROUP BY
+        1
+      HAVING
+        ((MEASURE(`orders_metrics`.`total_revenue`)) >= 5000.)
+    ) `t0`
+      ON (`orders_metrics`.`category` IS NOT DISTINCT FROM `t0`.`category`)
+GROUP BY
+  1;