Skip to content

Flink: Support reading Iceberg views in SQL - #17859

Open
talatuyarer wants to merge 1 commit into
apache:mainfrom
talatuyarer:flink-views-read
Open

Flink: Support reading Iceberg views in SQL#17859
talatuyarer wants to merge 1 commit into
apache:mainfrom
talatuyarer:flink-views-read

Conversation

@talatuyarer

Copy link
Copy Markdown
Contributor

This PR adds the read path for Iceberg views to Flink, the first step of the plan in #17858 With this change, views created through the Iceberg ViewCatalog API or by other engines such as Spark become queryable from Flink SQL.

Write operations such as CREATE VIEW, DROP VIEW, ALTER VIEW ... RENAME and ALTER VIEW AS are follow-ups tracked in #17858.

Resolution semantics and limitations

Flink's Catalog API gives an implementation no way to inject the resolution context that the Iceberg view spec defines, so two limitations are worth calling out:

  1. Stored default-catalog/default-namespace are not honored by the Flink planner. The view spec says engines should resolve unqualified references against the defaults stored in the view version, and Spark does this natively via its View#currentCatalog()/currentNamespace() hooks. Flink has no equivalent. It always resolves unqualified references against the view's own catalog and database. For views whose default-namespace equals the namespace they live in (the common case, and what the follow-up CREATE VIEW PR will always produce, the two coincide and resolution is correct. A view created by another engine with a different default-namespace may resolve unqualified references differently in Flink than in Spark/Trino. Fixing this properly needs a Flink-side API; until then this is documented behavior.

  2. Lenient dialect fallback. If a view has no "flink" SQL representation,View#sqlFor falls back to the closest available representation (e.g. Spark SQL). ANSI-compatible SQL works across engines (covered by tests), but non-portable SQL fails at parse time or, worse, could parse with different semantics. This is the same stance Spark takes today; a strict-dialect option is planned as a follow-up. See also the https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81 for the longer-term cross-engine story.

  3. Catalog names in qualified references are deployment-local. If the stored view SQL fully qualifies references with a catalog name, that name only resolves in deployments that register the Iceberg catalog under the same name. This is inherent to the view spec storing engine-local catalog names and applies to Spark equally; unqualified references (resolved per item 1) do not have this problem.

@github-actions github-actions Bot added the flink label Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant