Cross-Engine Access: Delta Sharing, Iceberg, Credential Vending
Estimated read: 3 min
Every row filter, column mask, and ABAC policy covered so far assumes queries go through Unity Catalog. That assumption breaks the moment another engine – a partner’s Snowflake account, a Trino cluster, a data scientist’s local PyIceberg session – needs to read the same data. The naive answer is to hand out a cloud storage key or an IAM role that reads the underlying Parquet/Delta files directly, but that’s precisely the storage-layer bypass the “security layer cake” lecture warned about: raw cloud IAM access sits beneath Unity Catalog, so a direct storage grant skips every mask and filter you just spent this section building.
Credential vending is the mechanism that closes that gap. Instead of handing out a long-lived storage key, Unity Catalog issues short-lived, scoped cloud credentials (typically expiring in about an hour) at query time, only after checking that the requesting principal has the Unity Catalog grants – and satisfies the row filters and column masks – for the object being read. The external engine still touches cloud storage directly for performance, but only after Unity Catalog has decided what it’s allowed to see.
Two governed paths sit on top of credential vending, for two different audiences:
| Path | Audience | What it grants access to |
|---|---|---|
| Delta Sharing | External organizations, or any platform (Spark, Pandas, Power BI) without a Unity Catalog account | Read-only shares – curated collections of tables, volumes, notebooks, or AI models, including masked/filtered dynamic views |
| Iceberg REST catalog | Iceberg-native engines (Trino, Flink, Snowflake, PyIceberg) reading Unity Catalog-managed tables | Direct table reads via the Iceberg REST protocol, backed by the same vended, short-lived credentials |
-- Delta Sharing: expose a governed, already-masked view -- not the raw table --
-- so the recipient inherits the same PII masking as an internal analyst would
CREATE SHARE finance_reporting_share;
ALTER SHARE finance_reporting_share ADD TABLE finance.accounts_receivable.sales_secure;
For Iceberg specifically, Unity Catalog’s table format interoperability (UniForm) generates Iceberg metadata alongside the Delta transaction log for the same physical table, so an external Iceberg reader connects through the REST catalog endpoint and gets vended, time-limited credentials scoped to exactly that table – no separate copy or export pipeline required.
Governance only survives the trip to an external engine if the shared object already carries the row filter or column mask – share the governed dynamic view or the ABAC-masked table, never the raw base table, or you’ve rebuilt the storage-bypass problem one layer up.
This is the piece that most legacy migrations underestimate: the org chart always has at least one downstream consumer who isn’t going to adopt Databricks, and the choice is between building them a real governed path now or discovering a shadow export pipeline in an audit later. See the Delta Sharing documentation and the Iceberg interoperability guide for the current client support matrix and credential-vending configuration.
| ← Previous: Tag-Based PII Masking | Next: The ABAC Tag Taxonomy and Masking Policy Pack → | |:—|—:|