As the data lakehouse ecosystem matured, a clear divide emerged between organizations using Delta Lake (primarily driven by Databricks) and those adopting the vendor-neutral Apache Iceberg standard. To bridge this divide and prevent data silos, Databricks introduced UniForm (Universal Format).
How UniForm Works
At their core, both Delta Lake and Apache Iceberg are metadata layers sitting on top of the exact same physical data files (usually Apache Parquet). They simply format their metadata differently (Delta uses JSON transaction logs, while Iceberg uses a hierarchical tree of JSON and Avro files).
When UniForm is enabled on a Delta table within Databricks, the system behaves asynchronously to generate interoperability:
- A data engineer writes data to a table using Databricks. Databricks natively writes Parquet files and updates the Delta transaction log.
- In the background, UniForm automatically reads the new Delta log and translates it into standard Apache Iceberg metadata (creating manifest lists and manifest files).
- The newly generated Iceberg metadata points to the exact same Parquet files that the Delta log points to.
Eliminating Data Duplication
The defining advantage of UniForm is that it eliminates the need to duplicate data. Before UniForm, if a data science team using Databricks (Delta) needed to share a petabyte of data with a BI team using Snowflake or Dremio (Iceberg), they had to physically copy all the Parquet files into a new Iceberg-formatted bucket. With UniForm, there is only one copy of the data; the only overhead is the trivial storage cost of maintaining dual metadata schemas.
Read-Only Interoperability
It is important to note that UniForm is primarily a one-way street designed for read access. Databricks remains the owner and writer of the Delta table. External engines query the UniForm-generated Iceberg metadata to execute high-performance reads. While external engines generally do not write back to the table through the Iceberg interface, this read-only access satisfies the vast majority of cross-platform data sharing and BI reporting use cases.
