Equality Deletes are the second mechanism introduced in Apache Iceberg Spec v2 for Merge-on-Read (MoR) workflows. While Position Deletes are fast to read, they are impossible for streaming engines to write without first executing an expensive scan to find the physical location of the data. Equality Deletes solve the streaming ingestion problem.
How They Work
When a streaming pipeline (like Apache Flink) receives a Change Data Capture (CDC) event indicating that user_id = 456 has been deleted, it writes an Equality Delete File. This file simply records the logical condition of the deletion: "Delete any row where user_id = 456." It does not specify which Parquet file contains that user, or what row index it is at.
At read time, the query engine is responsible for enforcing this condition. As it scans the data files, it essentially performs an anti-join against the equality delete files, dropping any row that matches the specified column values. To ensure data correctness, Iceberg uses sequence numbers: an equality delete only applies to data files that were written before the delete file was committed, preventing the accidental deletion of a new user_id = 456 that was inserted later.
The Performance Trade-off
Equality deletes represent the extreme end of the write vs. read performance trade-off. They are incredibly fast to write, making them perfect for low-latency streaming CDC where the system just needs to land the delete events into the lakehouse as quickly as possible.
However, they are expensive to read. The query engine must compare every row it reads against the criteria stored in the equality delete files. As the number of equality delete files grows over time, query performance will degrade significantly due to read amplification. For this reason, tables relying on equality deletes require aggressive and frequent compaction to convert those logical deletes into physical data rewrites (or convert them to position deletes), restoring the table to a clean, highly performant state for analytical queries.
