# Delta Lake OPTIMIZE and VACUUM

Table maintenance, read 10 Nov 2026

**The short answer.** Delta Lake compacts small files with OPTIMIZE, whose bin-packing is idempotent, and deletes unreferenced files with VACUUM, which never runs on its own and keeps 7 days by default. Databricks predictive optimization schedules both on Unity Catalog managed tables and skips ZORDER.

## OPTIMIZE

OPTIMIZE compacts a Delta table by rewriting small files into larger ones. It accepts a partition predicate with WHERE so you can limit the work, and is available in SQL and through deltaTable.optimize().executeCompaction() in Python and Scala. Bin-packing is idempotent: running it twice on the same data makes no second change. Readers keep snapshot isolation, so OPTIMIZE does not interrupt them, and it makes no data changes.

Z-ordering is OPTIMIZE ... ZORDER BY (col) and colocates related rows to help data skipping. It is not idempotent: each run tries to recluster all files in the partition. The docs say locality drops with each extra column and that columns without collected statistics make Z-ordering ineffective. They also note it balances tuple counts, not bytes, so wide recent rows can skew task times and output file sizes.

## Auto compaction and optimized writes

Auto compaction is enabled with the delta.autoOptimize.autoCompact table property or the spark.databricks.delta.autoCompact.enabled session setting, and by default uses 128 MB as the target file size. It triggers only when a partition has at least a minimum number of small files, set with spark.databricks.delta.autoCompact.minNumFiles.

Optimized writes shuffle data before writing so fewer, larger files are produced. They are disabled by default, can be set at table, session or writer level, and use a bin size of 512 MiB by default. The docs note the shuffle can increase write latency. Fabric documents the same auto compaction settings and, from Fabric Spark runtime 2.0 (Delta 4.2), an onCheckpointOnly mode that defers the check to checkpoints, typically every 10 commits.

## VACUUM

VACUUM removes files no longer referenced by a table that are older than the retention threshold, 7 days by default. It is not triggered automatically. It deletes data files only; log files are removed automatically after checkpoints and kept 30 days by default (delta.logRetentionDuration). Time travel to a version older than the retention period is lost after VACUUM.

Modes and options: the default FULL mode lists the table directory; LITE reads the transaction log instead and runs faster, failing with DELTA_CANNOT_VACUUM_LITE if the log was pruned. DRY RUN lists files to delete. USING INVENTORY takes a table or query of file paths to avoid a full listing. Deleting in parallel needs spark.databricks.delta.vacuum.parallelDelete.enabled set to true. The docs advise a retention interval of at least 7 days and include a safety check that blocks shorter intervals unless you turn it off.

## Who runs it for you

Databricks predictive optimization runs OPTIMIZE, VACUUM and ANALYZE on Unity Catalog managed tables, is on by default, and bills a serverless jobs SKU. It does not run ZORDER; on tables that use Z-order it ignores Z-ordered files. Its VACUUM retention follows delta.deletedFileRetentionDuration, default 7 days, and never goes below 7 days when it runs VACUUM FULL. On Databricks Runtime 18 LTS and above, DESCRIBE TABLE EXTENDED ... AS JSON shows why an operation was skipped.

Microsoft Fabric documents OPTIMIZE, Z-order and V-order on Delta tables and notes that liquid-clustered tables are only clustered when OPTIMIZE runs, so a compaction schedule or auto compaction is needed. Amazon EMR 7.14.0 bundles Delta 3.3.2-amzn-3; none of the EMR pages read describes a scheduled OPTIMIZE or VACUUM. See [compaction on each platform](/maintenance/compaction-on-each-platform/).

## Deletion vectors

With deletion vectors enabled, deleting a row records it separately instead of rewriting the whole Parquet file, and a later OPTIMIZE can apply them. The Delta docs include a table of which operations each Delta version supports with deletion vectors, so check your runtime version against it.

## Other maintenance pages

- [Iceberg compaction: rewrite_data_files, strategies, platforms](/maintenance/iceberg-compaction/)
- [Iceberg snapshot expiry and orphan file removal](/maintenance/snapshot-expiry-and-orphan-files/)
- [Apache Hudi table services: compaction, clustering, cleaning](/maintenance/hudi-table-services/)
- [Which platforms automate Iceberg compaction](/maintenance/compaction-on-each-platform/)

## Sources

- Delta Lake file management: https://docs.delta.io/latest/optimizations-oss.html (read 10 Nov 2026)
- Delta Lake utility commands (VACUUM): https://docs.delta.io/latest/delta-utility.html (read 10 Nov 2026)
- Delta Lake deletion vectors: https://docs.delta.io/latest/delta-deletion-vectors.html (read 10 Nov 2026)
- Databricks predictive optimization: https://docs.databricks.com/aws/en/optimizations/predictive-optimization (read 10 Nov 2026)
- Fabric table compaction: https://learn.microsoft.com/en-us/fabric/data-engineering/table-compaction (read 10 Nov 2026)
- Delta Lake on Amazon EMR: https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-delta.html (read 10 Nov 2026)
- Delta Lake latest release (GitHub API): https://api.github.com/repos/delta-io/delta/releases/latest (read 10 Nov 2026)

Last reviewed 10 Nov 2026. Canonical: https://datalakehousecompare.com/maintenance/delta-optimize-and-vacuum/
