Skip to content
Lakehouse Compare

Table maintenance, read 10 Nov 2026

Snapshot expiry and orphan file removal on Iceberg tables

Table formatsOpen the Compute Bill

expire_snapshots

Each write, update, delete, upsert or compaction in Iceberg produces a new snapshot while keeping the old data and metadata for snapshot isolation and time travel. The expire_snapshots procedure removes old snapshots and the data files that only those snapshots need. It never removes files that a non-expired snapshot still requires.

Arguments: older_than (default 5 days ago), retain_last (default 1), snapshot_ids, max_concurrent_deletes, stream_results and clean_expired_metadata. The docs advise setting stream_results to true to avoid Spark driver out-of-memory errors on large file counts. If older_than and retain_last are omitted, the table properties history.expire.max-snapshot-age-ms (432,000,000 ms, 5 days) and history.expire.min-snapshots-to-keep (1) apply. Snapshots still referenced by branches or tags are not removed, and the main branch never expires.

remove_orphan_files

Orphan files are files in the table location that no metadata file references, often left by failed jobs. remove_orphan_files deletes files older than older_than, which defaults to 3 days ago. It supports dry_run, a location argument, file_list_view for a precomputed listing, and prefix_mismatch_mode for storage schemes that differ in spelling.

Iceberg's maintenance page warns that removing orphan files with a retention interval shorter than the time any write takes to complete is dangerous, because an in-progress file can be treated as orphaned and deleted, which can corrupt the table. Run dry_run first on a table you have not cleaned before.

Old metadata files

Metadata JSON files accumulate as well. write.metadata.delete-after-commit.enabled is false by default and write.metadata.previous-versions-max is 100. The docs give an example: with delete-after-commit false and previous-versions-max 10, after 100 commits there are 10 tracked metadata files and 90 orphaned ones, and those 90 can only be removed by orphan file deletion.

What the managed services do

Sources are listed below; all read 2026-11-10.
ServiceSnapshot expiryOrphan filesNotes from the documentation
Amazon AthenaVACUUM expires snapshotsVACUUM also removes orphan filesRetention set with vacuum_max_snapshot_age_seconds; run by you
AWS Glue Data CatalogSnapshot retention optimizer: retention in days and maximum snapshots to keepOrphan file deletion optimizerOnly deletes orphans created after the optimizer's creation date; table location must not overlap other data
Amazon S3 TablesSnapshot management on by default: MinimumSnapshots 1, MaximumSnapshotAge 120 hoursUnreferenced file removal policy with NoncurrentDaysFails for the whole table if user tags, branches or history.expire properties are set
SnowflakeAutomatic for Snowflake-managed tables, no costNot supported for Snowflake-managed tables; contact SupportExternally managed tables: you run maintenance in your own engine
Starburst GalaxyALTER TABLE ... EXECUTE expire_snapshotsremove_orphan_filesRun by you

Order of operations

Expire snapshots first, then remove orphans, then compact on a schedule that does not overlap. Compaction creates new files and a new snapshot; the replaced files are only deleted once the snapshot that referenced them expires. A retention window shorter than your longest reader or time-travel need cuts off queries that rely on it. The compaction page covers the rewrite side.

Other maintenance pages

Sources

  1. Apache Iceberg Spark procedures, read 10 Nov 2026
  2. Apache Iceberg maintenance, read 10 Nov 2026
  3. Apache Iceberg configuration, read 10 Nov 2026
  4. Athena Iceberg optimization, read 10 Nov 2026
  5. AWS Glue snapshot retention, read 10 Nov 2026
  6. AWS Glue orphan file deletion, read 10 Nov 2026
  7. Amazon S3 Tables maintenance, read 10 Nov 2026
  8. Snowflake Iceberg table management, read 10 Nov 2026
  9. Starburst Galaxy Iceberg, read 10 Nov 2026

Last reviewed 10 Nov 2026.