Table maintenance, read 10 Nov 2026
Snapshot expiry and orphan file removal on Iceberg tables
Table formatsOpen the Compute Bill
expire_snapshots
Each write, update, delete, upsert or compaction in Iceberg produces a new snapshot while keeping the old data and metadata for snapshot isolation and time travel. The expire_snapshots procedure removes old snapshots and the data files that only those snapshots need. It never removes files that a non-expired snapshot still requires.
Arguments: older_than (default 5 days ago), retain_last (default 1), snapshot_ids, max_concurrent_deletes, stream_results and clean_expired_metadata. The docs advise setting stream_results to true to avoid Spark driver out-of-memory errors on large file counts. If older_than and retain_last are omitted, the table properties history.expire.max-snapshot-age-ms (432,000,000 ms, 5 days) and history.expire.min-snapshots-to-keep (1) apply. Snapshots still referenced by branches or tags are not removed, and the main branch never expires.
remove_orphan_files
Orphan files are files in the table location that no metadata file references, often left by failed jobs. remove_orphan_files deletes files older than older_than, which defaults to 3 days ago. It supports dry_run, a location argument, file_list_view for a precomputed listing, and prefix_mismatch_mode for storage schemes that differ in spelling.
Iceberg's maintenance page warns that removing orphan files with a retention interval shorter than the time any write takes to complete is dangerous, because an in-progress file can be treated as orphaned and deleted, which can corrupt the table. Run dry_run first on a table you have not cleaned before.
Old metadata files
Metadata JSON files accumulate as well. write.metadata.delete-after-commit.enabled is false by default and write.metadata.previous-versions-max is 100. The docs give an example: with delete-after-commit false and previous-versions-max 10, after 100 commits there are 10 tracked metadata files and 90 orphaned ones, and those 90 can only be removed by orphan file deletion.
What the managed services do
| Service | Snapshot expiry | Orphan files | Notes from the documentation |
|---|---|---|---|
| Amazon Athena | VACUUM expires snapshots | VACUUM also removes orphan files | Retention set with vacuum_max_snapshot_age_seconds; run by you |
| AWS Glue Data Catalog | Snapshot retention optimizer: retention in days and maximum snapshots to keep | Orphan file deletion optimizer | Only deletes orphans created after the optimizer's creation date; table location must not overlap other data |
| Amazon S3 Tables | Snapshot management on by default: MinimumSnapshots 1, MaximumSnapshotAge 120 hours | Unreferenced file removal policy with NoncurrentDays | Fails for the whole table if user tags, branches or history.expire properties are set |
| Snowflake | Automatic for Snowflake-managed tables, no cost | Not supported for Snowflake-managed tables; contact Support | Externally managed tables: you run maintenance in your own engine |
| Starburst Galaxy | ALTER TABLE ... EXECUTE expire_snapshots | remove_orphan_files | Run by you |
Order of operations
Expire snapshots first, then remove orphans, then compact on a schedule that does not overlap. Compaction creates new files and a new snapshot; the replaced files are only deleted once the snapshot that referenced them expires. A retention window shorter than your longest reader or time-travel need cuts off queries that rely on it. The compaction page covers the rewrite side.
Other maintenance pages
Sources
- Apache Iceberg Spark procedures, read 10 Nov 2026
- Apache Iceberg maintenance, read 10 Nov 2026
- Apache Iceberg configuration, read 10 Nov 2026
- Athena Iceberg optimization, read 10 Nov 2026
- AWS Glue snapshot retention, read 10 Nov 2026
- AWS Glue orphan file deletion, read 10 Nov 2026
- Amazon S3 Tables maintenance, read 10 Nov 2026
- Snowflake Iceberg table management, read 10 Nov 2026
- Starburst Galaxy Iceberg, read 10 Nov 2026
Last reviewed 10 Nov 2026.