# Small files problem

Glossary, read 10 Nov 2026

**The short answer.** The small files problem is the slowdown and metadata growth that follow when a table holds many data files far below the size an engine reads efficiently. Streaming and frequent micro-batch writes create them. Each file adds a metadata row and a file-open cost to every query that touches it.

The fix is compaction into larger files, run on a schedule or by a managed service. The target size is a setting, and managed services publish their own defaults.

Left alone, it raises both query time and the cost of metadata operations.

## A sourced example

The Iceberg maintenance page says more data files means more metadata in manifest files, and its sample rewriteDataFiles call combines small files into 500 MB files for the partition with date 2020-08-18.

## Sources

- Iceberg documentation: maintenance: https://iceberg.apache.org/docs/latest/maintenance/ (read 10 Nov 2026)

Last reviewed 10 Nov 2026. Canonical: https://datalakehousecompare.com/glossary/small-files-problem/
