Side by side, read 10 Nov 2026
Databricks and Open stack on EMR and EKS: the published facts in the same rows
Databricks profileOpen stack on EMR and EKS profile
The same rows for both
| Row | Databricks | Open stack on EMR and EKS |
|---|---|---|
| Kind | Managed lakehouse built around Delta Lake | Self-run Spark with open table formats |
| Does well | Per Databricks documentation, Photon keeps the Spark Catalyst planner, works with existing Spark API code and falls back to the Spark runtime for unsupported operations (read 2026-11-10). Per Databricks documentation, Delta tables can generate Iceberg metadata from Runtime 14.3 LTS, so Iceberg clients read Delta data without rewriting files (read 2026-11-10). | Per AWS documentation, EMR 7.14.0 packages Spark 3.5.8-amzn-0, Delta 3.3.2-amzn-3, Iceberg 1.10.1-amzn-1 and Hudi 1.0.2-amzn-3, so all three formats are available (read 2026-11-10). AWS ran TPC-DS 3 TB on 9 x r5d.4xlarge and reports $4.81 for EMR 7.12 against $17.65 for open-source Spark 3.5.6 with Iceberg 1.10.0, with repositories to repeat it. |
| Limited | Per Databricks documentation, serverless compute supports only Spark Connect APIs, so RDD APIs, R and Maven coordinates are unsupported, and ANSI mode on by default can change results of existing jobs. Per Databricks, compute is billed in DBUs in addition to the cloud bill for the same instances, so a Jobs Compute run pays Databricks and AWS separately; the Compute Bill line adds $12.24 of DBUs to $20.16 of EC2 (read 2026-11-10). | Per the Apache Iceberg documentation, users run table maintenance themselves: expire snapshots, remove orphan files and compact with rewriteDataFiles as jobs they schedule (read 2026-11-10). A cost comparison of open-stack Spark against managed lakehouses by a party other than AWS or a vendor: none found on 2026-11-10. |
| Published speed results | Docs claim Photon gives up to 5x better price/performance for data and analytics workloads versus other cloud data warehouses; no method is given on this page. Databricks announced an audited 100TB TPC-DS result of 32,941,245 QphDS for Databricks SQL, 2.2x the previous record held by Alibaba, with total system cost 10% lower on list pricing. This is a SQL warehouse result, not a Spark batch ETL result. The same Databricks blog says research from Barcelona Supercomputing Center found Databricks 2.7x faster and 12x better in price-performance than Snowflake on a TPC-DS derivative. This is a vendor-reported claim of a third-party study; the study itself was not fetched. | AWS ran TPC-DS 3 TB (104 SparkSQL queries, 3 rounds, 9 x r5d.4xlarge, Hadoop catalog) and reports EMR 7.12 averaged 1,349.62 s versus 6,113.92 s for open-source Spark 3.5.6 with Iceberg 1.10.0 and 1,535.62 s for EMR 7.5. AWS notes results are not comparable to official TPC-DS. The run is by the vendor of EMR. |
| Published prices and cost results | Databricks bills compute in DBUs at per-second granularity with pay-as-you-go or committed-use discounts, and no up-front costs. The pricing pages render their rates by script; the AWS pricing data file those pages load lists Jobs Compute at $0.15 per DBU on the plan it names Premium (r6i.4xlarge: 4.08 DBU per hour). The file names no region. Jobs pricing offers serverless (Performance Optimized and Standard modes) and classic clusters; the jobs pricing page maps plan names across clouds (its Premium tier on Azure corresponds to Enterprise on AWS and GCP), while the AWS pricing data file lists the $0.15 Jobs Compute rate under a plan it names Premium. Photon instances consume DBUs at a different rate. | EMR on EC2 charges an EMR price in addition to the EC2 price (and EBS), billed per second with a one-minute minimum. AWS's example in us-east-1: c4.2xlarge EMR charge $0.105/hour next to EC2 $0.398/hour; one master and two core nodes for a month total $1,101.57. Spot instances are up to a 90% discount versus On-Demand. EMR on EKS adds an uplift per vCPU-hour and per GB-hour to EKS/EC2 or Fargate costs, plus $0.10 per hour for each EKS cluster. AWS's worked example uses $0.007084 per vCPU-hour and $0.000777875 per GB-hour (100 vCPU and 300 GB for 30 minutes = $0.4709 EMR uplift). The uplift applies to any pod using an EMR runtime image, including with the Spark operator. EMR Serverless is billed per vCPU-hour and GB-hour. AWS's worked example uses $0.052624 per vCPU-hour and $0.0057785 per GB-hour for a job with 100 vCPU and 750 GB for 30 minutes, then 200 more vCPU for 15 minutes, totaling $9.596275. In the same AWS benchmark the cost was $4.81 for EMR 7.12, $5.47 for EMR 7.5 and $17.65 for open-source Spark 3.5.6 with Iceberg 1.10.0. Cost formula uses r5d.4xlarge at $1.152/hour plus $0.27/hour EMR plus EBS, On-Demand. |
| Code changes and migration | Photon is a native C++ vectorized engine that keeps Spark's Catalyst planner and works with existing Spark API code; unsupported operations fall back to the Spark runtime. Serverless compute for notebooks and jobs supports only Spark Connect APIs; Spark RDD APIs, R, Maven coordinates, global temp views, DBFS mounts with instance profiles and Databricks Container Services are not supported. ANSI mode is on by default and the default time zone is UTC, which can change results of existing jobs. | The EMR runtime for Spark provides 'full API compatibility with open source Apache Spark and Apache Iceberg', so existing Spark jobs run as is. Optimized runtimes are used by EMR on EC2, EMR Serverless, EMR on EKS, EMR on Outposts and AWS Glue. Self-run Spark on EKS uses the open-source Spark operator and needs no job rewrite. |
| Tuning and table upkeep | Photon is on by default for classic all-purpose compute, jobs compute and classic pipelines; API users must set runtime_engine to PHOTON. Photon instance types use DBUs at a different rate than the same instance without Photon. On serverless compute Databricks allocates and manages compute so users do not provision clusters; the Spark UI and Spark logs are not available, query profile and client-side logs are used instead. Serverless compute runs workloads without provisioning compute in your cloud account, and Databricks says this reduces the need to manage compute resources. It requires Unity Catalog to reach external data sources. | Iceberg docs say users must run maintenance: expire snapshots, remove old metadata files and delete orphan files are 'recommended'; compacting data files with the Spark rewriteDataFiles action (for example target-file-size-bytes of 500 MB) and rewriting manifests are 'optional' maintenance, run as jobs you schedule. Hudi merge-on-read tables need compaction to merge log files into base files and bound read latency. Compaction has two steps, scheduling (plan written to the timeline) and execution; the docs describe strategies for choosing which file slices to compact. Delta Lake OSS docs list file management features the user applies: compaction (bin-packing) with OPTIMIZE, auto compaction, data skipping, Z-ordering, multi-part checkpointing, log compactions and optimized write. |
| Spark versions and table formats | Databricks Runtime 18 LTS ships Apache Spark 4.1.0 (released June 10, 2026); Runtime 17.3 LTS ships Spark 4.0.0; Runtime 16.4 LTS and 15.4 LTS ship Spark 3.5.x. Runtime 19 ships Spark 4.2.0 (July 23, 2026). Delta Lake tables can generate Iceberg metadata (UniForm, Iceberg reads) from Databricks Runtime 14.3 LTS so Iceberg clients read Delta data without rewriting files. | Amazon EMR 7.14.0 packages Spark 3.5.8-amzn-0, Delta 3.3.2-amzn-3, Iceberg 1.10.1-amzn-1 and Hudi 1.0.2-amzn-3. All three table formats are available on EMR. EMR on EKS supports the Kubernetes Operator for Apache Spark (Spark operator) as a job submission model with the EMR runtime on your own EKS clusters. The Kubeflow Spark Operator is Apache-2.0 licensed with Helm installation. |
| Where it runs | Databricks on AWS lists many AWS regions including us-east-1 (US East, Northern Virginia), us-east-2, us-west-1, eu-west-1, eu-central-1, ap-southeast-2 and us-gov-west-1; the platform is also hosted on Azure and Google Cloud. Some features are only in a subset of regions. | Amazon EMR has regional endpoints in the US (us-east-1, us-east-2, us-west-1, us-west-2) and many other AWS regions per the AWS general reference endpoints table. Spark runs on EC2 instances the user chooses (the AWS benchmark used r5d.4xlarge). |
| How open its benchmarks are | None found on 10 Nov 2026. | AWS publishes the setup in GitHub repositories (aws-samples/emr-on-eks-benchmark branch tpcds-v2.13_iceberg, and aws-samples/emr-spark-benchmark) with build instructions, so the TPC-DS Iceberg benchmark can be repeated. |
| Compute Bill | Databricks on AWS (Jobs Compute): $32.40 | Open stack: Spark on plain EC2: $20.16 |
How to read this page
The rows hold what each vendor or a named third party published, with no conclusion drawn. Benchmarks in the two columns ran on different hardware, data and baselines, so their numbers are not comparable across columns. The Compute Bill method explains the cost lines.
Sources
- Databricks Data Intelligence Platform on AWS: docs.databricks.com/aws/en/compute/photon, read 10 Nov 2026
- Databricks Data Intelligence Platform on AWS: docs.databricks.com/aws/en/compute/serverless/limitations, read 10 Nov 2026
- Databricks Data Intelligence Platform on AWS: docs.databricks.com/aws/en/compute/serverless, read 10 Nov 2026
- Databricks Data Intelligence Platform on AWS: docs.databricks.com/aws/en/release-notes/runtime, read 10 Nov 2026
- Databricks Data Intelligence Platform on AWS: docs.databricks.com/aws/en/delta/uniform, read 10 Nov 2026
- Databricks Data Intelligence Platform on AWS: docs.databricks.com/aws/en/resources/supported-regions, read 10 Nov 2026
- Databricks Data Intelligence Platform on AWS: databricks.com/blog/2021/11/02/databricks-sets-official-data-warehousing, read 10 Nov 2026
- Databricks Data Intelligence Platform on AWS: databricks.com/product/aws-pricing, read 10 Nov 2026
- Databricks Data Intelligence Platform on AWS: databricks.com/product/pricing/jobs, read 10 Nov 2026
- Open stack: Spark on EMR or EKS with Iceberg, Delta or Hudi: aws.amazon.com/emr/pricing, read 10 Nov 2026
- Open stack: Spark on EMR or EKS with Iceberg, Delta or Hudi: aws.amazon.com/blogs/big-data/run-apache-spark-and-iceberg-4-5x-faster-t, read 10 Nov 2026
- Open stack: Spark on EMR or EKS with Iceberg, Delta or Hudi: docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-app-versions-7.x, read 10 Nov 2026
- Open stack: Spark on EMR or EKS with Iceberg, Delta or Hudi: docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/spark-operato, read 10 Nov 2026
- Open stack: Spark on EMR or EKS with Iceberg, Delta or Hudi: iceberg.apache.org/docs/latest/maintenance, read 10 Nov 2026
- Open stack: Spark on EMR or EKS with Iceberg, Delta or Hudi: hudi.apache.org/docs/compaction, read 10 Nov 2026
- Open stack: Spark on EMR or EKS with Iceberg, Delta or Hudi: docs.delta.io/latest/optimizations-oss.html, read 10 Nov 2026
- Open stack: Spark on EMR or EKS with Iceberg, Delta or Hudi: docs.aws.amazon.com/general/latest/gr/emr.html, read 10 Nov 2026
- Open stack: Spark on EMR or EKS with Iceberg, Delta or Hudi: github.com/aws-samples/emr-on-eks-benchmark, read 10 Nov 2026
Last reviewed 10 Nov 2026.