{
 "name": "Lakehouse Compute Bill",
 "edition": "2026-11-10",
 "read_date": "2026-11-10",
 "source": "https://datalakehousecompare.com/datasets/",
 "license": "CC BY 4.0",
 "cite": "Lakehouse Compare, \"Lakehouse Compute Bill\", edition 10 Nov 2026, https://datalakehousecompare.com/datasets/ (CC BY 4.0).",
 "description": "Nightly Spark batch job, 10 nodes for 2 hours (us-east-1): the stated cost on each of 12 lines from published unit prices, with the formula and assumptions.",
 "rows": [
  {
   "group": "same EC2 nodes",
   "line": "Amazon EMR on EC2",
   "how_billed": "EC2 instance-hours plus an EMR price per instance-hour, each billed per second with a one-minute minimum.",
   "formula": "20 node-hours x ($1.008 EC2 + $0.252 EMR) = 20 x $1.26 = $25.20.",
   "cost_usd": 25.2,
   "cost_note": "Includes EC2 and the EMR price for ten r6i.4xlarge workers. Excludes the primary (driver) node, EBS volumes, S3 requests, data transfer and public IPv4 charges.",
   "unit_prices": "EC2 r6i.4xlarge (16 vCPU, 128 GiB), Linux, shared tenancy, on-demand: 1.008 per instance-hour; EMR price for r6i.4xlarge: 0.252 per instance-hour",
   "sources": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/us-east-1/index.json https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/ElasticMapReduce/current/us-east-1/index.json",
   "read_date": "2026-11-10"
  },
  {
   "group": "same EC2 nodes",
   "line": "Amazon EMR on EKS (EC2 nodes)",
   "how_billed": "EC2 instance-hours for the nodes, plus an EMR charge per vCPU-hour and per GB-hour of the pods. AWS also charges $0.10 per hour for each EKS cluster; that fee is excluded from this line.",
   "formula": "EC2: 20 node-hours x $1.008 = $20.16. EMR vCPU: 160 vCPU x 2 hours = 320 vCPU-hours x $0.007084 = $2.26688. EMR memory: 1,280 GB x 2 hours = 2,560 GB-hours x $0.000777875 = $1.99136. Total $20.16 + $2.26688 + $1.99136 = $24.41824, which rounds to $24.42.",
   "cost_usd": 24.42,
   "cost_note": "Includes EC2 and the EMR uplift for pods that request the full vCPU and memory of all ten nodes. Excludes the EKS cluster fee ($0.10 per hour per cluster, as on the open stack line), because it depends on how long the cluster stays up, not on the two-hour job; and it excludes the driver pod, EBS, S3 requests, data transfer and public IPv4 charges.",
   "unit_prices": "EC2 r6i.4xlarge (16 vCPU, 128 GiB), Linux, shared tenancy, on-demand: 1.008 per instance-hour; EMR on EKS, EC2 launch type, vCPU: 0.007084 per vCPU-hour; EMR on EKS, EC2 launch type, memory: 0.000777875 per GB-hour; EKS cluster fee: 0.1 per cluster-hour",
   "sources": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/us-east-1/index.json https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/ElasticMapReduce/current/us-east-1/index.json https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/ElasticMapReduce/current/us-east-1/index.json https://aws.amazon.com/eks/pricing/",
   "read_date": "2026-11-10"
  },
  {
   "group": "same EC2 nodes",
   "line": "Amazon EMR Serverless",
   "how_billed": "Per vCPU-hour and per GB-hour of the workers while they run, plus storage beyond the free ephemeral amount. No instances to manage.",
   "formula": "vCPU: 160 x 2 = 320 vCPU-hours x $0.052624 = $16.83968. Memory: 1,280 GB x 2 = 2,560 GB-hours x $0.0057785 = $14.79296. Total $16.83968 + $14.79296 = $31.63264, which rounds to $31.63.",
   "cost_usd": 31.63,
   "cost_note": "Includes vCPU and memory charges for workers that total 160 vCPU and 1,280 GB for two hours. Excludes extra ephemeral storage (the $0.000111 per GB-hour rate applies only beyond the free amount), the driver, S3 requests and data transfer. Workers are billed only while they run, so a job that scales in and out would be billed for less than the full two hours; that is not modeled.",
   "unit_prices": "EMR Serverless, x86 vCPU: 0.052624 per vCPU-hour; EMR Serverless, x86 memory: 0.0057785 per GB-hour; EMR Serverless, ephemeral storage beyond the free amount: 0.000111 per GB-hour",
   "sources": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/ElasticMapReduce/current/us-east-1/index.json https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/ElasticMapReduce/current/us-east-1/index.json https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/ElasticMapReduce/current/us-east-1/index.json",
   "read_date": "2026-11-10"
  },
  {
   "group": "same EC2 nodes",
   "line": "Databricks on AWS (Jobs Compute)",
   "how_billed": "DBUs consumed per node-hour (rate by instance type) times a price per DBU for the compute type and plan, plus the EC2 bill for the same nodes in the customer's account.",
   "formula": "DBUs: 20 node-hours x 4.08 DBU per node-hour = 81.6 DBU. 81.6 x $0.15 = $12.24. EC2: 20 x $1.008 = $20.16. Total $12.24 + $20.16 = $32.40.",
   "cost_usd": 32.4,
   "cost_note": "Includes the Databricks charge at the Premium plan Jobs Compute rate and the EC2 bill for the same ten r6i.4xlarge nodes, which is paid to AWS separately. Both Databricks figures come from the JSON data file that the Databricks pricing calculator page loads, because the pricing pages themselves are drawn by script; that file does not label a region, and the line assumes it applies to us-east-1. The Enterprise plan rate in the same file is $0.20 per DBU. Excludes the driver node, storage and S3 requests, and the Photon, serverless and committed-use options.",
   "unit_prices": "EC2 r6i.4xlarge (16 vCPU, 128 GiB), Linux, shared tenancy, on-demand: 1.008 per instance-hour; Databricks Jobs Compute, Premium plan, AWS: 0.15 per DBU; DBUs consumed by one r6i.4xlarge (a quantity, not a dollar price): 4.08 DBU per node-hour",
   "sources": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/us-east-1/index.json https://www.databricks.com/en-pricing-assets/data/pricing/AWS.json https://www.databricks.com/en-pricing-assets/data/pricing/AWS.json",
   "read_date": "2026-11-10"
  },
  {
   "group": "same EC2 nodes",
   "line": "Open stack: Spark on plain EC2",
   "how_billed": "EC2 instance-hours only. There is no managed-service charge; the team runs open-source Spark itself.",
   "formula": "10 nodes x 2 hours = 20 node-hours. 20 x $1.008 = $20.16.",
   "cost_usd": 20.16,
   "cost_note": "Includes EC2 on-demand compute for the ten workers. Excludes the driver node, EBS volumes, S3 requests, data transfer, and the engineering time to operate Spark, Iceberg and the cluster manager. If the same open stack ran on a self-managed EKS cluster, AWS would add $0.10 per hour for each EKS cluster (Amazon EKS pricing page, read 2026-11-10). That fee is excluded here, as on the EMR on EKS line, because it depends on how long the cluster stays up, not on the two-hour job.",
   "unit_prices": "EC2 r6i.4xlarge (16 vCPU, 128 GiB), Linux, shared tenancy, on-demand: 1.008 per instance-hour",
   "sources": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/us-east-1/index.json",
   "read_date": "2026-11-10"
  },
  {
   "group": "capacity mapped by a stated assumption",
   "line": "BigQuery (Enterprise edition slots)",
   "how_billed": "Per slot-hour for the edition used, billed per second with a one-minute minimum. A slot is described by Google as a virtual CPU.",
   "formula": "160 slots (one per vCPU of the reference job) x 2 hours = 320 slot-hours. 320 x $0.06 = $19.20.",
   "cost_usd": 19.2,
   "cost_note": "Includes slot compute only. Excludes storage, and excludes any data movement into or out of AWS. BigQuery runs SQL on Google's slots, not Spark on EC2, so this line prices the same amount of vCPU time, not the same Spark job. The text of the pricing page lists one rate for the Enterprise edition without showing which region the rate belongs to, so the region is not confirmed here.",
   "unit_prices": "BigQuery Enterprise edition, pay as you go, compute: 0.06 per slot-hour",
   "sources": "https://cloud.google.com/bigquery/pricing",
   "read_date": "2026-11-10"
  },
  {
   "group": "capacity mapped by a stated assumption",
   "line": "Dremio Cloud (Large engine)",
   "how_billed": "DCUs consumed per hour by engine size while the engine runs, times a price per DCU.",
   "formula": "Large engine = 256 DCUs per hour. 256 x 2 hours = 512 DCUs x $0.20 = $102.40.",
   "cost_usd": 102.4,
   "cost_note": "Includes engine DCUs for two hours of one Large replica. Excludes storage, LLM token charges and data transfer. The engines page lists executors per size but not the vCPU or memory of an executor, so which size equals the reference workload is an assumption. An XLarge engine (16 executors, 512 DCUs per hour) would give $204.80. Dremio Cloud runs SQL in its own engine, not Spark on EC2. The Dremio pricing page returned an access error to the fetch tool on 2026-11-10 and the $0.20 rate was read through a second fetch method.",
   "unit_prices": "Dremio Cloud DCU: 0.2 per DCU; Large engine replica, 8 executors: 256 DCUs per hour (a quantity, not dollars)",
   "sources": "https://www.dremio.com/pricing https://docs.dremio.com/dremio-cloud/admin/engines/",
   "read_date": "2026-11-10"
  },
  {
   "group": "capacity mapped by a stated assumption",
   "line": "DualBird on Amazon EMR (F2 instances)",
   "how_billed": "EC2 F2 instance-hours plus the EMR price for F2 instance-hours. DualBird's plugin price: none found on 2026-11-10.",
   "formula": "Infrastructure only, at the same node count and hours as the other lines: 20 node-hours x ($1.98 EC2 + $0.495 EMR) = 20 x $2.475 = $49.50.",
   "cost_usd": 49.5,
   "cost_note": "Includes only EC2 and EMR charges for ten f2.6xlarge nodes for two hours. The DualBird plugin price: none found on 2026-11-10, so it is not included. Excludes the primary node, EBS, S3 requests and data transfer. This line prices instances at the node count and hours of the reference workload. It does not say that ten F2 nodes are needed, or that fewer would do; DualBird sizing guidance: none found on 2026-11-10, and its benchmarks are its own (see the benchmarks list). DualBird is not a lakehouse. It speeds up Spark on EMR and EKS; a team on a fully managed platform cannot add it. It documents no Spark version list and no table-format support list; Iceberg appears only in its own compaction benchmark.",
   "unit_prices": "EC2 f2.6xlarge (24 vCPU, 256 GiB, one FPGA), Linux, on-demand: 1.98 per instance-hour; EMR price for f2.6xlarge: 0.495 per instance-hour",
   "sources": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/us-east-1/index.json https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/ElasticMapReduce/current/us-east-1/index.json",
   "read_date": "2026-11-10"
  },
  {
   "group": "capacity mapped by a stated assumption",
   "line": "Microsoft Fabric (F128 capacity)",
   "how_billed": "Pay-as-you-go capacity units (CU) per hour for the whole capacity SKU while it is running, not per job. One CU maps to two Spark VCores, per Microsoft.",
   "formula": "160 vCPU treated as 160 Spark VCores = 80 CU needed. The next SKU down, F64, has 64 CU, which is fewer than the 80 CU needed, so F64 is too small and the smallest F SKU with at least 80 CU is F128 (128 CU). 128 CU x 2 hours = 256 CU-hours x $0.18 = $46.08.",
   "cost_usd": 46.08,
   "cost_note": "Includes the F128 capacity for two hours. F128 is billed in full even though 80 CU are used, because F64 (64 CU) is too small for the stated mapping. Excludes OneLake storage, the Power BI license items Microsoft lists separately, and any pause and resume overhead. Fabric runs on Azure, not AWS, so this line prices the Azure US East capacity for the same vCPU count. The Fabric pricing page showed no rate in its text on 2026-11-10; the rate comes from the Azure Retail Prices API.",
   "unit_prices": "Fabric capacity, pay as you go, US East (Azure): 0.18 per CU-hour",
   "sources": "https://prices.azure.com/api/retail/prices?$filter=serviceName eq 'Microsoft Fabric' and armRegionName eq 'eastus'",
   "read_date": "2026-11-10"
  },
  {
   "group": "capacity mapped by a stated assumption",
   "line": "Onehouse (Quanton, bring-your-own-cloud)",
   "how_billed": "Onehouse Compute Units (OCU), where 1 OCU is a 4-core instance for 1 hour in the customer's account, plus support costs. The customer also pays AWS for the instances.",
   "formula": "OCU quantity: 160 vCPU / 4 cores = 40 OCU per hour x 2 hours = 80 OCU. Price per OCU: None published on 2026-11-10, so the Onehouse charge is not computed. AWS instance part, if the same r6i.4xlarge nodes were used: 20 x $1.008 = $20.16.",
   "cost_usd": null,
   "cost_note": "None published on 2026-11-10: a dollar price per OCU. The Onehouse pricing URL returned HTTP 404 and the billing documentation states the OCU definition but no dollar rate. The $1.008 EC2 price is shown only to size the AWS part; Onehouse's documentation lists its own standard instance types (Graviton m8g sizes), so the instances actually used would differ. Support costs are also unpublished.",
   "unit_prices": "EC2 r6i.4xlarge (16 vCPU, 128 GiB), Linux, shared tenancy, on-demand: 1.008 per instance-hour",
   "sources": "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/us-east-1/index.json",
   "read_date": "2026-11-10"
  },
  {
   "group": "capacity mapped by a stated assumption",
   "line": "Snowflake (XL standard warehouse)",
   "how_billed": "Credits per warehouse-hour by size, billed per second with a 60-second minimum, times a price per credit by edition, cloud and region.",
   "formula": "XL warehouse = 16 credits per hour. 16 x 2 hours = 32 credits x $3.00 = $96.00.",
   "cost_usd": 96,
   "cost_note": "Includes warehouse credits for two hours of an XL standard warehouse at the Enterprise on-demand rate. Excludes storage, cloud services credits beyond the daily allowance, and data transfer. The table lists credits per hour for each size but not the vCPU or memory behind each size, so which size equals the reference workload is an assumption. For the same hours, the Standard edition ($2.00 per credit) gives $64.00 and Business Critical ($4.00) gives $128.00; a 2XL warehouse (32 credits per hour) doubles each figure. Snowflake runs SQL and Snowpark in its own engine, not Spark on EC2.",
   "unit_prices": "Standard warehouse, XL: 16 credits per hour (a quantity, not dollars); On-demand credit, Enterprise edition, AWS US East (Northern Virginia): 3 per credit",
   "sources": "https://www.snowflake.com/legal-files/CreditConsumptionTable.pdf https://www.snowflake.com/legal-files/CreditConsumptionTable.pdf",
   "read_date": "2026-11-10"
  },
  {
   "group": "capacity mapped by a stated assumption",
   "line": "Starburst Galaxy (Pro tier)",
   "how_billed": "Credits per cluster-hour, by number of workers, times a price per credit that depends on tier, cloud and region. Galaxy runs Trino SQL, not Spark.",
   "formula": "The published table is 1 worker = 6 credits per hour, 8 workers = 48, 16 workers = 96 (linear). Ten workers is not listed; at the same 6 per worker it is 60 credits per hour. 60 x 2 hours = 120 credits x $0.50 = $60.00.",
   "cost_usd": 60,
   "cost_note": "Includes Galaxy credits for a ten-worker cluster for two hours at the Pro tier. Excludes storage, S3 requests, data transfer and the tier features. Enterprise tier is $0.75 per credit and Mission Critical $1.00 on the same page, which would give $90.00 and $120.00 for the same credits. The worker size in a Galaxy cluster is not stated on the page, so ten Galaxy workers are not shown to equal ten r6i.4xlarge nodes.",
   "unit_prices": "Galaxy Pro credit, AWS US East (N. Virginia), on demand: 0.5 per credit; Galaxy cluster consumption, 1 worker: 6 credits per hour (a quantity, not dollars)",
   "sources": "https://www.starburst.io/pricing/pricing-details/ https://www.starburst.io/pricing/pricing-details/",
   "read_date": "2026-11-10"
  }
 ]
}