Check Your Knowledge

Test your understanding of the cost iceberg, compute arbitrage, cluster policies, Photon spill, and the all-purpose ETL anti-pattern before moving on to FinOps engineering.

  1. In the “cost iceberg” analogy, what sits above the waterline – the part most cost reviews stop at? A. Cross-cloud egress B. Cluster compute (DBUs) C. Untracked Delta history and orphaned checkpoints D. Photon licensing bundled into the DBU rate

  2. Which of the following is described as a “below the waterline” cost that a compute-only review typically misses? A. The DBU rate of an all-purpose cluster B. Idle all-purpose clusters left running between interactive sessions C. The instance type selected for a jobs cluster D. The Spark version pinned in a cluster policy

  3. Why does jobs compute cost less per DBU than an all-purpose cluster running the identical instance type? A. Jobs compute uses slower, cheaper hardware B. Jobs compute is created for one run and torn down after, with no idle-sharing capability to pay for C. Jobs compute does not run Photon D. Jobs compute is only available on a free tier

  4. What question does the lecture recommend asking to decide which compute type a workload should run on? A. How large is the dataset? B. Does a human need to interact with this compute session while it runs? C. Is the workload written in SQL or Python? D. How many tables does the workload read from?

  5. In a cluster policy JSON definition, what does "cluster_type": {"type": "fixed", "value": "job"} accomplish? A. It sets the default Spark version for all clusters B. It structurally prevents compute created under that policy from becoming a long-lived, shared all-purpose cluster C. It disables Photon for all clusters under the policy D. It forces all clusters to auto-terminate after 15 minutes

  6. Why is it important to audit existing always-on clusters after rolling out a new jobs-compute-only policy? A. Cluster policies are enforced at creation time, not applied retroactively to clusters already running under the old rules B. Policies automatically delete non-compliant clusters within 24 hours C. Existing clusters are exempt from billing until the audit completes D. Policies only apply to serverless compute, never to classic clusters

  7. Why can a Photon-accelerated query end up costing more, not less, than the same query without Photon? A. Photon always uses a larger instance type B. When memory pressure forces a shuffle or join to spill to disk, the slower disk I/O can make the query run longer, and billing is by compute-time C. Photon disables auto-termination D. Photon requires a dedicated all-purpose cluster

  8. What is the recommended first step for fixing a query that’s spilling to disk, before resizing the cluster? A. Switch the workload to a smaller instance type B. Prune and cluster the underlying data so less is scanned and shuffled in the first place C. Disable Photon entirely D. Move the workload to an all-purpose cluster

  9. According to the anti-pattern lecture, what is the actual, solvable objection engineers raise against moving scheduled ETL off all-purpose clusters? A. Jobs compute cannot run Python code B. Debuggability – there’s no live cluster to attach to when a scheduled job fails C. Jobs compute does not support Delta Lake D. All-purpose clusters are required for Unity Catalog access

  10. Which feature lets a failed multi-task job be re-run starting only from the failed task, on fresh jobs compute, instead of re-running the whole job on an all-purpose cluster to reproduce the failure? A. Liquid Clustering B. Repair Run C. Credential vending D. Auto Loader

Answer Key

  1. B – Cluster compute (DBUs) is the visible, easy-to-track cost that most reviews stop at.
  2. B – Idle all-purpose clusters left running between sessions is exactly the kind of hidden cost below the waterline.
  3. B – Jobs compute is ephemeral and single-purpose, so it’s priced without the idle-sharing capability all-purpose compute carries.
  4. B – Whether a human needs to interact with the session determines whether all-purpose is buying something real or is just habit.
  5. B – A fixed cluster_type: job value makes it impossible for compute under that policy to become a long-lived all-purpose cluster.
  6. A – Policies govern cluster creation; clusters already running under old rules are untouched until separately addressed.
  7. B – Disk spill slows the query, and since billing is by compute-time, a slower spilling query costs more despite Photon.
  8. B – Pruning and clustering reduces the data scanned and shuffled, addressing the cause of spill before touching hardware.
  9. B – Debuggability is the real, solvable concern – not a reason to leave production ETL on the pricier all-purpose rate permanently.
  10. B – Repair Run re-executes just the failed task with the same parameters on fresh jobs compute, without re-running the entire job.

| ← Previous: Anti-Pattern: ETL on All-Purpose Because It’s Easier to Debug | Next: FinOps Engineering: System Tables and Chargeback → | |:—|—:|

Back to top

Independent, self-authored Databricks companion notes.

This site uses Just the Docs, a documentation theme for Jekyll.