The System-Table Query Pack and Governance Guardrails

Estimated read: 3 min

Close this section out with the reference material you’ll actually reach for during a live FinOps review, plus the governance rules that keep the system catalog itself from becoming a liability.

The query pack. Beyond the billing join from the first lecture, three more system tables round out a working FinOps toolkit:

-- Top 10 most expensive clusters, last 7 days
SELECT c.cluster_name, SUM(u.usage_quantity * p.pricing.default) AS spend_usd
FROM system.billing.usage u
JOIN system.compute.clusters c ON u.usage_metadata.cluster_id = c.cluster_id
JOIN system.billing.list_prices p
  ON u.sku_name = p.sku_name AND u.cloud = p.cloud
  AND u.usage_start_time >= p.price_start_time
  AND (u.usage_start_time < p.price_end_time OR p.price_end_time IS NULL)
WHERE u.usage_date >= current_date() - INTERVAL 7 DAYS
GROUP BY 1 ORDER BY 2 DESC LIMIT 10;

-- Who queried the billing schema itself this month (audit trail on the FinOps data)
SELECT user_identity.email, action_name, event_time
FROM system.access.audit
WHERE request_params.full_name_arg LIKE 'system.billing%'
  AND event_date >= date_trunc('month', current_date());

system.compute.clusters (records cluster configuration, joinable on cluster_id) fills in the human-readable names and node types that usage_metadata only gives you as IDs. system.access.audit is the account’s audit log – every action, every user, every API call – and its role here is specifically to answer “who’s looking at cost data,” which matters once chargeback numbers start influencing budget decisions.

Governance guardrails. The system catalog is Unity Catalog-governed like any other catalog, which means the access-control mistakes that leak sensitive data elsewhere in your lakehouse apply here too, with a cost-data-specific twist:

  • Scope SELECT on system.billing tightly. Raw usage and pricing data reveals per-workload spend at a granularity most organizations don’t want broadly readable – grant it to a FinOps role and the dashboard’s service principal, not to account users.
  • Never expose system.access.audit broadly. It’s the audit trail itself; read access to it should be as restricted as write access to a production audit log in any legacy compliance regime.
  • Front raw system tables with governed views, not direct grants, the same ABAC pattern from the Unity Catalog governance section – a view that pre-joins usage and list_prices and exposes only team-scoped rows to a given group lets you hand a team lead self-serve access to their spend without granting them visibility into every other team’s.
  • Treat retention-policy and chargeback-taxonomy tables as governed data too – they determine what gets billed to whom and, eventually, what gets deleted, so they deserve the same change-control rigor as the tag schema they encode.

One more query pattern earns a place in the pack because it’s the question every FinOps review eventually reaches: “is this month’s number consistent with the taxonomy we agreed on.”

-- Month-over-month spend by team, to catch taxonomy drift or a sudden untagged spike
SELECT
  date_trunc('month', u.usage_date) AS month,
  COALESCE(u.custom_tags['team'], 'Untagged') AS team,
  SUM(u.usage_quantity * p.pricing.default) AS spend_usd
FROM system.billing.usage u
JOIN system.billing.list_prices p
  ON u.sku_name = p.sku_name AND u.cloud = p.cloud
  AND u.usage_start_time >= p.price_start_time
  AND (u.usage_start_time < p.price_end_time OR p.price_end_time IS NULL)
WHERE u.usage_date >= add_months(current_date(), -6)
GROUP BY 1, 2
ORDER BY 1, 3 DESC;

Run that one on the same cadence as the chargeback dashboard’s monthly review, before the meeting rather than during it – a team whose “Untagged” bucket jumped from 2% to 15% of their spend this month is a tagging regression worth a five-minute Slack message, not a surprise that surfaces for the first time in front of Finance.

Put together, the query pack and the guardrails around it are what separate a FinOps program you can defend in an audit from a set of ad hoc queries someone runs when a bill looks high. The system tables give you the raw truth; the governance rules make sure that truth is trustworthy, attributable, and safe to hand to the people who’ll act on it.

Reference: System tables overview, audit log system table, and compute system tables.


| ← Previous: Auto-TTL: The 2026 Frontier | Next: Check Your Knowledge → | |:—|—:|


Back to top

Independent, self-authored Databricks companion notes.

This site uses Just the Docs, a documentation theme for Jekyll.