Delta Lake - Deep Dive

The storage layer underneath every table in this guide: what Delta Lake actually is and why its transaction log matters, creating and managing tables, batch vs. streaming reads/writes, time travel and rollback, routine maintenance (OPTIMIZE/VACUUM), the MERGE INTO upsert engine, idempotent writes, schema enforcement and evolution, the VARIANT type, and table constraints. This is the longest section in Part 1 for a reason – it’s the foundation everything from Unity Catalog onward assumes you already know cold.

flowchart TD
    A[Transaction Log] --> B[ACID writes]
    A --> C[Time Travel]
    A --> D[RESTORE]
    B --> E[MERGE INTO<br/>idempotent upserts]
    B --> F[Schema Enforcement<br/>+ Evolution]
    E --> G[OPTIMIZE + VACUUM<br/>maintenance]
    F --> H[Constraints<br/>NOT NULL, CHECK]

Lectures

#	Lecture	Est. Read
1	What is Delta Lake and why it matters	17 min read
2	Creating and Managing Delta tables	15 min read
3	Reading and writing Delta - batch and streaming	14 min read
4	Time Travel - querying historical versions	10 min read
5	OPTIMIZE, VACUUM and Data Retention	16 min read
6	RESTORE and Rollback Strategies	9 min read
7	MERGE INTO - the Delta Upsert Engine	17 min read
8	DELETE, UPDATE and Idempotent writes	11 min read
9	Schema Enforcement and Schema Evolution	14 min read
10	Type Widening and the Variant Data Type	13 min read
11	Table constraints - NOT NULL, CHECK, and Identity Columns	11 min read
12	Check Your Knowledge	Quiz

| ← Previous: Check Your Knowledge | Next: What is Delta Lake and why it matters → | |:—|—:|

Delta Lake - Deep Dive

Lectures

Table of contents