StepRight Capstone Project

A complete, hands-on Databricks data engineering project built end-to-end: ingestion, transformation layers, gold-layer reporting, orchestration, unit testing, data quality monitoring, and CI/CD deployment.

Part 1 taught each Databricks pillar in isolation – Delta Lake, Unity Catalog, the medallion pattern, and the three Lakeflow pillars each got their own section, with only small, self-contained examples. Part 2 fuses all of it into one project you build start to finish: StepRight, a direct-to-consumer footwear and apparel retailer, taken from an empty catalog through bronze, silver, and gold layers, an orchestrated job, a unit and integration test suite, a data quality dashboard, and a CI/CD-driven deployment. Treat it as the dress rehearsal for Part 3: the same discipline – design before code, test before deploy, prove data quality rather than assume it – is what separates a demo pipeline from one an enterprise trusts with its legacy warehouse migration.

Sections

#	Section	Items
1	StepRight - Project Foundation	6
2	StepRight - Ingestion Layer	6
3	StepRight - Transformation Layers	5
4	StepRight - Gold Layer Reporting and Analysis	5
5	StepRight - Orchestration and Job Scheduling	4
6	StepRight - Unit Testing	4
7	StepRight - Data Quality Monitoring	5
8	StepRight - Integration Testing, Packaging and Deployment	7

---

| ← Previous: Check Your Knowledge | Next: StepRight - Project Foundation → | |:—|—:|

---

Table of contents

• StepRight - Project Foundation
• StepRight - Ingestion Layer
• StepRight - Transformation Layers
• StepRight - Gold Layer Reporting and Analysis
• StepRight - Orchestration and Job Scheduling
• StepRight - Unit Testing
• StepRight - Data Quality Monitoring
• StepRight - Integration Testing, Packaging and Deployment