Data Engineering

Things I learn,
build, and break.

Honest writeups on pipelines, distributed systems, and cloud infrastructure.

Posts

Coming soon
Medallion Architecture Lakehouse

Medallion Architecture: Why the Layers Actually Matter

Coming soon
Kafka Streaming

Scaling a Real-Time IoT Pipeline to 4 Lakh Concurrent Connections

Coming soon
CDC PySpark

Implementing CDC in PySpark Without a Framework

Coming soon
GCP BigQuery

GCP Data Engineering Stack: What Each Service Actually Does

Coming soon
Airflow Orchestration

Airflow DAG Patterns That Don't Break in Production