Streaming Pipeline with Kafka
Design a real-time streaming data pipeline using Apache Kafka with schema registry, consumer groups, and exactly-once processing semantics.
More in Data Engineering & ETL Prompts
Data Engineering: Design an ETL Pipeline with Airflow
Airflow DAG for a daily ETL with data quality checks.
dbt: Modeling Standards for a New Project
dbt project layout and naming conventions for a new data team.
Streaming: Design a Kafka Topic + Schema Strategy
Kafka topic design, partitioning, and schema evolution plan.
Data Engineering: CDC Pipeline from Postgres to Warehouse
Change data capture from Postgres to a warehouse via Debezium.