Data Engineering: Design an ETL Pipeline with Airflow
Airflow DAG for a daily ETL with data quality checks.
Prompts for data pipelines, ETL workflows, data warehouse design, Apache Airflow DAGs, dbt models, and data quality frameworks.
15 Prompts gefunden
Airflow DAG for a daily ETL with data quality checks.
dbt project layout and naming conventions for a new data team.
Kafka topic design, partitioning, and schema evolution plan.
Change data capture from Postgres to a warehouse via Debezium.
Great Expectations suite for a critical table.
Generate production-ready Airflow DAGs with task dependencies, error handling, SLA monitoring, and dynamic tas...
Design a layered dbt project structure with staging, intermediate, and mart models following best practices fo...
Design a data warehouse schema with fact and dimension tables, slowly changing dimensions, and query-optimized...
Design a robust error handling framework for ETL pipelines with dead letter queues, retry strategies, data qua...
Build a data quality framework with schema validation, statistical anomaly detection, referential integrity ch...
Design a real-time streaming data pipeline using Apache Kafka with schema registry, consumer groups, and exact...
Design a data lake structure with medallion architecture, partitioning strategies, file format selection, and...