Hey there, fellow data engineers! I’m currently working on a retail data analytics project and I’m stuck at the Snowflake+DBT stage. I’ve managed to set up the data pipeline from Airflow to S3, but now I’m not sure how to approach data quality checks and transformation using DBT. My SQL skills are somewhere between beginner and intermediate, so I’m looking for some guidance.
If you’re like me, you’ve probably seen the DBT core tutorial on the DBT website, but now it’s only available for DBT cloud. So, where do you start? How do you set up data quality checks and transformation? Are there any resources that can help?
In this post, I’ll share my journey of getting started with DBT and how I overcame my initial struggles. I’ll cover the basics of DBT, how to set up data quality checks, and some tips for data transformation. By the end of this post, you’ll have a better understanding of how to approach the DBT stage and how to make the most out of this powerful tool.
So, let’s dive in!