Blog
Tag: #data-engineering (12)
-
Process JSON with jq
Tired of processing those API responses? jq will make you love JSON. Let's see how to filter and transform JSON to get what you want.
-
Format SQL Code for dbt
Stop fighting over spaces and commas in SQL. Use sqlfmt to settle debates and move on to the real work.
-
Load Data to Snowflake
Snowflake makes it easy to analyze data. The hardest part is getting data into snowflake. Let's see how it's done!
-
Slowly Changing Dimensions
Good data modeling updates records without losing the old ones. Kimball introduces slowly changing dimension tables to do just that.
-
What Is A Parquet File?
Parquet is all over the modern data stack. But do you understand how it works?
-
Python Slicing
You've heard of Python slices. We're going deeper into the world of slicing to process sequences more efficiently.
-
Airflow Incremental Loading
Big data can be difficult data. Design your Airflow pipelines to process data more efficiently with incremental loading.
-
Airflow Scheduling
Enjoy life and let Airflow handle the boring stuff. We're talking about how you can schedule those tasks to run for you.
-
Accio Database
Let's summon a database by scraping the Potter DB API!
-
Airflow Params
Want to make friends? Design your Airflow DAGs with Params. Your co-workers (or you 6 months from now) will thank you.
-
BigQuery Flavor of SQL
What are some of the unique features of BigQuery's GoogleSQL? Let's dive into some helpful syntax for day-to-day querying.
-
What is Data Engineering? (For Your Mom)
Have you ever had trouble explaining your job as a data engineer? This one's for you, Mom.