Orchestrate data workflows w/ PrefectMar 15, 2023
It's one thing to build a data stack. It's another to orchestrate it.
Plus, most modern stacks consist of many different tools which makes it all the more challenging.
Fortunately, there are tools designed to address this exact issue (task orchestration) and make our lives as data engineers easier.
So in today's video, I want to show you how to use a great open-source task orchestration tool called Prefect.
Prefect is a tool I've personally used on recent consulting projects to automate data workflows and have started to really enjoy using.
By the end of this video you'll understand what Prefect is all about and how you can easily start using it in your own data stack.
What will you learn?
- What Prefect is & how data teams use it
- How to create your first data workflow
- How to run workflows on Prefect Cloud (for free)
Looking for more? Here are 3 other ways I can help you:
- The Playbook for dbt™ - Learn exactly how to use dbt™ in a modern data workflow with best practices so you can confidently build & automate any project from scratch.
- Consulting - Hire me as a hands-on consultant to help complete your next data project.
- Sponsorship - Promote your product or brand to 5,000+ email subscribers and/or 18k+ YouTube subscribers.
Level-up your abilities as a Data Engineer, faster.
Learn new data engineering tips, tricks and best practices every Wednesday.