#042: Data Automation (CI/CD) with a Real Life Example
May 17, 2023One of the most fun aspects of being a data engineer is creating different automations.
One area that's really important is CI/CD, which stands for Continuous Integration and Continuous Deployment.
This is where you can automate your testing and release strategy.
But I also understand that this concept can be a little vague if you haven't seen it in action.
So in today's video I'll show you a real-life example of how to use Github to make this happen.
This is just one example of why people really like code-based tools because of the ability to automate and do things like this.
This can be applied not only to your deployments but as we'll mostly cover in this video the idea of automating your data quality checks.
Enjoy!
What will you learn?
- How data workflows are created on Github
- How to leverage pre-built automations
- How workflows get triggered
Looking for more? Here are 4 other ways I can help you:
- Modern Data Mastery (Course) - A clear & straightforward overview of concepts & strategies in modern data stacks so you can navigate your data career with confidence.
- The Playbook for dbt™ (Course) - A complete project-based course on dbt™ sharing functionality, best practices & strategies so you're ready to contribute on any team.
- Consulting - Hire me as a hands-on consultant to help complete your next data project.
- Sponsorship - Promote your product or brand to 5,000+ email subscribers and/or 21k+ YouTube subscribers.
When you're ready, here are 3 ways I can help you further:
1. The Modern Engineer Playbook (Online Course): Master the art of building modern data architectures. Get an end-to-end blueprint for building simple, structured & scalable data pipelines with modern tools.Â
2. The Playbook for dbt™ (Online Course): Learn to build, automate & scale dbt™ with confidence. Discover common strategies teams follow, avoid costly rookie mistakes & get actionable tips based on real-world experience, not a textbook.
3. Consulting Services:Â Request to work 1:1 with Michael Kahan to help you and your team implement a modern data architecture with a simple, scalable structure.