#059: Data Architecture 101: Kappa (Real-Time Data)

All things being equal, we'd all want real-time data.

This is the holy-grail of data engineering and removes most delays to insights.

One architecture approach that makes this possible is known as the Kappa Architecture.

It focuses on real-time data loading & processing rather than waiting on batches.

No brainer, right?

Well, back here in the real-world, things aren't made equal. 

Different architecture approaches require different levels of complexity (aka skill requirements & infrastructure).

Which impacts design & maintenance time.

Which all tends to mean higher overall cost of ownership.

All that to say, just because it's technically possible, doesn't always mean it's the best choice for your team.

Truthfully, I find it's usually not necessary.

BUT that doesn't mean you shouldn't be aware of it and be able to evaluate it.

So in this video we'll review at a high level what the Kappa architecture is about.

By the end, you'll understand the key points and be able to decide whether or not it makes sense for your team.

What will you learn? 

• What is the Kappa Architecture approach
• Key scenarios to consider when deciding if it's for you
• An example approach (with tools) 

Enjoy!