Data Stacks, Hash Values & More (YouTube Q&A)

There has been a lot of movement in the data landscape lately, so instead of picking one topic I went back through recent comments and answered them one at a time. This round covers whether Microsoft Fabric changes the calculus on tooling, whether you should create a dbt snapshot for every source, and whether aggregation logic belongs in its own CTE or in the final select. It also covers whether SSIS counts as dead technology, whether a composite of a natural key and an effective date beats a hashed surrogate key, whether ETL and ELT can both live in the same stack, whether hash keys are too slow compared to auto incrementing keys, and why I never use dbt snapshots to build dimensions. The answers are below, one per question, in the order they came up.

Is Microsoft Fabric a good way to deliver data tooling in 2025?

Chris asked how useful Microsoft's approach is, framing Fabric as a form of traditional data layering.

My honest view is that strategy matters more than tools. The best tooling available still ends in chaos if the strategy underneath it is wrong.

The questions that decide the outcome are the same whatever logo is on the platform:

• How do you handle naming conventions?
• How is the database organized?
• How do you schedule and orchestrate the work?
• How do you approach data modeling?

Why the timing made this interesting

In October 2025, dbt and Fivetran announced they were merging. Two of the most significant tools in the modern data world, combining.

We had a lot of separate point tools and they are consolidating, which strikes me as a natural progression. Whether that is good or bad is opinion, and the same goes for Fabric.

Should you create a dbt snapshot for every source?

The short answer is no. Context always matters, but as a default I do not recommend one snapshot per source.

A snapshot in dbt records the history of a source table. If you have a slowly changing source whose history you do not want to lose, a snapshot does that job well.

The cost is maintenance, not storage

Every snapshot is another file you have to manage, and you can never delete one without throwing away the history it accumulated. The earlier you start, the more history you collect, so waiting has a cost too.

But most teams I see love having history and rarely use it. Adding a snapshot later is not difficult, so start without them and add one when losing that history would actually hurt.

Should dimension logic go in its own CTE or in the final CTE?

The question was about grouping booking dates by user ID: does that belong inside the dim_booking CTE or in the final CTE?

Hard to answer precisely without the environment in front of me. The real question is whether aggregations get their own CTE or get folded into the final select.

My rule

If it is a distinct block of logic, give it its own CTE and bring the result into the final one. An aggregation is a distinct block of logic.

Then the final CTE is mostly just the column list you want to return. You should not be making many changes there.

with bookings as (

    select * from {{ ref('stg_bookings') }}

),

booking_metrics as (

    select
        user_id,
        min(booking_date) as first_booking_date,
        max(booking_date) as last_booking_date,
        count(*)          as booking_count
    from bookings
    group by 1

),

final as (

    select
        user_id,
        first_booking_date,
        last_booking_date,
        booking_count
    from booking_metrics

)

select * from final

It depends on the situation, but that is the shape I reach for.

Is SSIS old technology you should move off of?

A comment on one of my earliest videos, on building an ETL package in SQL Server Integration Services, said modern cloud tools like dbt and Snowflake seem easier.

Technically yes, SSIS is older technology. But old is not the same as useless, and a lot depends on who is running it.

Why I would still pick dbt

I prefer dbt because transformation logic as code is easier to live with. SSIS is graphical, which makes version control and change tracking awkward, and that is my complaint about graphical tools generally.

So the commenter is not wrong.

What I would tell a team on it

If I were starting over today, I would not start there. But SSIS that is managed and organized properly can still be useful, and working software nobody is fighting with is not an emergency.

If you are on it and it works, you can run it a long time before you need to replace everything.

Can you use a natural key and an effective date instead of hashing?

This one went deep. The argument: build a composite key from the natural key plus an effective date, because hashing the whole row just repeats work the slowly changing dimension pipeline already did.

A unique ID and a date do produce a unique value. If that is working for you, use it.

Where I would push back

I agree you should not hash every column in the row. Identify the columns that actually determine uniqueness and use those. Two or three is fine, a few more is fine.

The goal is a value unique across every record, not a hash of everything you have.

Composite keys get awkward when there is no true natural key in the source, which happens more often than you would like. I prefer to control uniqueness in the warehouse, because then I know exactly what makes a row unique and can change that definition later.

Is there a place for both ETL and ELT in a modern stack?

The proposal was two routes: ETL for complex transformations outside the database, ELT for simpler ones inside it, with an orchestration layer coordinating both. Yes, you can do that.

What I reach for

I like landing everything in one centralized place and transforming on top of it: raw data into a landing zone in a cloud database, then changes applied there. That keeps it consistent and easy to track.

An orchestration layer on top is fine, but I would not assume you need one on day one. With a single pipeline, a cron job or a scheduled job in dbt may be all you need.

The question I would actually ask

Is the extra route going to change a decision? As engineers we like building complex designs, and it is genuinely fun to do.

But if you simplified to one database, a few transformations and a batch refresh, would the insight be 95 percent as good, or 99 percent? Every extra layer brings monitoring and maintenance with it.

Aren't hash keys slower than auto incrementing surrogate keys?

The comment argued that auto incrementing keys create lookup dependencies that block parallel loading, while integer keys join more efficiently for BI tools downstream.

Is a hash key slow? It is relative. Slower than an auto incrementing integer, sure.

Prove it is a problem first

My question back is whether you have shown it is slow enough to matter. On Snowflake, Databricks or BigQuery it is not noticeably slow, maybe a few milliseconds.

Unless you have very large volumes and complex joins, that is not worth redesigning around, and most of the people I work with are nowhere near that point.

Why I still prefer the hash

The same input always produces the same output. Drop and recreate a table and the hash keys come back identical, because the source columns did not change.

Auto incrementing keys do not behave that way. Recreate the table and the ordering shifts, so 1, 2, 3, 4, 5 is not the same set of rows it was before, and the same build in another environment produces different keys.

I believe the word for this is idempotent: same inputs, same results, however many times you run it.

The storage trade-off is real

A hash key does take more space than an auto incrementing ID, and integer joins are more efficient. Both mattered more in older database systems.

On current cloud warehouses the impact on performance and cost is small enough that I would test before treating it as a constraint.

Why not use dbt snapshots to build dimensions?

Snapshots handle the slowly changing dimension process for you, so the question is fair: why recreate the comparison logic and version columns yourself? Technically you can. I never do.

Control over warehouse logic

For warehouse tables I want control over the logic. Hand it to the snapshot feature and you are locked into how that feature does things, when I would rather own the comparison.

They are built for different jobs

A snapshot is designed to follow one table and track its history. A dimension usually pulls several tables together into one entity.

If your dimension is effectively a copy of one source table, you can probably get away with it. Most are not, and using a snapshot to merge sources puts it outside its purpose.

It may work at first, until you want to customize something and end up reconfiguring around the feature's assumptions.

The rule I use

A snapshot is the same idea as tracking raw history through an ingestion tool, with the transformation layer doing it instead of the loader.

• Use snapshots on top of raw data, where the job is preserving history from one source.
• Avoid snapshots when you are merging sources or doing custom data modeling.

Related reading

• Data Modeling, dbt Configs & More (YouTube Q&A)
• dbt Snapshots, Workflow & More (YouTube Q&A)
• The Key To Building A Reliable Data Model
• 3 Strategy Killers for Data Teams

Final takeaway

Almost every question here resolved the same way: the technically interesting answer and the right answer for a small team are often not the same thing. With clients I keep coming back to whether the extra snapshot, the extra pipeline or the extra layer changes a decision anyone makes, and most of the time it does not.