How to Create a 3 Layer Data Model Pipeline
May 01, 2024A three layer data model pipeline moves raw source data through staging, warehouse and marts, and each layer has exactly one job. Staging is a set of views, one per raw source table, that rename columns, fix data types and apply simple formatting a single time. The warehouse layer builds the actual data model, usually a Kimball style star schema of facts and dimensions, and it reads only from those staging views. Marts join the warehouse tables into wider, flattened tables that dashboards, spreadsheets and data feeds connect to, so end users never touch the warehouse directly. It's the structure I put in place whether a team is starting from scratch or cleaning up a warehouse that has grown out of control, and the same idea scales well past the small example below.
Key takeaways
- Three layers, three jobs. Staging cleans, warehouse models, marts serve. Each layer only reads from the one before it.
- Staging models are one to one with raw source tables and deploy as views, so you aren't paying to store a second copy of the source.
- Renames, type casts and simple case statements happen once in staging. Doing the same conversion in five downstream queries is how teams end up with conflicting numbers.
- The warehouse layer is where star schema modeling lives. Because staging already handled the cleanup, this layer can focus on real business logic.
- Marts only pull from the warehouse and are the only thing reporting tools connect to. Splitting marts by department or customer gives you control over visibility and security.
- Prefer a few wide, reusable mart tables over one mart per report. One per report creates fluff that goes stale.
- Most of the value is mental clarity, not query performance. When everyone knows where things live, changes get made faster and with less fear.
The three layers at a high level
Picture a simplified data stack. Sources sit on the left, an ingestion tool loads them into a central landing zone, and from there the data moves across a series of layers before it lands in reporting.
A data warehouse is the central hub for most data teams, and in my experience it's also where the headaches concentrate. Either what's been built has gotten unorganized, or the team is starting fresh and doesn't know which conventions to adopt.
The approach I follow splits the warehouse into three layers:
- Staging. A cleaned up, one to one copy of each raw source table.
- Warehouse. Where the data modeling happens.
- Marts. The tables users and reporting tools actually query.
One rule holds it together
Each layer reads only from the one before it. Staging reads raw. Warehouse reads staging. Marts read warehouse.
Nothing skips a step, and that single rule is what makes the whole thing work.
If you've seen my videos on star schema modeling, this is how the two fit together:
- The staging layer is the same staging.
- The warehouse layer is where the star schema lives.
- Marts are the wider, flattened tables built on top of it.
Layer 1: Staging
A staging model is a one to one representation of a raw source table, with a naming convention that tells you where it came from.
The important design decision is that I deploy these as views, not tables. A view is an abstraction on top of the source table, so you aren't doubling your storage for what is essentially the same data with better column names.
What happens in staging
The work here is deliberately boring:
- Renames. Bring every column to a consistent snake_case.
- Type casts. Turn a date stored as a string into an actual date.
- Simple case statements. Convert a status code into a readable label or a boolean.
- Formatting. Round or standardize numbers.
These are all things you can do right up front, one time, so that nobody downstream has to do them again.
Layer 2: Warehouse
The warehouse layer is where most of the data modeling takes place. For me that usually means a Kimball style star schema: fact tables holding the events and measurements, dimension tables holding the context around them.
The key constraint is that every model in this layer pulls from the cleaned up staging views, never directly from the source.
That restriction feels small, but it's what keeps the layer focused. Since renames and conversions were already handled, the warehouse can be dedicated to the custom logic that's actually specific to your business.
Layer 3: Marts
Marts pull only from warehouse tables. Not staging, not source.
These are the tables connected to reporting, whether that's a dashboard, a spreadsheet or a data feed to another system, and everything in the layer is built with that audience in mind.
Split marts by audience
I often break marts into separate buckets, by department or by customer, because different user groups need different slices of the data. Isolating them gives you more control over security and visibility.
The users on the other end don't need to understand how the warehouse is structured. You're serving it up to them.
Fewer, wider tables
One preference worth stating: I don't like creating a mart table one to one for each report. Some people do, but it gets messy and creates a lot of fluff that goes out of date.
I'd rather have a few wider tables that get reused across several purposes.
Example: the staging layer
The example project is small on purpose, but I've applied this same structure to much larger architectures and it holds up.
Two raw schemas, one staging schema
On the database side there are two raw schemas, one for data loaded from Google Sheets and one for NBA game data. In a more modern cloud database the raw data often sits in a completely separate database, but here it's all in one place.
Next to the raw schemas is a staging schema containing no tables, only views, each one named so you can tell at a glance which source it came from.
The dbt project mirrors the same shape: a models directory with staging, warehouse and marts folders, and inside staging a subfolder per source, gsheets and nba.
That alignment between the project and the database is intentional. Keep it clean, keep it organized, and only add complexity later when you need it.
What a staging model looks like
Here's the shape of a staging model for a staff sheet loaded from Google Sheets. The source has:
- All capitalized column names
- An
AS_OF_DATEstored as a string - Nothing derived
with source as (
select * from {{ source('gsheets', 'STAFF') }}
),
renamed as (
select
STAFF_ID as staff_id,
FULL_NAME as staff_name,
TEAM_NAME as team_name,
case when STATUS = 'A' then true else false end as is_active,
cast(HIRE_DATE as date) as hire_date,
cast(extract(year from cast(HIRE_DATE as date)) as integer) as year_hired,
cast(AS_OF_DATE as date) as as_of_date
from source
)
select * from renamed
Select from the raw table and you see capitalized names and a string date. Select from the view and you get:
- Clean snake_case names
as_of_dateas a real date- A new
year_hiredinteger that didn't exist in the source
The games model from the NBA source does the same kind of work, in that case flattening a JSON object into columns and adding a few simple booleans. In both cases the model reads from one source, one table, and only cleans it up.
Why one time matters
This looks simple, and it is. The point is that it's handled once, here, instead of what I see most teams do, which is redo the same conversions in several places.
That repetition leaves you open to human error and to two reports that disagree.
Repeat the pattern for every raw table, store the views in a staging schema in production, and you can group them by name.
Example: the warehouse layer
In the dbt project I split the warehouse folder into facts and dimensions, but when deployed they all land in the same warehouse schema as tables.
Every one of them references a staging model with ref('stg_...'), so you can see the lineage from the first line.
Business logic, not formatting
What stands out in the dimension models is how little is happening.
There will be cases where you need more logic, especially when you combine several staging models or need an intermediate common table expression in between, and that work belongs here.
But you aren't repeating the simple renaming and type conversion, because staging already did it. So the layer reads as business logic rather than formatting.
Example: the marts layer
The mart in this project is nba_games_detail. It joins the fact and dimension tables back together into one wide table with everything about each game on a given day, which is what an analyst or a dashboard actually wants to work with.
with games as (
select * from {{ ref('fct_games') }}
),
teams as (
select * from {{ ref('dim_teams') }}
),
final as (
select
games.game_id,
games.game_date,
home.team_name as home_team_name,
away.team_name as away_team_name,
games.home_score,
games.away_score,
games.home_score > games.away_score as home_team_won
from games
left join teams as home on games.home_team_key = home.team_key
left join teams as away on games.away_team_key = away.team_key
)
select * from final
Keep mart logic light
There may be a little renaming here that's specific to a report, but the goal is to limit it. Staying modular means a change gets made once and flows through.
You can add custom logic in a mart if needed. Most of the time it's better to handle it in the fact table at a low enough grain and simply pull it through.
Why data modeling still matters
It's tempting to get cute, build everything as one big table and skip the modeling. I think that leaves you exposed to overly complicated logic that's painful to maintain.
With three layers there's a clear process from raw to staging to warehouse to marts, with reporting on the end. Your team knows:
- Where everything lives
- How to update it
- Which conventions to follow
Most of this is about mental clarity, not performance. If you spend your time figuring out where something should go, or worrying about what you might break, that's time not spent on changes that actually move the business.
A structure everyone understands gives that time back.
Key terms
Staging layer
A set of views, one per raw source table, that rename columns, cast types and apply simple formatting once so downstream layers never repeat it.
Warehouse layer
The layer where the data model is built, typically a star schema of fact and dimension tables, reading only from staging views.
Marts layer
Wide, flattened tables built only from warehouse tables and connected directly to reporting tools, often grouped by department or customer.
Star schema
A Kimball style data model made up of fact tables holding events and measures, surrounded by dimension tables holding descriptive context.
Source
In dbt, a declared raw table that staging models read from with the source() function.
Common questions
Should staging models be views or tables?
Views, in almost every case. A staging model is a cleaned up copy of a source table, so materializing it as a table doubles your storage for no benefit. If a staging view is slow because the source is huge, that's a signal to look at the warehouse layer, not to turn staging into tables.
Can a mart read directly from a staging model?
I'd avoid it. The reason marts read only from the warehouse is that the warehouse is where business logic and keys are defined. If a mart reaches past it to staging, you've created a second path for the same logic and the two will eventually disagree.
How many mart tables should I create?
Fewer than you think. A handful of wide tables that serve several reports is easier to maintain than one table per dashboard. Add a new mart when a user group genuinely needs a different slice of the data, not every time someone asks for a report.
Does the three layer model work outside dbt?
Yes. The layers are a database structure, three schemas with clear rules about what reads from what. dbt makes it easy to mirror that structure in a project and manage the dependencies, but stored procedures or any other transformation tool can follow the same design.
How is this different from the medallion architecture?
It's a close cousin. Bronze, silver and gold roughly line up with raw, staging plus warehouse, and marts. The difference is mostly in emphasis: the three layer model is explicit that the middle layer is a data model, not just cleaner data.
Related reading
- Data Architecture with dbt: Putting It All Together
- What Are Intermediate Models in dbt?
- A Simple 4-Step Process for Creating dbt Models
- The True Value of a Data Presentation Layer
Final takeaway
This is the default structure I set up when I come into a small data team, whether they're building a warehouse for the first time or untangling one that grew without a plan. It's simple enough to explain in a meeting and strict enough that it holds up as the project grows.
Additional Free Resources
Starter Guides & Checklists
Explore additional free resources built on the same patterns I use with real clients so you can build your own with structure and confidence. Topics include data architecture, modeling and more specifically for small data teams.