Avoiding Vendor Lock-in With Your Data Stack

architecture leadership Jun 24, 2026

Vendor lock-in is the fear of picking something today that you cannot walk away from in three years. A more useful way to frame it comes from Jeff Bezos by way of the book Fundamentals of Data Engineering: sort each decision into a one way door or a two way door, reversible or irreversible, and spend your worry on the ones you genuinely cannot undo. For most small data teams, the database and the ingestion tool are two way doors, because a clear strategy means you can move data from one place to another without the architecture falling apart. The decision that is closest to a one way door is not a tool at all. It is your data model, because it compounds every week the team works on it.

Key takeaways

Treat architecture choices as one way doors or two way doors. Most tool decisions are two way doors and do not deserve months of agonizing.
Strategy is more durable than tooling. If you know where the data is going and in what layers, swapping a tool is work, not a rebuild.
Open table formats like Iceberg, Delta Lake and DuckLake reduce lock-in, and they are also easy to over-engineer on a small team.
Custom ingestion gives you full control and hands you every API change and upgrade. Prebuilt connectors give that back in exchange for less flexibility.
The data model is the most irreversible decision, because it is the one the team keeps building on after the tools are installed.
Decide where the layers are before you write much logic. Raw, staging, warehouse and marts, or a medallion structure, as long as it is explicit.
AI tools help most with consistency and documentation. They should not be the ones making the irreversible calls.

One way doors and two way doors

When you lead a data team or you are setting up an architecture from scratch, the stress is rarely about the work itself. It is the worry that you are making a decision you will regret in six months.

I was rereading Fundamentals of Data Engineering by Joe Reis and Matt Housley, and the section on what makes a good data architecture includes a quote from Jeff Bezos about his decision making at Amazon. He sorted decisions into one way doors and two way doors.

What the framework asks

A one way door is a decision you cannot come back from, or at least not easily. A two way door is one you can revert without it being the end of the world.

The useful move is deciding which kind you are standing in front of. Most of the decisions that keep data leads up at night turn out to be two way doors, and naming that takes the pressure off.

It also connects to something I say constantly: strategic decisions matter more than individual tools. Set the architecture up well and swapping tools in and out gets much easier.

Example 1: picking a database

The database is a core component. It is where all of your data lives, and it is where the fear of lock-in shows up first.

That fear is a big reason open table formats are getting popular. Iceberg, Delta Lake and the DuckLake architecture all lean toward more control, more compatibility across tools, and less dependence on one vendor.

Why that is not automatically the answer

Put that way it sounds like a no-brainer, and for the record I am a fan. These formats are genuinely innovative.

But if you do not have much data or much complexity, you probably do not need to go that route yet. You can still move your data around without overhauling your whole strategy.

If you do have a lot of data or real complexity, an open format can be the thing that makes you feel in control. The point is to choose on purpose rather than out of anxiety.

The questions I would ask instead

What is your team's internal skill set? Not what you could learn, what you can run next quarter.
What are the dynamics of your data? Volume, change rate, and how many sources are coming.
Where is your effort best spent? Bandwidth on a small team is the real constraint.

Do you want to spend your time managing an architecture with multiple hops and levels, or accept a bit more lock-in and spend that time fixing the data model and presenting data to the business?

Example 2: ingestion tooling

Ingestion is the second place this comes up. For some teams the whole process is heavily custom, to the point where moving away from it would be genuinely hard.

Off the shelf tools with prebuilt connectors make that easier to leave. You are mostly pointing a source at a destination.

The trade-off runs both ways

Custom. You own all the logic and have full control. You also own every API change, every upgrade, and the time that takes.
Prebuilt. Someone else handles the connector internals. You trade some flexibility for time back, and worst case you switch tools.

Either way, this is a two way door as long as your strategy is in place. If you know where the data needs to end up, ingestion is just the process of getting it there.

Example 3: your data model

The first two examples were tools. This one is not, and that is exactly why it matters more.

Data modeling is about your internal strategy and your team's ability to organize logic consistently. Tools help, but they do not decide it for you.

This is the closest thing to an irreversible decision in most stacks. Tools get set up once and then sit there, while the model is an ongoing process, so the choices compound in both good and bad directions.

Not impossible, just expensive

I am not saying you can never recover. Migrations and remodeling projects happen all the time.

But this is the area that becomes a much bigger headache than any tool selection, so it deserves the most thought up front.

Get the pipeline layers explicit

On a tactical level, be clear about the modeling pipeline you are putting data through. I usually describe it as a raw landing zone, a staging layer, a warehouse layer and a marts layer.

You might follow a medallion structure instead. What matters is that the layers are named and the team understands where things go.

Then when someone new joins, or you need to revert or change something, you are not tearing down the architecture to do it.

Where AI fits in

This is an area where AI tools genuinely help: organization, consistency and documentation. Those are exactly the things humans get lazy about.

Everybody writes SQL a little differently. Some people document, some do not, and on a one or two person team the structure tends to drift.

Stay in the driver's seat

If you hand all of these decisions to an AI tool and you are not driving, it will take you somewhere and you will not fully understand how you got there.

If you understand the strategy and you know which decisions are irreversible, the same tool becomes useful. It improves your prompting, your discussions and your documentation without quietly choosing your architecture.

Key terms

Vendor lock-in

The feeling of being stuck with a specific tool or platform you may not want to keep three to five years from now, usually because your data or logic cannot move easily.

One way door decision

A choice you cannot reverse without serious cost, as opposed to a two way door you can walk back if it turns out to be wrong.

Open table format

A storage format such as Iceberg or Delta Lake that keeps table data readable by multiple engines, so your data is less tied to a single vendor's database.

Prebuilt connector

A ready made integration from an ingestion platform that handles the source API, authentication and schema changes so your team does not write that code.

Data modeling pipeline

The named sequence of layers your data passes through, such as raw, staging, warehouse and marts, which is the part of the stack hardest to change later.

Common questions

What actually causes vendor lock-in on a data team?

It is less about the contract and more about how much logic lives inside one product. If your transformations, orchestration and business rules are all expressed in a single vendor's proprietary syntax, moving means rewriting them. Data itself is usually the easier part to move.

Do I need Iceberg or Delta Lake to avoid lock-in?

Not necessarily. Open table formats help when you have real volume, multiple engines, or a long horizon, but for a small team they can be extra architecture to maintain for a risk you do not have yet. The portability of your model matters more than the file format.

Is switching data warehouses realistic?

It is work, but it is not a rebuild if your layers are clear. Teams do it regularly. The pain scales with how much undocumented logic sits outside the modeling layer, not with the size of the data.

Why is the data model harder to reverse than the tools?

Because the team keeps adding to it. A tool is installed once, while the model grows every week, and every report and downstream process inherits its assumptions. Changing it means renegotiating all of that at the same time.

How do I decide without over-thinking it?

Ask whether the decision is a one way door. If it is reversible, pick the option that fits your team's skills today and move on. Save the long deliberation for the model and the layer structure.

Final takeaway

The teams I work with who worry least about lock-in are not the ones with the most portable tools. They are the ones with a clear strategy for where data goes and what each layer is for, because that is what makes a tool swap an inconvenience instead of a crisis.

Additional Free Resources

Starter Guides & Checklists

Explore additional free resources built on the same patterns I use with real clients so you can build your own with structure and confidence. Topics include data architecture, modeling and more specifically for small data teams.

Browse Resources