#022: The Hidden Cost of Open Source

Dec 03, 2022

Open source tools are free…right?

They offer full code access and you don’t have to pay for it!

But the truth is, the biggest cost in any implementation is not the tool itself.

Today, I‘ll share how open source tools have a big hidden cost - your time:

  1. Time to setup
  2. Time to support
  3. Time to optimize

 

Setup is rarely straightforward

With open source tools, you’ll need to custom fit it to your infrastructure.

And you’ll often hit errors or dependency issues.

While this is expected, developers tend to go in circles for far too long.

The tool may be free, but the developer’s time is not.

Make sure you have the right skillset - before starting.

The setup process can either be a great experience or a massive headache.

 

Remember: Every hour spent during setup is an hour not building a new data model.

 

When things break, it’s on you

As developers, building custom things gives us a sense of pride and value.

But if we build it, we also need to maintain it.

There’s no dedicated support team standing by.

This includes troubleshooting, integrating and on-boarding new team members.

Be deliberate in your design, provide great documentation and think long-term.

A short-sighted decision today can become a huge problem tomorrow.

 

Remember: Every hour spent fixing a bug is an hour not learning from stakeholders.

 

Is it customized or just hacked together?

Lastly, most tools or infrastructure designs have a “best practice”.

When you pay for a service they handle this on your behalf.

But the open source route means it’s up to you to adhere to it.

Or not to.

Use this opportunity to plan as a team and learn together.

You can also join tool-specific Slack/Discourse communities.

 

Remember: Every hour spent refactoring is an hour not adding new features.

 

Open source tools provide amazing value when used correctly.

And for the record, I’m a huge advocate and often recommend them.

But be careful not to assume open source = free.

Always consider the hidden cost of your time to setup, maintain and optimize.

 

PS - Many open source data tools fall under what's referred to as The Modern Data Stack.

To see a collection of most of these tools, check out my free PDF guide called The Starter Guide for the Modern Data Stack.

Level-up your abilities as a Data Engineer, faster.

Learn new data engineering tips, tricks and best practices every Wednesday.

Other Recent Posts

Data Automation (CI/CD) with a Real Life Example

May 17, 2023

3 Ways to Deploy Data Projects

May 10, 2023

The Importance of Virtual Environments

Apr 26, 2023

How to Create a Virtual Machine on GCP

Apr 19, 2023