I started out my career as an analytics engineer and then a data engineer for a small but growing company that wanted to be data driven, but didn’t have the resources to build out a full data platform.

I was tasked with building out data pipelines to move data from our data warehouse to our data lake.

It didn’t take long for me to realize there weren’t a whole lot of tools that were designed to just build simple and scalable data pipelines for those that airflow or dagster were overkill for.

Since then, I’ve moved on to other companies and found similar issues and decided to take my crack at building a pipeline tool. The Python version is currently open source and available on PyPi.

Welcome to Pipeweave.