Biapy's Bookmarks

DLT-META

https://databrickslabs.github.io/dlt-meta/

apache-spark dataflow data-lake data-pipeline declarative metadata source-available

Added 3 months ago

HelixML

https://helix.ml/

ai ai-agent api commercial data-pipeline rag source-available

Added 4 months ago

Singer

https://www.singer.io/

apache2-licensed data-pipeline data-transformation etl foss open-source

Added 4 months ago

Meltano

https://meltano.com/

data-integration data-pipeline declarative etl foss mit-licensed open-source

Added 4 months ago

Bruin

https://getbruin.com/

apache2-licensed data-pipeline data-transformation etl foss open-source python self-hosted sql web-app

Added 5 months ago

Semlib

https://semlib.anish.io/

data-analytics data-pipeline foss library llm mit-licensed open-source python semantic

Added 7 months ago

Flow PHP

https://flow-php.com/

data-pipeline development foss framework mit-licensed open-source php

Added 8 months ago

Dataframely

https://dataframely.readthedocs.io/en/latest/

bsd3-licensed data-pipeline data-validation foss open-source polars python

Added 11 months ago

Mage AI

https://www.mage.ai/

apache2-licensed data-pipeline elt etl foss framework open-source workflow

Added 1 year ago

Apache Kafka

https://kafka.apache.org/

apache2-licensed big-data data-integration data-pipeline data-stream event-stream foss kafka open-source self-hosted

Added 1 year ago

superglue

https://superglue.cloud/

api-gateway data-pipeline foss legacy machine-learning open-source self-hosted web-app

Added 1 year ago

Numaflow

https://numaflow.numaproj.io/

data-pipeline data-stream foss kubernetes open-source

Added 1 year ago

Sparrow

https://sparrow.katanaml.io/

ai data-pipeline foss llm ocr open-source parser self-hosted web-app

Added 1 year ago

Data Engineering Zoomcamp

https://github.com/DataTalksClub/data-engineering-zoomcamp

data-engineering data-pipeline data-science e-learning open-source

Added 1 year ago

SQLMesh

https://sqlmesh.readthedocs.io/en/stable/

data-analytics data-pipeline data-transformation dbt foss open-source self-hosted sql web-app

Added 1 year ago

ComfyUI

https://www.comfy.org/

data-pipeline foss genai image open-source self-hosted stable-diffusion web-app

Added 1 year ago

Airbyte

https://airbyte.com/

data-integration data-pipeline elastic-licensed etl llm open-source self-hosted

Added 1 year ago

Dagster

https://dagster.io/

data-pipeline data-science data-stream foss observability open-source orchestration self-hosted

Added 1 year ago

data stack in a box

https://github.com/wisemuffin/nsw-doe-data-stack-in-a-box

data-analytics data-pipeline data-science data-stack data-stream open-source self-hosted

Added 1 year ago

Windmill

https://www.windmill.dev/

automation data-pipeline developer-experience devops open-source scheduler self-hosted web-app workflow

Added 1 year ago

GlareDB

https://glaredb.com/

apache api csv data-analytics database data-pipeline data-science dbms distributed json open-source parquet postgresql rest s3 snowflake sql

Added 2 years ago

Apache Airflow

https://airflow.apache.org/

airflow apache2-licensed data-pipeline elt etl foss open-source python scheduler workflow

Added 4 years ago