Migrate Databases and Stream Data Between Database Systems.

DBConvert Streams (DBS) is a cutting-edge distributed platform designed for data migration between heterogeneous databases and real-time data replication. It simplifies the process of transferring data between on-premises or cloud databases, including relational databases and data warehouses.

• DBConvert Streams @ GitHub.

Related contents:

• Change Data Capture Tools @ Dev Genius' Medium.

Open-source vector similarity search for Postgres.

Related contents:

• Sharding pgvector @ PgDog.

A ledger implementation in PostgreSQL.

Related contents:

• Ledger Implementation in PostgreSQL @ Paul Gross's Blog.

An embeddable, scalable, extremely fast graph database.

Embeddable property graph database management system built for query speed and scalability. Implements Cypher. Kuzu is an embedded graph database built for query speed and scalability. Kuzu is optimized for handling complex analytical workloads on very large databases and provides a set of retrieval features, such as a full text search and vector indices. Our core feature set includes:

• Kuzu @ GitHub.

Related contents:

• Fully In-Browser Graph RAG with Kuzu-Wasm @ Kuzu.

1000x Faster Analytics in Postgres. Postgres-native Data Warehouse.

pg_mooncake is a Postgres extension that adds columnar storage and vectorized execution (DuckDB) for fast analytics within Postgres. Postgres + pg_mooncake ranks among the top 10 fastest in ClickBench.

• pg_mooncake @ GitHub.

Postgre SQL Operator for Kubernetes. Run PostgreSQL. The Kubernetes way.

CloudNativePG is the Kubernetes operator that covers the full lifecycle of a highly available PostgreSQL database cluster with a primary/standby architecture, using native streaming replication.

• CloudNativePG @ GitHub.
• 0️⃣ CNPG-I Scale-to-Zero Plugin @ GitHub.

Related contents:

• Plongez dans le monde de CloudNativePG #4 - Les sauvegardes ! @ Dalibo :fr:.
• Making Postgres scale to zero with CNPG @ xata.
• Kubernetes Finally Solves Its Biggest Problem: Managing Databases @ The New Stack.

Lightweight NoDB for Deno & React.

GoatDB is a real-time, version-controlled database for Deno, React, and low-friction deployments. It’s ideal for prototyping, self-hosting, single-tenant apps, as well as ultra light multi-tenant setups without heavy backends or complex DBs.

• GoatDB @ GitHub.

Distributed SQLite.

LiteFS is a distributed file system that transparently replicates SQLite databases. You can run your application like it’s running against a local on-disk SQLite database but behind the scenes the database is replicated to all the nodes in your cluster. With LiteFS, you can run your database right next to your application on the edge. You can run LiteFS anywhere!

• LiteFS @ GitHub.

SQL Indexing and Tuning e-Book for developers. Use The Index, Luke covers Oracle, MySQL, PostgreSQL, SQL Server, ...

A site explaining SQL indexing to developers—no crap about administration. SQL indexing is the most effective tuning method—yet it is often neglected during development. Use The Index, Luke explains SQL indexing from grounds up and doesn’t stop at ORM tools like Hibernate.

A database backup management and automation tool. Velld provides a user-friendly interface for scheduling, managing, and monitoring database backups, ensuring data security and easy recovery.

A PostgreSQL assistant for developers designed to help understand and optimize PostgreSQL database performance.

PgAssistant is an open-source tool designed to help developers understand and optimize their PostgreSQL database performance. It provides insights into database behavior, identifies schema-related issues, and assists in correcting them. Additionally, PgAssistant makes it easy to manage a library of specific SQL queries useful for your project through a simple JSON file.

Parseable is a disk less, cloud native database for logs, observability, security, and compliance. Parseable is built with focus on simplicity & resource efficiency.

• Parseable @ GitHub.

All-in-One Database Version Management Tool.

Related contents:

• Local WordPress with Herd, DBngin, and WP-CLI @ Brian Coords.

A Minimalistic Wrapper for IndexedDB.

Dexie.js is a wrapper library for indexedDB - the standard database in the browser. IndexedDB is the portable database for all browser engines. Dexie.js makes it fun and easy to work with.

• Dexie.js @ GitHub.

Visual tool to explore SQLite databases page-by-page, the way they're stored on disk and the way SQLite sees them. A small GUI application built in redbean that lets you explore your SQLite databases "page by page" the way SQLite sees them.

Database Change Management & CI/CD Automation, Database DevOps.

Automated, secure, and compliant database change management pipelines that accelerate delivery and reduce toil. Liquibase helps millions of developers track, version, and deploy database schema changes.

• Liquibase @ GitHub.

A powerful, AI-native Python client library for Redis. Leverage the speed, flexibility, and reliability of Redis for real-time data to supercharge your AI application.

• RedisVL @ GitHub.

Related contents:

• You need more than a vector database @ Redis.

DocumentDB offers a native implementation of document-oriented NoSQL database, enabling seamless CRUD operations on BSON data types within a PostgreSQL framework. Powering vCore-based Azure Cosmos DB for MongoDB.

Related contents:

• Microsoft builds open source document database on PostgreSQL, suggests FerretDB as front end @ The Register.
• DocumentDB: Open-Source Announcement @ Microsoft Open Source.

Visualizing and understanding PostgreSQL EXPLAIN plans made easy.

A VueJS component to show a graphical vizualization of a PostgreSQL execution plan.

• PostgreSQL execution plan visualizer 2 @ GitHub.

review Postgres query plans quickly.

A simple yet powerful tool to help you speed up queries. pgMustard speeds up your journey from knowing which query is a problem to working out what can be done about it.

Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. You can use Amazon DynamoDB to create a database table that can store and retrieve any amount of data, and serve any level of request traffic. Amazon DynamoDB automatically spreads the data and traffic for the table over a sufficient number of servers to handle the request capacity specified by the customer and the amount of data stored, while maintaining consistent and fast performance.

Related contents:

• DynamoDB Dashboard @ GitHub.
• dynamodb-admin @ GitHub.
• #304.bin - Bilan 2024: Le début de la révolution avec Quentin Adam @ <ifttd>.

Query SQL database through an array-like API.

Kronotop is a Redis-compatible, distributed and transactional document database backed by FoundationDB.

Kronotop's main focus is building a document database that supports MQL-like query language, ACID transactions and on-disk storage engine with a primary-standby replication model.

A native MacOS Postgres client.

Searchlight is a native macOS, open-source PostgreSQL client designed to be simple, lightweight, and developer-friendly. It offers a clean and intuitive interface for managing databases and running queries without the complexity of full-scale database management tools. Searchlight focuses on delivering a smooth and efficient experience for developers who need quick and easy access to their PostgreSQL databases.

The lightweight, user-friendly, distributed relational database built on SQLite. Fault tolerance and high availability with zero hassle.

• rqlite @ GitHub.

Related contents:

• How rqlite is tested @ Vallified.

Open Source Metrics Engine. Distributed TSDB and Query Engine, Prometheus Sidecar, Metrics Aggregator, and more such as Graphite storage and query engine.

M3 is a Prometheus compatible, easy to adopt metrics engine that provides visibility for some of the world’s largest brands.

• M3 @ GitHub.

Chproxy is an HTTP proxy and load balancer for ClickHouse.

• Chproxy @ GitHub.

Open Source Airtable Alternative & Backend as a Service. 🚀 Private first self-hosted no code database & BaaS.

UNDB is a no-code platform that can also serve as a Backend as a Service (BaaS). It is based on SQLite and can be packaged into a binary file using Bun for backend service. Additionally, it can be deployed as a service via Docker, offering a UI for table management.

• Undb @ GitHub.

Data Runs Better on SDF. Transform Data Better with SDF. SDF is the fastest way to build a scalable, reliable, and optimized data warehouse.

SDF is a developer platform for data that scales SQL understanding across an organization, empowering all data teams to unlock the full potential of their data.

SDF is a multi-dialect SQL compiler, transformation framework, and analytical database engine. It natively compiles SQL dialects, like Snowflake, and connects to their corresponding data warehouses to materialize models.

• SDF Docs.
• SDF @ GitHub.

Source: Testing is Not Enough: Transforming Data Quality with Write, Audit, Publish using SDF Build @ SDF Blog.

✨A CLI tool for analyzing queries and DB tables for MySQL.

The Database Toolkit for Python.

SQLAlchemy is the Python SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL.

It provides a full suite of well known enterprise-level persistence patterns, designed for efficient and high-performing database access, adapted into a simple and Pythonic domain language.

• SQLAlchemy @ GitHub.
• SqlAlchemy, l'ORM Python - Partie 1 @ DevSecOps :fr:.

Distributed SQL Databases

Fastest serverless distributed SQL database for always available applications.

• AWS announces Aurora DSQL, a new distributed SQL database that promises virtually unlimited scalability @ TechCrunch.

Self-hosted street address database

A SQLite3 database file with over 150 million U.S. and Canada address records. Indexed for fast queries, even on fairly slow hardware.

MySQL Performance Tuning Tool.

• Founder & Developer Interview With Roman Agabekov @ OpenLampTech.

Open-source sQL AI Agent. Text2SQL made Easy!

Wren AI is an open-source SQL AI Agent that empowers data, product, and business teams to access insights through AI chat, built-in well designed intuitive UI and UX, integrating seamlessly with tools like Excel and Google Sheets.

• Wren AI @ GitHub.

Build Reliable Backends 10x Faster, Scale to Millions with 1 Click.

DBOS is a serverless platform for building highly reliable applications. What takes days to build on AWS takes minutes on DBOS.

• DBOS Transact in Python @ GitHub.
• DBOS Transact in Go @ GitHub.

Related contents:

• Running Durable Workflows in Postgres using DBOS @ Supabase.

Limbo is a work-in-progress, in-process OLTP database management system, compatible with SQLite.

Related contents:

• Episode #626: Turso is rewriting SQLite in Rust @ Changelog Interviews.
• Introducing Limbo: A complete rewrite of SQLite in Rust @ Turso.

How to reduce complexity and move faster? Just Postgres for everything.

The goal of this repository is to demonstrate how to use Postgres for Everything.

• Just Use Postgres for Everything @ Amazing CTO.

Type safe queries using the Rust type system.

The goal of this library is to allow writing relational database queries using familiar Rust syntax. The library should guarantee that a query can not fail if it compiles. This already includes preventing use after free for row ids passed between queries and even database migrations!

The open table format for analytic datasets.

Iceberg is a high-performance format for huge analytic tables. Iceberg brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work with the same tables, at the same time.

• Apache Iceberg @ GitHub.

Related contents:

• PyIceberg: Current State and Roadmap @ Ju Data Engineering Newsletter.
• The Equality Delete Problem in Apache Iceberg @ Data Engineer Things's Medium.
• How I Saved Millions by Restructuring Iceberg Metadata @ Gautham Gondi's Medium.
• High Throughput Ingestion with Iceberg @ Adobe Tech Blog's Medium.
• Scaling Iceberg Writes with Confidence: A Conflict-Free Distributed Architecture for Fast, Concurrent, Consistent Append-Only Writes @ e6data.

Powerful Database Client.

Outerbase Studio is a fully-featured, lightweight GUI client for managing Turso, LibSQL, Cloudflare D1, rqlite, MySQL, and PostgreSQL. It runs entirely in your browser, so there's no need to download anything.

• Outerbase Studio @ GitHub.

The world's most advanced open source database.

Related contents:

• What I Wish Someone Told Me About Postgres @ ChallahScript.
• Logical replication in Postgres: Basics @ EDB.
• Database sharding case study: PostgreSQL @ The Polymathic Engineer.
• I Tried Every Open Source Database and Found Only One That is Actually Useful @ Hacker Noon.
• How direct TLS can speed up your connections @ Marc Bowes.
• PostgreSQL pour remplacer Redis @ Rodolphe Bréard :fr:.
• One more reason to choose Postgres over MySQL @ tanin.
• From 99.9% to 99.99%: Building PostgreSQL Resilience into Your Product Architecture @ Stormatics.
• Handling unique indexes on large data in PostgreSQL @ Volodymyr Potiichuk.
• Postgres Internals Deep Dive: Process Architecture @ EDB Postgres.
• PostgreSQL maintenance without superuser @ boringSQL.
• Postgres’ Original Project Goals: The Creators Totally Nailed It @ Chunchy Data.
• Redis is fast - I'll cache in Postgres @ Dizzy zone.
• PostgreSQL MVCC Explained (Simply!) @ SauravDhakal's Substack.
• Postgres Partitioning Best Practices: Sofia's Story @ Karen Jex.
• Tuning AIO in PostgreSQL 18 @ Tomas Vondra.
• Advanced PostgreSQL Indexing: Multi-Key Queries and Performance Optimization @ Frontend Masters.
• The best worst hack that saved our bacon @ Jefferson Heard.
• Going down the rabbit hole of Postgres 18 features @ xata.
• JIT: so you want to be faster than an interpreter on modern CPUs… @ Pinaraf's website.
• Postgres 18: OLD and NEW Rows in the RETURNING Clause @ crunchy data.
• Benchmarking Postgres 17 vs 18 @ PlanetScale.
• Is Postgres Read Heavy or Write Heavy? (And Why You Should You Care) @ crunchy data.
• Pipelining in psql (PostgreSQL 18) @ PostgreSQL Notes - Daniel Vérité.

NoSQL Database. Life is an Adventure — Your Database Shouldn't Be.

ACID Document Database.

• RavenDB @ GitHub.

Related contents:

• Optimizing by 170,000%(!) by not being silly @ Oren Eini.

The Fastest In-Memory Data Store. A modern replacement for Redis and Memcached.

Dragonfly is a drop-in Redis replacement that cuts costs and boosts performance. Designed to fully utilize the power of modern cloud hardware and deliver on the data demands of modern applications, Dragonfly frees developers from the limits of traditional in-memory data stores.

• Dragonfly @ GitHub.

The power of programming, in your database.

TypeDB is a next-gen database with a modern programming paradigm that lets you build data applications faster, safer, and more elegantly. Its intuitive and powerful data model unifies the strengths of relational, document and graph databases without their shortcomings. TypeQL, its groundbreaking query language, is declarative, functional, and strongly-typed, drastically simplifying data handling and logic. So now, even the most nested and interconnected datasets can be managed with ease. With TypeDB, we’ve reinvented the database for the modern programming era.

• TypeDB @ GitHub.

An Object-Document Mapper (ODM) for Doctrine ORM leveraging new JSON types of modern RDBMS.

Debezium is an open source distributed platform for change data capture. Start it up, point it at your databases, and your apps can start responding to all of the inserts, updates, and deletes that other apps commit to your databases. Debezium is durable and fast, so your apps can respond quickly and never miss an event, even when things go wrong.

• Debezium @ GitHub.

Related contents:

• Code first CDC from Postgres to ClickHouse with Debezium, Redpanda, and MooseStack @ Fileonefour (Code-First CDC to ClickHouse with Debezium, Redpanda, and MooseStack @ GitHub).
• Change Data Capture Tools @ Dev Genius' Medium.
• The Equality Delete Problem in Apache Iceberg @ Data Engineer Things' Medium.

Agile MySQL with S3-Based Storage. Improving the popular MySQL to be more developer-friendly and easier to manage, streamlining your app lifecycle.

WeSQL is an innovative MySQL distribution that adopts a compute-storage separation architecture, with storage backed by S3 (and S3-compatible systems). It can run on any cloud, ensuring no vendor lock-in.

• WeSQL @ GitHub.

Create, edit, browse SQLite databases.

• SQLiteStudio @ GitHub.

⚡ Lightweight Firebase alternative built to run anywhere.

Experience the power of a lightweight, feature-rich backend that seamlessly integrates into your framework of choice. Start building smarter today.

bknd simplifies app development by providing fully functional backend for data management, authentication, workflows and media. Since it's lightweight and built on Web Standards, it can be deployed nearly anywhere, including running inside your framework of choice. No more deploying multiple separate services!

• bknd @ GitHub.

PoWA is a PostgreSQL Workload Analyzer that gathers performance stats and provides real-time charts and graphs to help monitor and tune your PostgreSQL servers.

• PoWA @ GitHub.
• Linux Pratique - Supervision d'une instance PostgreSQL @ Dalibo :fr:.

pg_activity is a top like application for PostgreSQL server activity monitoring.

• pg_activity @ GitHub.
• Linux Pratique - Supervision d'une instance PostgreSQL @ Dalibo :fr:.

PostgreSQL Remote Control.

temBoard is a powerful management tool for PostgreSQL. It allows to observe, optimize, or configure PostgreSQL instances.

• temBoard @ GitHub.
• Linux Pratique - Supervision d'une instance PostgreSQL @ Dalibo :fr:.

PostgreSQL log analyzer.

pgBadger is a PostgreSQL log analyzer built for speed with fully detailed reports and professional rendering.

• pgBadger @ GitHub.
• Linux Pratique - Supervision d'une instance PostgreSQL @ Dalibo :fr:.
• Analyse des traces de PostgreSQL avec pgBadger @ Diamond Connect :fr:.

A Go test harness that starts an ephemeral PostgreSQL server.

Package postgrestest provides a test harness that starts an ephemeral PostgreSQL server. It is tested on macOS, Linux, and Windows. It can cut down the overhead of PostgreSQL in tests up to 90% compared to spinning up a postgres Docker container: starting a server with this package takes roughly 650 milliseconds and creating a database takes roughly 20 milliseconds.

• Testing with Go and PostgreSQL: ephemeral DBs @ Michael Stapelberg.

cloudless database for realtime apps.

Realtime database, runs anywhere. Install Fireproof in your front-end app or edge function, and sync data via any backend.

Add collaboration to any app with Fireproof. Access data from JavaScript servers and edge functions. Use live queries to update your UI automatically when the ledger changes. Connect realtime sync and those changes will sync between browsers and backend functions. Apps built this way are multi-player by default.

• Fireproof @ GitHub.

Source: Fireproof - une base de données moderne pour les applications Web @ Korben :fr:.

the AI-native open-source embedding database. The fastest way to build Python or JavaScript LLM apps with memory! Chroma is the open-source AI application database. Batteries included.

Embeddings, vector search, document storage, full-text search, metadata filtering, and multi-modal. All in one place. Retrieval that just works. As it should be.

• Chroma @ GitHub.

Related contents:

• ChromaDB: An Open-source vector embedding database @ Futuresmart AI Blog.

PostgreSQL zero-downtime migrations made easy.

pgroll is an open source command-line tool that offers safe and reversible schema migrations for PostgreSQL by serving multiple schema versions simultaneously. It takes care of the complex migration operations to ensure that client applications continue working while the database schema is being updated. This includes ensuring changes are applied without locking the database, and that both old and new schema versions work simultaneously (even when breaking changes are being made!). This removes risks related to schema migrations, and greatly simplifies client application rollout, also allowing for instant rollbacks.

libSQL is a fork of SQLite that is both Open Source, and Open Contributions.

We aim to evolve it to suit many more use cases than SQLite was originally designed for, and plan to use third-party OSS code wherever it makes sense.

AgensGraph is a cutting-edge multi-model graph database designed for modern complex data environments. By supporting both relational and graph data models simultaneously, AgensGraph allows developers to seamlessly integrate legacy relational data with the flexible graph data model within a single database. AgensGraph is built on the robust PostgreSQL RDBMS, providing a highly reliable, fully-featured platform ready for enterprise use.