.svg){kind=icon}
emailStateful Computations over Data Streams.
Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments, perform computations at in-memory speed and at any scale.
Related contents:
@ Confluent
A Datacenter Scale Distributed Inference Serving Framework.
NVIDIA Dynamo is a high-throughput low-latency inference framework designed for serving generative AI and reasoning models in multi-node distributed environments. Dynamo is designed to be inference engine agnostic (supports TRT-LLM, vLLM, SGLang or others) and captures LLM-specific capabilities.
Related contents:
A Friendly Federated AI Framework.
A unified approach to federated learning, analytics, and evaluation. Federate any workload, any ML framework, and any programming language.
Postgre SQL Operator for Kubernetes.
Run PostgreSQL.
The Kubernetes way.
CloudNativePG is the Kubernetes operator that covers the full lifecycle of a highly available PostgreSQL database cluster with a primary/standby architecture, using native streaming replication.
Related contents:
A Reliable Stream Storage System. Streaming as a new software defined storage primitive.
Pravega is an open source distributed storage service implementing Streams. It offers Stream as the main primitive for the foundation of reliable storage systems: a high-performance, durable, elastic, and unlimited append-only byte stream with strict ordering and consistency.
Connect home devices into a powerful cluster to accelerate LLM inference. More devices mean faster performance, leveraging tensor parallelism and high-speed synchronization over Ethernet.
Supports Linux, macOS, and Windows. Optimized for ARM and x86_64 AVX2 CPUs.
Related contents:
Distributed SQLite.
LiteFS is a distributed file system that transparently replicates SQLite databases. You can run your application like it’s running against a local on-disk SQLite database but behind the scenes the database is replicated to all the nodes in your cluster. With LiteFS, you can run your database right next to your application on the edge. You can run LiteFS anywhere!
A distributed tracing system.
Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in service architectures. Features include both the collection and lookup of this data.
If you have a trace ID in a log file, you can jump directly to it. Otherwise, you can query based on attributes such as service, operation name, tags and duration. Some interesting data will be summarized for you, such as the percentage of time spent in a service, and whether or not operations failed.
A Cloud Native Distributed Storage System.
CubeFS is a new generation cloud-native open source storage system that supports access protocols such as S3, HDFS, and POSIX. It is widely applicable in various scenarios such as big data, AI/LLMs, container platforms, separation of storage and computing for databases and middleware, data sharing and protection，etc.
Related contents:
Raft is a consensus algorithm that is designed to be easy to understand. It's equivalent to Paxos in fault-tolerance and performance. The difference is that it's decomposed into relatively independent subproblems, and it cleanly addresses all major pieces needed for practical systems. We hope Raft will make consensus available to a wider audience, and that this wider audience will be able to develop a variety of higher quality consensus-based systems than are available today.
Related contents:
Tahoe-LAFS is a Free and Open decentralized cloud storage system. It distributes your data across multiple servers. Even if some of the servers fail or are taken over by an attacker, the entire file store continues to function correctly, preserving your privacy and security.
Distributed SQL Databases
Fastest serverless distributed SQL database for always available applications.
Program against your datacenter like it’s a single pool of resources.
Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, Jenkins, Spark, Aurora, and other frameworks on a dynamically shared pool of nodes.
Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.
A Coherent Software Configuration Management System
Freenet is a distributed, decentralized alternative to the centralized World Wide Web, designed to unleash a new era of innovation and competition, while protecting freedom of speech and privacy.
The heart of Freenet is the Core, which runs on users' computer, smartphone, or other devices. The Core is tiny, less than 5 MB, allowing it to be installed in a matter of seconds and is compatible with a wide range of hardware.
RemoteLocal Environments to build distributed applications.
Development environment as a service. Building distributed applications isn’t complex anymore!
With Kloudlite’s unified remote local environments, integrate the comfort of local coding with the power of remote environments
Kloudlite is an open-source platform designed to provide seamless and secure development environments for building distributed applications. It connects local workspaces with remote Kubernetes environments via a WireGuard network, allowing developers to access services and resources with production-level parity. With Kloudlite, there’s no need for build or deploy steps during development— With service intercepts, your changes are reflected in real time, enhancing productivity and reducing the development loop.
SaunaFS is a distributed file system.
Welcome to SaunaFS, a robust distributed POSIX file system meticulously designed to revolutionize your storage solutions by offering unmatched efficiency, security, and redundancy. At its core, SaunaFS is a distributed file system primarily written in C++, inspired by the pioneering concepts introduced by Google File System.
an open source Distributed SQL Database.
YDB is a versatile open source Distributed SQL Database that combines high availability and scalability with strong consistency and ACID transactions. It accommodates transactional (OLTP), analytical (OLAP), and streaming workloads simultaneously.
Scalable. Reliable. MySQL-compatible. Cloud-native. Database.
Vitess is a database clustering system for horizontal scaling of MySQL.
Distributed Async Await. A dead simple programming model for modern applications.
Resonate's Distributed Async Await is a new programming model that simplifies coding for the cloud. It ensures code completion even if hardware or software failures occur during execution. The programming model does this with just functions and promises, making it trivial to build coordinated and reliable distributed applications.