.svg){kind=icon}
The best way of working with Protocol Buffers. Elastic, self-hosted Kafka with Advanced Semantic Intelligence
Guarantee streaming data quality and slash cloud costs 10x with Bufstream, a drop-in replacement for Apache Kafka®.
Bufstream is a Kafka-compatible streaming system which stores records directly in an object storage service like S3.
Self-hosted LiveSync is a community-implemented synchronization plugin, available on every obsidian-compatible platform and using CouchDB or Object Storage (e.g., MinIO, S3, R2, etc.) as the server.
Better Upload is a file upload library for React that makes it very simple and easy to upload files directly to any S3-compatible service. It handles both the client and your server.
While Better Upload is designed to work with Next.js, it can also work with any framework / backend server that uses standard Request and Response objects. Like Remix and Hono!
S3oosh allows users to upload multiple files at once to S3 Buckets. It provides a drag-and-drop interface for users to easily upload files to a S3 Bucket. The component supports various file types and allows users to set maximum file count, maximum file size, and accepted file types.
Command Line S3 Client and S3 Backup for Windows, Linux: s3cmd, s3express.
S3cmd is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage or DreamHost DreamObjects. It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc.
An embedded database built on object storage.
Unlike traditional LSM-tree storage engines, SlateDB writes data to object storage to provide bottomless storage capacity, high durability, and easy replication.
SlateDB is an embedded storage engine built as a log-structured merge-tree. Unlike traditional LSM-tree storage engines, SlateDB writes data to object storage (S3, GCS, ABS, MinIO, Tigris, and so on). Leveraging object storage allows SlateDB to provide bottomless storage capacity, high durability, and easy replication. The trade-off is that object storage has a higher latency and higher API cost than local disk.
A Little Tiny (S3 compatible) file storage server
The free SaaS template with superpowers
A free, open-source SaaS app starter for React & Node.js with superpowers. Production-ready. Community-driven.
Open Source Distributed POSIX File System for Cloud. JuiceFS is a distributed POSIX file system built on top of Redis and S3.
JuiceFS is a high-performance POSIX file system released under Apache License 2.0, particularly designed for the cloud-native environment. The data, stored via JuiceFS, will be persisted in Object Storage (e.g. Amazon S3), and the corresponding metadata can be persisted in various compatible database engines such as Redis, MySQL, and TiKV based on the scenarios and requirements.
With JuiceFS, massive cloud storage can be directly connected to big data, machine learning, artificial intelligence, and various application platforms in production environments. Without modifying code, the massive cloud storage can be used as efficiently as local storage.
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
An open-source distributed storage service you can self-host to fullfill many needs.
Garage is an S3-compatible distributed object storage service designed for self-hosting at a small-to-medium scale.
Garage is designed for storage clusters composed of nodes running at different physical locations, in order to easily provide a storage service that replicates data at these different locations and stays available even when some servers are unreachable. Garage also focuses on being lightweight, easy to operate, and highly resilient to machine failures.
Backup multiple database types on a scheduled basis with many customizable options.
This will build a container for backing up multiple types of DB Servers.
Backs up CouchDB, InfluxDB, MySQL/MariaDB, Microsoft SQL, MongoDB, Postgres, Redis servers.
Your Data Pipeline, Simplified. GlareDB: An analytics DBMS for distributed data.
Data exists everywhere: your laptop, Postgres, Snowflake and as files in S3. It exists in various formats such as Parquet, CSV and JSON. Regardless, there will always be multiple steps spanning several destinations to get the insights you need.
GlareDB is designed to query your data wherever it lives using SQL that you already know.
Easy file sharing from the command line.
Easy and fast file sharing from the command-line. This code contains the server with everything you need to create your own instance. Transfer.sh currently supports the s3 (Amazon S3), gdrive (Google Drive), storj (Storj) providers, and local file system (local).
Parallel S3 and local filesystem execution tool.
s5cmd is a very fast S3 and local filesystem execution tool. It comes with support for a multitude of operations including tab completion and wildcard support for files, which can be very handy for your object storage workflow while working with large number of files.
Fully featured and highly configurable SFTP server with optional HTTP/S, FTP/S and WebDAV support. Several storage backends are supported: local filesystem, encrypted local filesystem, S3 (compatible) Object Storage, Google Cloud Storage, Azure Blob Storage, SFTP.
Duplicati is a free, open source, backup client that securely stores encrypted, incremental, compressed backups on cloud storage services and remote file servers.
simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system.
Mountpoint for Amazon S3 is a simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system. With Mountpoint for Amazon S3, your applications can access objects stored in Amazon S3 through file operations like open and read. Mountpoint for Amazon S3 automatically translates these operations into S3 object API calls, giving your applications access to the elastic storage and throughput of Amazon S3 through a file interface.
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
Kopia is a fast and secure open-source backup/restore tool that allows you to create encrypted snapshots of your data and save the snapshots to remote or cloud storage of your choice, to network-attached storage or server, or locally on your machine. Kopia does not 'image' your whole machine. Rather, Kopia allows you to backup/restore any and all files/directories that you deem are important or critical.
Ceph is a distributed object, block, and file storage platform.
Reliable and scalable storage designed for any organization.
Use Ceph to transform your storage infrastructure. Ceph provides a unified storage service with object, block, and file interfaces from a single cluster built from commodity hardware components.