.svg){kind=icon}
email
Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
Related contents:
A unified metadata lake across all your sources, formats, cloud providers, and regions in a federated architecture.
World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
Apache Gravitino is a high-performance, geo-distributed, and federated metadata lake. It manages metadata directly in different sources, types, and regions, providing users with unified metadata access for data and AI assets.
Insights, Unlocked in Real Time.
Apache Pinot
: The real-time analytics open source platform for lightning-fast insights, effortless scaling, and cost-effective data-driven decisions.
Related contents:
The open table format for analytic datasets.
Iceberg is a high-performance format for huge analytic tables. Iceberg brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work with the same tables, at the same time.
The best way of working with Protocol Buffers. Elastic, self-hosted Kafka with Advanced Semantic Intelligence
Guarantee streaming data quality and slash cloud costs 10x with Bufstream, a drop-in replacement for Apache Kafka
.
Bufstream is a Kafka-compatible streaming system which stores records directly in an object storage service like S3.
Seamless multi-master sync, that scales from Big Data to Mobile, with an Intuitive HTTP/JSON API and designed for Reliability.
CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents with your web browser, via HTTP. Query, combine, and transform your documents with JavaScript. CouchDB works well with modern web and mobile apps. You can distribute your data, efficiently using CouchDB’s incremental replication. CouchDB supports master-master setups with automatic conflict detection.
Open source analytics infrastructure. Fast and scalable. No bloat. GDPR compliant.
A single production-ready Docker image built on ClickHouse, Kafka, and Node.js for tracking events, users, page views, and interactions.
The Apache Hadoop project develops open-source software for reliable, scalable, distributed computing.
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.
An Open Source Data Lake Platform.
Apache Hudi is a transactional data lake platform that brings database and data warehouse capabilities to the data lake. Hudi reimagines slow old-school batch data processing with a powerful new incremental processing framework for low latency minute-level analytics.
Les technologies numériques sont incroyablement puissantes et redéfinissent le fonctionnement de notre société. Pour les acteurs qui œuvrent pour l'intérêt général, la technologie peut parfois être un levier démutiplicateur d'impacts positifs, cependant et malheureusement ces acteurs n'ont souvent pas les ressources technologiques ou humaines pour accélérer leur action citoyenne. Data for Good existe pour rétablir l'équilibre.
Interactive SQL. Analyze petabyte-scale data where it lives with ease and flexibility.
Amazon Athena is a serverless, interactive analytics service built on open-source frameworks, supporting open-table and file formats. Athena provides a simplified, flexible way to analyze petabytes of data where it lives. Analyze data or build applications from an Amazon Simple Storage Service (S3) data lake and 30 data sources, including on-premises data sources or other cloud systems using SQL or Python. Athena is built on open-source Trino and Presto engines and Apache Spark frameworks, with no provisioning or configuration effort required.
DataFusion is a very fast, extensible query engine for building high-quality data-centric systems in Rust, using the Apache Arrow in-memory format.
DataFusion is great for building projects such as domain specific query engines, new database platforms and data pipelines, query languages and more. It lets you start quickly from a fully working engine, and then customize those features specific to your use.
Open Source Distributed POSIX File System for Cloud. JuiceFS is a distributed POSIX file system built on top of Redis and S3.
JuiceFS is a high-performance POSIX file system released under Apache License 2.0, particularly designed for the cloud-native environment. The data, stored via JuiceFS, will be persisted in Object Storage (e.g. Amazon S3), and the corresponding metadata can be persisted in various compatible database engines such as Redis, MySQL, and TiKV based on the scenarios and requirements.
With JuiceFS, massive cloud storage can be directly connected to big data, machine learning, artificial intelligence, and various application platforms in production environments. Without modifying code, the massive cloud storage can be used as efficiently as local storage.
open source big data platform.
Trunk Data Platform is an Open Source, free, Hadoop distribution.
Druid is a high performance, real-time analytics database that delivers sub-second queries on streaming and batch data at scale and under load.
XetHub brings speedy access and Git-based collaboration to large scale repositories of data, code, or any combination of files.
Our instant mount feature makes it possible to access GBs and TBs of data in seconds at the speed of localhost, while our de-duplication algorithm stores data and differences efficiently to save money and speed up development cycles.
XetHub is ideal for teams who already use Git to track their code changes, and want to leverage the power of infinite history, pull requests, and difference-based tracking for larger assets such as datasets or media files. Managing complete projects with familiar Git semantics makes change tracking and continuous integration a breeze, especially for workflows that use code to generate or augment assets.
Graph Database Management System.
Neo4j Graph Data Platform. Blazing-Fast Graph, Petabyte Scale.
With proven trillion+ entity performance, developers, data scientists, and enterprises rely on Neo4j as the top choice for high-performance, scalable analytics, intelligent app development, and advanced AI/ML pipelines.
ClickHouse is a free analytics DBMS for big data.
ClickHouse is an open-source column-oriented database management system that allows generating analytical data reports in real-time.
Open big JSON, CSV Files: Online Viewer, Explorer and Converter.
View and convert big data files.
View large or small files right in your browser and export them in any format.
Daily Earth Data to See Change and Make Better Decisions.
Planet provides daily satellite data that helps businesses, governments, researchers, and journalists understand the physical world and take action.