Semantic DataFrames for humans and agents.

fenic turns AI-assisted exploration of structured and unstructured data into reusable, inspectable DataFrame pipelines.

It's a DataFrame query engine for semantic data processing, with AI operators — extract, classify, summarize, embed, semantic join, and more — built into the query model. Use it to turn documents, transcripts, logs, eval traces, tickets, tables, and APIs into typed rows and repeatable workflows.

• Fenic @ GitHub.

DataFrames for the new era.

Dataframes powered by a multithreaded, vectorized query engine, written in Rust

Polars is a DataFrame interface on top of an OLAP Query Engine implemented in Rust using Apache Arrow Columnar Format as the memory model.

Polars is an open-source library for data manipulation, known for being one of the fastest data processing solutions on a single machine. It features a well-structured, typed API that is both expressive and easy to use.

• Polars @ GitHub.

Related contents:

• Polars at Decathlon: Ready to Play? @ Decathlon Digital.
• Decathlon Switches to Polars to Optimize Data Pipelines and Infrastructure Costs @ InfoQ.
• DuckDB beats Polars for 1TB of data @ Confessions of a Data Guy.

Real-Time Event Streaming Platform. Streaming CDC, stream processing, low-latency serving, and Iceberg management.

RisingWave is a real-time event streaming platform designed to offer the simplest and most cost-effective way to process, analyze, and manage real-time event data — with built-in support for the Apache Iceberg™ open table format. It provides both a Postgres-compatible SQL interface and a DataFrame-style Python interface.

RisingWave can ingest millions of events per second, continuously join and analyze live streams with historical data, serve ad-hoc queries at low latency, and persist fresh, consistent results to Apache Iceberg™ or any other downstream system.

• RisingWave @ GitHub.

Related contents:

• The Case for Hybrid Cache for Object Stores @ Data Engineer Things' Medium.
• You Gotta Push If You Wanna Pull @ Gunnar Morling.

ArcticDB is a DataFrame Database.

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Built for the modern Python Data Science ecosystem, ArcticDB transforms your ability to handle complex real world data with Incredibly fast proven Petabyte scale.

• ArticDB @ GitHub.
• La veille des Ours n°32 @ Bearstech's LinkedIn :fr:.

DataFusion is a very fast, extensible query engine for building high-quality data-centric systems in Rust, using the Apache Arrow in-memory format.

DataFusion is great for building projects such as domain specific query engines, new database platforms and data pipelines, query languages and more. It lets you start quickly from a fully working engine, and then customize those features specific to your use.

• DataFusion @ GitHub.