Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

Lance is a modern columnar data format optimized for machine learning and AI applications. It efficiently handles diverse multimodal data types while providing high-performance querying and versioning capabilities.

• Lance @ GitHub.

Related contents:

• Lance takes aim at Parquet in file format joust @ The Register.

Python Data Analysis Library.

pandas is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool. Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more

• pandas @ GitHub.

Related contents:

• Leveraging Pandas and SQL Together for Efficient Data Analysis @ KD nuggets.

🚀 Async-Powered Pandas.

Lightweight Pandas monkey-patch that adds async support to map, apply, applymap, aggregate, and transform, enabling seamless handling of async functions with controlled max_parallel execution.

Conversational Data Analysis.

PandasAI is a Python platform that makes it easy to ask questions to your data in natural language. It helps non-technical users to interact with their data in a more natural way, and it helps technical users to save time, and effort when working with data.

PandasAI is a Python library that integrates generative artificial intelligence capabilities into pandas, making dataframes conversational. Chat with your database (SQL, CSV, pandas, polars, mongodb, noSQL, etc). PandasAI makes data analysis conversational using LLMs (GPT 3.5 / 4, Anthropic, VertexAI) and RAG.

• PandasAI @ GitHub.

Turn your pandas dataframe into a Tableau-style User Interface for visual analysis.

PyGWalker: A Python Library for Exploratory Data Analysis with Visualization