OpenZL delivers high compression ratios while preserving high speed, a level of performance that is out of reach for generic compressors.

OpenZL takes a description of your data and builds from it a specialized compressor optimized for your specific format. OpenZL consists of a core library and tools to generate specialized compressors — all compatible with a single universal decompressor. It is designed for engineers that deal with large quantities of specialized datasets (like AI workloads for example) and require high speed for their processing pipelines.

• OpenZL @ GitHub.

Related contents:

• Introducing OpenZL: An Open Source Format-Aware Compression Framework @ Engineering at Meta.

vTunnel is a tool that proxies IP traffic between guest and host networks by using the VSOCK protocol.

This project provides the ability to tunnel IP traffic through the hypervisor so that connections can be proxied into or out of virtual machines.

Related contents:

• Episode 634: Config Confessions @ Linux Unplugged.

As data volumes continue to grow in fields like machine learning and scientific computing, optimizing fundamental operations like matrix multiplication becomes increasingly critical. Blosc2's chunk-based approach offers a new path to efficiency in these scenarios.

Blosc is a high performance compressor optimized for binary data (i.e. floating point numbers, integers and booleans, although it can handle string data too). It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Blosc main goal is not just to reduce the size of large datasets on-disk or in-memory, but also to accelerate memory-bound computations.

• C-Blosc2 @ GitHub.

Related contents:

• Compress Better, Compute Bigger @ ironArray.