processing
The CSV magician.
xan is a command line tool that can be used to process CSV files directly
from the shell.
It can easily preview, filter, slice, aggregate, sort, join CSV files,
and exposes a large collection of composable commands that can be chained
together to perform a wide variety of typical tasks.
Related contents:
Fast, simple, scalable, Docker-ready HTTP microservice for high-level image processing.
Fast HTTP microservice written in Go for high-level image processing backed by bimg and libvips. imaginary can be used as private or public HTTP service for massive image processing with first-class support for Docker & Fly.io. It's almost dependency-free and only uses net/http native package without additional abstractions for better performance.
Pure Java cross-platform image processing framework that provides features for image and video frame processing, multi-threading image processing, GUI integration, extensibility via plug-ins, unit text automation among other things.
Stateful Computations over Data Streams.
Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments, perform computations at in-memory speed and at any scale.
Related contents:
- Flink AI: Hands-On FEDERATED_SEARCH()—Search a Vector Database with Confluent Cloud for Apache Flink® @ Confluent.
- You Gotta Push If You Wanna Pull @ Gunnar Morling.
- Why We Ditched Flink Table API Joins: Cutting State by 75% with DataStream Unions @ Zalando Engineering Blog.
- The KFC Architecture Blueprint: Kafka, Flink, and ClickHouse @ Big Data Boutique.
- S3 is the perfect place to store data, until you try to search it @ Gordon Murray.
- From 40 seconds to under 10: rebuilding incident detection on OpenTelemetry, Apache Kafka, and Apache Flink on Kubernetes @ CNCF.
Unified stream and batch data processing that's serverless, fast, and cost-effective.
The Unified Apache Beam Model. The easiest way to do batch and streaming data processing. Write once, run anywhere data processing for mission-critical production workloads.
Apache Beam is a unified programming model for Batch and Streaming data processing. Apache Beam is a unified model for defining both batch and streaming data-parallel processing pipelines, as well as a set of language-specific SDKs for constructing pipelines and Runners for executing them on distributed processing backends, including Apache Flink, Apache Spark, Google Cloud Dataflow, and Hazelcast Jet.