job-queue
Kubernetes-native Job Queueing.
Kueue is a set of APIs and controller for job queueing. It is a job-level manager that decides when a job should be admitted to start (as in pods can be created) and when it should stop (as in active pods should be deleted).
Related contents:
⚡ High-performance job queue for Bun. SQLite persistence, DLQ, cron jobs, S3 backups.
Sophisticated job processing framework. Process Billions of Jobs Without Losing Sleep.
Thousands of Elixir and Python applications rely on Oban Pro for background jobs with advanced workflows, distributed concurrency, highly optimized performance, and a trove of enterprise grade features.
Related contents:
Database-backed Active Job backend.
Solid Queue is a DB-based queuing backend for Active Job, designed with simplicity and performance in mind.
Besides regular job enqueuing and processing, Solid Queue supports delayed jobs, concurrency controls, recurring jobs, pausing queues, numeric priorities per job, priorities by queue order, and bulk enqueuing (enqueue_all for Active Job's perform_all_later).
CLI & Library for Sequential, Distributed, POSIX-style job queue processing
Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler. Filespooler is particularly suited to distributed and Asynchronous Communication.
Distributed Task Queue
Celery is a simple, flexible, and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system.
The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).