A flexible, configuration driven CLI for Apache Airflow local development.

Airlift is a Command Line Interface (CLI) tool designed to provide a local development environment for Apache Airflow with a simple but flexible interface. It is built on top of the official Airflow Helm Chart.

An SDK for working with LLMs and AI Agents from Apache Airflow, based on Pydantic AI.

It allows users to call LLMs and orchestrate agent calls directly within their Airflow pipelines using decorator-based tasks. The SDK leverages the familiar Airflow @task syntax with extensions like @task.llm, @task.llm_branch, and @task.agent.

Workflow Orchestration Made Simple. Stop wondering about your workflows. The orchestrator for people who have done orchestration before.

Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines. Prefect is an orchestration and observability platform for building, observing, and triaging workflows. It's the simplest way to transform Python code into an interactive workflow application.

Prefect allows you to expose your workflows through an API so teams dependent on you can programmatically access your pipelines, business logic, and more. Prefect also allows you to standardize workflow development and deployment across your organization.

• Prefect @ GitHub.

Related contents:

• An Open-Source Maintainer's Guide to Saying No @ Mostly Harmless.

Airflow is a platform created by the community to programmatically author, schedule and monitor workflows.

• Apache Airflow @ GitHub.

Related contents:

• Apache Airflow Configuration and Tuning @ DZone.
• Exploring Apache Airflow for Batch Processing Scenario @ DZone.
• What Is Apache Airflow @ Seattle Data Guy.
• Alternatives to Talend – How To Migrate Away From Talend For Your Data Pipelines @ Seattle Data Guy.