Run LLMs at home, BitTorrent-style.

You load a part of the model, then join a network of people serving its other parts. Single‑batch inference runs at up to 6 tokens/sec for Llama 2 (70B) and up to 4 tokens/sec for Falcon (180B) — enough for chatbots and interactive apps.

• Petals @ GitHub.

Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.

Mesh LLM pools GPUs and memory across machines and exposes the result as one OpenAI-compatible API at http://localhost:9337/v1. Start one node, add more nodes later, and let the mesh decide whether a model runs locally, routes to a peer, or uses Skippy stage splits for models that are too large for one box.

• Mesh LLM @ GitHub.

GPU worker client for the Talos network. Pairs with your Talos account, serves open-model inference jobs over a WebSocket, and reports uptime for payouts.

• Talos worker @ GitHub.

Pipeline-parallel LLM inference across GPUs on separate machines.

Pipeline-parallel LLM inference across GPUs on separate machines. A model too large for any single card is split into contiguous blocks of layers — one shard per GPU — and a request is served by streaming activations through the shards in order. No datacenter, no single host, and no node ever holds the whole model.

Use the idle time on your computer (Windows, Mac, Linux, or Android) to cure diseases, study global warming, discover pulsars, and do many other types of scientific research. It's safe, secure, and easy: