llm
🌟 OpenVoiceOS is an open-source platform for smart speakers and other voice-centric devices.
OpenVoiceOS is a community-driven, open-source voice AI platform for creating custom voice-controlled interfaces across devices with NLP, a customizable UI, and a focus on privacy and security.
Connect home devices into a powerful cluster to accelerate LLM inference. More devices mean faster performance, leveraging tensor parallelism and high-speed synchronization over Ethernet.
Supports Linux, macOS, and Windows. Optimized for ARM and x86_64 AVX2 CPUs.
Related contents:
R1 1776 is a DeepSeek-R1 reasoning model that has been post-trained by Perplexity AI to remove Chinese Communist Party censorship. The model provides unbiased, accurate, and factual information while maintaining high reasoning capabilities.
Related content:
Local LLM-Powered Coding Assistant for Vim.
VimLM is a Vim plugin that provides an LLM-powered assistant for code editing by allowing users to interact with a local LLM model through Vim commands and automatically ingesting code context.
Scira (Formerly MiniPerplx) is a minimalistic AI-powered search engine that helps you find information on the internet. Powered by Vercel AI SDK! Search with models like Grok 2.0.
Data processing with ML, LLM and Vision LLM.
Sparrow is an innovative open-source solution for efficient data extraction and processing from various documents and images. It seamlessly handles forms, bank statements, invoices, receipts, and other unstructured data sources. Sparrow stands out with its modular architecture, offering independent services and pipelines all optimized for robust performance.
Related contents:
A real-time chat application integrated with N8N workflows for automated responses and processing.
Retrieve, aggregate, filter, evaluate, rewrite and serve RSS feeds using Large Language Models for fun, research and learning purposes.
UglyFeed is a simple application designed to retrieve, aggregate, filter, rewrite, evaluate and serve content (RSS feeds) written by a large language model.
🪄 Create rich visualizations with AI
Data Formulator is an application from Microsoft Research that uses large language models to transform data, expediting the practice of data visualization.
Data Formulator is an AI-powered tool for analysts to iteratively create rich visualizations. Unlike most chat-based AI tools where users need to describe everything in natural language, Data Formulator combines user interface interactions (UI) and natural language (NL) inputs for easier interaction. This blended approach makes it easier for users to describe their chart designs while delegating data transformation to AI.
LLM Frontend for Power Users.
SillyTavern provides a single unified interface for many LLM APIs (KoboldAI/CPP, Horde, NovelAI, Ooba, Tabby, OpenAI, OpenRouter, Claude, Mistral and more), a mobile-friendly layout, Visual Novel Mode, Automatic1111 & ComfyUI API image generation integration, TTS, WorldInfo (lorebooks), customizable UI, auto-translate, more prompt options than you'd ever want or need, and endless growth potential via third-party extensions.
Your Novel Writing Toolbox. Write, without restriction.
Novelcrafter's got your back, arming you with the tools you need to dream, draft, and deliver your story.
build ml models in natural language and minimal code.
Create machine learning models with minimal code by describing what you want them to do in plain words. You explain the task, and the library builds a model for you, including data generation, feature engineering, training, and packaging.
OpenAI just published Deep Research, a very nice assistant that can perform deep searches on the web to answer user questions.
However, their agent has a huge downside: it's not open. So we've started a 24-hour rush to replicate and open-source it. Our resulting open-Deep-Research agent took the #1 rank of any open submission on the GAIA leaderboard! ✨
Related contents:
For better or for worse, LLMs are here to stay. We all read content that they produce online, most of us interact with LLM chatbots, and many of us use them to produce content of our own.
In a series of five- to ten-minute lessons, we will explain what these machines are, how they work, and how to thrive in a world where they are everywhere.
You will learn when these systems can save you a lot of time and effort. You will learn when they are likely to steer you wrong. And you will discover how to see through the hype to tell the difference. ?
LLM Gateway to provide model access, logging and usage tracking across 100+ LLMs. All in the OpenAI format.
Related contents:
- You need more than a vector database @ Redis.
- À la découverte de liteLLM, une plateforme pour les gouverner tous @ Arneo's YouTube :fr:.
- TeamPCP supply chain attack spreads to PyPI @ Reversing Labs.
- How a Poisoned Security Scanner Became the Key to Backdooring LiteLLM @ snyk.
- Building a Kubernetes-native pattern for AI infrastructure at scale @ The New Stack.
- My minute-by-minute response to the LiteLLM malware attack @ futuresearch.
- A popular Python library just became a backdoor to your entire machine @ XDA.
- LLMs on Kubernetes Part 1: Understanding the threat model @ CNCF.
The World's Leading AI Platform for Enterprise. The all-in-one platform for private and secure AI.
Cohere brings you cutting-edge multilingual models, advanced retrieval, and an AI workspace tailored for the modern enterprise — all within a single, secure platform.
Related contents:
LLM inference in C/C++.
The main goal of llama.cpp is to enable LLM inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud.
Related contents:
- Everything I've learned so far about running local LLMs @ null program.
- Run large and small language models with llama.cpp (DeepSeek-R1, Phi-4) @ Modal Docs.
- Faire tourner un LLM localement sur votre ordinateur @ Quoi de neuf les devs ? :fr:.
- Optimizing Performance with llama.cpp @ Home Assistant community.
- Grand saut dans le déploiement sur site d'un serveur LLM à moindres privilèges @ Synacktiv :fr:.
- INSTALLER llama.cpp SUR LINUX POUR DE L'IA EN LOCAL (INFÉRENCE) @ Adrien Linuxtricks' YouTube :fr:.
Never Forget Your Achievements. Get recognized for all your hard work.
Bragdoc helps you get recognized for your hard work by tracking your achievements and creating beautiful documents to share with your boss.
A Systems View of LLMs on TPUs.
This book aims to demystify the art of scaling LLMs on TPUs. We try to explain how TPUs work, how LLMs actually run at scale, and how to pick parallelism schemes during training and inference that avoid communication bottlenecks.
Simple, secure, and reproducible packaging for AI/ML projects.
KitOps is an open source DevOps tool that packages and versions your AI/ML model, datasets, code, and configuration into a reproducible artifact called a ModelKit. ModelKits are built on existing standards, ensuring compatibility with the tools your data scientists and developers already use.
The Cloud Built for AI.
Train, fine-tune and deploy AI models with RunPod.
Globally distributed GPU cloud for your AI workloads. Deploy any GPU workload seamlessly, so you can focus less on infrastructure and more on running ML models.
Rent GPUs.
Vast.ai is the market leader in low-cost cloud GPU rental. Use one simple interface to save 5-6X on GPU compute.
Secure & reliable LLMs. Test & secure your LLM apps. Open-source LLM testing used by 51,000+ developers.
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.
Related contents:
Open Universal Machine Intellingence. E2E Foundation Model Research Platform. Everything you need to build state-of-the-art foundation models, end-to-end.
Oumi is a fully open-source platform that streamlines the entire lifecycle of foundation models - from data preparation and training to evaluation and deployment. Whether you're developing on a laptop, launching large scale experiments on a cluster, or deploying models in production, Oumi provides the tools and workflows you need.
A short introduction to RLHF and post-training focused on language models.
Use your locally running AI models to assist you in your web browsing.
Page Assist is an open-source browser extension that provides a sidebar and web UI for your local AI model. It allows you to interact with your model from any webpage.
Your Open-Source Personal Assistant.
Leon is your open-source personal assistant who can live on your server. He does stuff when you ask him to.
Enable AI to control your browser. Make websites accessible for AI agents.
We make websites accessible for AI agents by extracting all interactive elements, so agents can focus on what makes their beer taste better.
Related contents:
Open Source LLM Engineering Platform. Traces, evals, prompt management and metrics to debug and improve your LLM application.
🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
Related contents:
Prompt Engineering, Evaluation, and Observability for LLM apps.
Your End-to-End Collaborative Open Source End-to-End LLM Engineering Platform. Agenta provides integrated tools for prompt engineering, versioning, evaluation, and observability—all in one place.
The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place.
A Prompt Manager that focuses on On-Premise and developer experience.
Embedditor is the open-source MS Word equivalent for embedding that helps you get the most out of your vector search.
⚡ GUI for editing LLM vector embeddings. No more blind chunking. Upload content in any file extension, join and split chunks, edit metadata and embedding tokens + remove stop-words and punctuation with one click, add images, and download in .veml to share it with your team.
LangGraph is a library for building stateful, multi-actor applications with LLMs, used to create agent and multi-agent workflows.
Related contents:
Build and ship AI products on a single, collaborative platform
Take AI products from early-stage ideas to production-grade features with tooling for experimentation, evaluation, deployment, monitoring, and collaboration.
Related contents:
The open-source visual AI programming environment and TypeScript library.
Rivet, the IDE for creating complex AI agents and prompt chaining, and embedding it in your application.
Related contents:
Enable generative AI applications to automate multistep tasks by seamlessly connecting with company systems, APIs, and data sources.
Related contents:
Your on-machine AI agent, automating engineering tasks seamlessly. an open-source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM.
Related contents:
An opinionated Laravel package that extends FakerPHP and uses openai-php/laravel to generate fake data.
A Laravel package that extends FakerPHP by adding an AI-powered data generator using OpenAI. This allows you to generate more realistic and context-aware fake data in your Laravel applications.
Related contents:
Cerebras Inference The world’s fastest inference -70x faster than GPU clouds,128K context, 16-bit precision.
Cerebras Inference Llama 3.3 70B runs at 2,200 tokens/s and Llama 3.1 405B at 969 tokens/s – over 70x faster than GPU clouds. Get instant responses to code-gen, summarization, and agentic tasks.
Related contents:
Conférences sur la générative AI.
Related contents:
The Production-Ready Open Source AI Framework.
AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
OpenTelemetry-native GenAI and LLM Application Observability.
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.
Related contents:
Open-source observability for your LLM application, based on OpenTelemetry.
OpenLLMetry is a set of extensions built on top of OpenTelemetry that gives you complete observability over your LLM application. Because it uses OpenTelemetry under the hood, it can be connected to your existing observability solutions - Datadog, Honeycomb, and others.
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator.
ONNX Runtime inference can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc.
Related contents:
An app that brings language models directly to your phone.
PocketPal AI is a pocket-sized AI assistant powered by small language models (SLMs) that run directly on your phone. Designed for both iOS and Android, PocketPal AI lets you interact with various SLMs without the need for an internet connection.
Related contents:
Cursor for Designers.
The power of Cursor for your own React website. Onlook lets you visually edit your React website and write your changes back to code in real-time.
The open source Cursor for Designers. Design directly in your live React app and publish your changes to code.
Orange Intelligence is a powerful, fully customizable productivity tool for macOS. With its elegant floating window interface, you can capture, process, and replace text seamlessly across any application. Whether you're running basic text processing, leveraging the power of large language models (LLMs) like OpenAI or local LLaMA, or creating complex agent systems, Orange Intelligence empowers you to work smarter, faster, and better.
Apple Intelligence is closed, limited, and inflexible. Orange Intelligence brings the power of customization and open source innovation to macOS, making it the perfect productivity tool for developers, researchers, and AI enthusiasts.
Easy, fast, and cheap LLM serving for everyone.
vLLM is a fast and easy-to-use library for LLM inference and serving.
Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evloved into a community-driven project with contributions from both academia and industry.
Related contents:
- How to serve LLMs with vLLM and OVHcloud AI Deploy @ OVHcloud.
- Episode 616: From Boston to bootc @ Linux Unplugged.
- What is vLLM @ RedHat.
- Faire tourner un LLM localement sur votre ordinateur @ Quoi de neuf les devs ? :fr:.
- Inside vLLM: Anatomy of a High-Throughput LLM Inference System @ Aleksa Gordić blog.
- vLLM : Maîtriser l'Inference Haute Performance pour les LLM @ DevSecOps :fr:.
- Docker Model Runner now supports vLLM on Windows @ Docker.
- Running a self-hosted LLM in Kubernetes with vLLM @ CNCF.
Open Repository of Web Crawl Data.
Common Crawl maintains a free, open repository of web crawl data that can be used by anyone.
Related contents:
15 trillion tokens of the finest data the 🌐 web has to offer.
The 🍷 FineWeb dataset consists of more than 15T tokens of cleaned and deduplicated english web data from CommonCrawl. The data processing pipeline is optimized for LLM performance and ran on the 🏭 datatrove library, our large scale data processing library.
🍷 FineWeb was originally meant to be a fully open replication of 🦅 RefinedWeb, with a release of the full dataset under the ODC-By 1.0 license. However, by carefully adding additional filtering steps, we managed to push the performance of 🍷 FineWeb well above that of the original 🦅 RefinedWeb, and models trained on our dataset also outperform models trained on other commonly used high quality web datasets (like C4, Dolma-v1.6, The Pile, SlimPajama, RedPajam2) on our aggregate group of benchmark tasks.
Related contents:
The Annual Conference on Neural Information Processing Systems.
Related content:
Multi-modal modular data ingestion and retrieval.
DataBridge is an open source library for natural language search and management of multi-modal data. Get started by installing databridge now!
DataBridge is a powerful document processing and retrieval system designed for building intelligent document-based applications. It provides a robust foundation for semantic search, document processing, and AI-powered document interactions.
Run AI with an API. Run and fine-tune open-source models. Deploy custom models at scale. All with one line of code.
Thousands of models contributed by our community. All the latest open-source models are on Replicate. They’re not just demos — they all actually work and have production-ready APIs.
AI shouldn’t be locked up inside academic papers and demos. Make it real by pushing it to Replicate.
Related contents:
Pack your codebase into AI-friendly formats.
📦 Repomix (formerly Repopack) is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, and Gemini.
Discover, download, and run local LLMs.
Related contents:
- #104 Développer des projets IA - introduction @ Double Slash :fr:.
- Faire tourner un LLM localement sur votre ordinateur @ Quoi de neuf les devs ? :fr:.
- Drames et dramas d’août @ Le RDV Tech podcast.
- LM Studio : Faire tourner son IA (LLM) facilement (Chat, Developpement, ...) @ Adrien Linuxtricks' YouTube :fr:.
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrated remarkable performance on reasoning. With RL, DeepSeek-R1-Zero naturally emerged with numerous powerful and interesting reasoning behaviors. However, DeepSeek-R1-Zero encounters challenges such as endless repetition, poor readability, and language mixing. To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates cold-start data before RL.
Related contents:
A fast tool to read text-based files in a repository or directory, chunk them, and serialize them for LLM consumption.