llm
Open Source LLM Engineering Platform. Traces, evals, prompt management and metrics to debug and improve your LLM application.
🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
Related contents:
Prompt Engineering, Evaluation, and Observability for LLM apps.
Your End-to-End Collaborative Open Source End-to-End LLM Engineering Platform. Agenta provides integrated tools for prompt engineering, versioning, evaluation, and observability—all in one place.
The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place.
A Prompt Manager that focuses on On-Premise and developer experience.
Embedditor is the open-source MS Word equivalent for embedding that helps you get the most out of your vector search.
⚡ GUI for editing LLM vector embeddings. No more blind chunking. Upload content in any file extension, join and split chunks, edit metadata and embedding tokens + remove stop-words and punctuation with one click, add images, and download in .veml to share it with your team.
LangGraph is a library for building stateful, multi-actor applications with LLMs, used to create agent and multi-agent workflows.
Related contents:
Build and ship AI products on a single, collaborative platform
Take AI products from early-stage ideas to production-grade features with tooling for experimentation, evaluation, deployment, monitoring, and collaboration.
Related contents:
The open-source visual AI programming environment and TypeScript library.
Rivet, the IDE for creating complex AI agents and prompt chaining, and embedding it in your application.
Related contents:
Enable generative AI applications to automate multistep tasks by seamlessly connecting with company systems, APIs, and data sources.
Related contents:
Your on-machine AI agent, automating engineering tasks seamlessly. an open-source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM.
Related contents:
An opinionated Laravel package that extends FakerPHP and uses openai-php/laravel to generate fake data.
A Laravel package that extends FakerPHP by adding an AI-powered data generator using OpenAI. This allows you to generate more realistic and context-aware fake data in your Laravel applications.
Related contents:
Cerebras Inference The world’s fastest inference -70x faster than GPU clouds,128K context, 16-bit precision.
Cerebras Inference Llama 3.3 70B runs at 2,200 tokens/s and Llama 3.1 405B at 969 tokens/s – over 70x faster than GPU clouds. Get instant responses to code-gen, summarization, and agentic tasks.
Related contents:
Conférences sur la générative AI.
Related contents:
The Production-Ready Open Source AI Framework.
AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
OpenTelemetry-native GenAI and LLM Application Observability.
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.
Open-source observability for your LLM application, based on OpenTelemetry.
OpenLLMetry is a set of extensions built on top of OpenTelemetry that gives you complete observability over your LLM application. Because it uses OpenTelemetry under the hood, it can be connected to your existing observability solutions - Datadog, Honeycomb, and others.
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator.
ONNX Runtime inference can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc.
Related contents:
An app that brings language models directly to your phone.
PocketPal AI is a pocket-sized AI assistant powered by small language models (SLMs) that run directly on your phone. Designed for both iOS and Android, PocketPal AI lets you interact with various SLMs without the need for an internet connection.
Related contents:
Cursor for Designers.
The power of Cursor for your own React website. Onlook lets you visually edit your React website and write your changes back to code in real-time.
The open source Cursor for Designers. Design directly in your live React app and publish your changes to code.
Orange Intelligence is a powerful, fully customizable productivity tool for macOS. With its elegant floating window interface, you can capture, process, and replace text seamlessly across any application. Whether you're running basic text processing, leveraging the power of large language models (LLMs) like OpenAI or local LLaMA, or creating complex agent systems, Orange Intelligence empowers you to work smarter, faster, and better.
Apple Intelligence is closed, limited, and inflexible. Orange Intelligence brings the power of customization and open source innovation to macOS, making it the perfect productivity tool for developers, researchers, and AI enthusiasts.
Easy, fast, and cheap LLM serving for everyone.
vLLM is a fast and easy-to-use library for LLM inference and serving.
Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evloved into a community-driven project with contributions from both academia and industry.
Related contents:
- How to serve LLMs with vLLM and OVHcloud AI Deploy @ OVHcloud.
- Episode 616: From Boston to bootc @ Linux Unplugged.
- What is vLLM @ RedHat.
- Faire tourner un LLM localement sur votre ordinateur @ Quoi de neuf les devs ? :fr:.
- Inside vLLM: Anatomy of a High-Throughput LLM Inference System @ Aleksa Gordić blog.
- vLLM : Maîtriser l'Inference Haute Performance pour les LLM @ DevSecOps :fr:.
Open Repository of Web Crawl Data.
Common Crawl maintains a free, open repository of web crawl data that can be used by anyone.
Related contents:
15 trillion tokens of the finest data the 🌐 web has to offer.
The 🍷 FineWeb dataset consists of more than 15T tokens of cleaned and deduplicated english web data from CommonCrawl. The data processing pipeline is optimized for LLM performance and ran on the 🏭 datatrove library, our large scale data processing library.
🍷 FineWeb was originally meant to be a fully open replication of 🦅 RefinedWeb, with a release of the full dataset under the ODC-By 1.0 license. However, by carefully adding additional filtering steps, we managed to push the performance of 🍷 FineWeb well above that of the original 🦅 RefinedWeb, and models trained on our dataset also outperform models trained on other commonly used high quality web datasets (like C4, Dolma-v1.6, The Pile, SlimPajama, RedPajam2) on our aggregate group of benchmark tasks.
Related contents:
The Annual Conference on Neural Information Processing Systems.
Related content:
Multi-modal modular data ingestion and retrieval.
DataBridge is an open source library for natural language search and management of multi-modal data. Get started by installing databridge now!
DataBridge is a powerful document processing and retrieval system designed for building intelligent document-based applications. It provides a robust foundation for semantic search, document processing, and AI-powered document interactions.
Run AI with an API. Run and fine-tune open-source models. Deploy custom models at scale. All with one line of code.
Thousands of models contributed by our community. All the latest open-source models are on Replicate. They’re not just demos — they all actually work and have production-ready APIs.
AI shouldn’t be locked up inside academic papers and demos. Make it real by pushing it to Replicate.
Related contents:
Pack your codebase into AI-friendly formats.
📦 Repomix (formerly Repopack) is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, and Gemini.
Discover, download, and run local LLMs.
Related contents:
- #104 Développer des projets IA - introduction @ Double Slash :fr:.
- Faire tourner un LLM localement sur votre ordinateur @ Quoi de neuf les devs ? :fr:.
- Drames et dramas d’août @ Le RDV Tech podcast.
- LM Studio : Faire tourner son IA (LLM) facilement (Chat, Developpement, ...) @ Adrien Linuxtricks' YouTube :fr:.
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrated remarkable performance on reasoning. With RL, DeepSeek-R1-Zero naturally emerged with numerous powerful and interesting reasoning behaviors. However, DeepSeek-R1-Zero encounters challenges such as endless repetition, poor readability, and language mixing. To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates cold-start data before RL.
Related contents:
A fast tool to read text-based files in a repository or directory, chunk them, and serialize them for LLM consumption.
Explore Agent Recipes
Explore common agent recipes with ready to copy code to improve your LLM applications.
Related contents:
This is a simple demonstration of more advanced, agentic patterns built on top of the Realtime API. - openai/openai-realtime-agents
Open-source document processing platform built for knowledge workers.
Rowfill helps extract, analyze, and process data from complex documents, images, PDFs and more with advanced AI capabilities.
structured-logprobs is an open-source Python library that enhances OpenAI's structured outputs by providing detailed information about token log probabilities.
This library is designed to offer valuable insights into the reliability of an LLM's structured outputs. It works with OpenAI's Structured Outputs, a feature that ensures the model consistently generates responses adhering to a supplied JSON Schema. This eliminates concerns about missing required keys or hallucinating invalid values.
Automate and clean your inbox. Clean Up Your Inbox In Minutes.
Bulk unsubscribe from newsletters, automate your emails with AI, block cold emails, and view your analytics. Open-source.
Everything to Markdown.
E2M converts various file types (doc, docx, epub, html, htm, url, pdf, ppt, pptx, mp3, m4a) into Markdown. It’s easy to install, with dedicated parsers and converters, supporting custom configs. E2M offers an all-in-one, flexible, and open-source solution.
Open weights LLM for French, English, German, Spanish and Italian.
Related contents:
LLM génératifs ouverts et performants.
Le Consortium OpenLLM France réunit 17 acteurs qui se sont rassemblés dans le prolongement de la création de la communauté OpenLLM France qui fédère à ce jour un écosystème de près de 200 entités (laboratoires publics de recherche, fournisseurs potentiels de données, acteurs technologiques spécialisés, fournisseurs de cas d'usage...). Ces acteurs échangent de manière publique et transparente depuis le début de l’été 2023 sur le serveur Discord de la communauté.
A unified interface for LLMs. Better prices, better uptime, no subscription.
Related contents:
🐢 Open-Source Evaluation & Testing for AI & LLM systems.
The testing framework dedicated to ML models, from tabular to LLMs. Control risks of performance, bias and security issues in AI systems.
Fetch an entire site and save it as a text file (to be used with AI models).
LLM Agent and Evaluation Framework for Autonomous Penetration Testing.
We introduce HackSynth, a novel Large Language Model (LLM)-based agent capable of autonomous penetration testing. HackSynth's dual-module architecture includes a Planner and a Summarizer, which enable it to generate commands and process feedback iteratively. To benchmark HackSynth, we propose two new Capture The Flag (CTF)-based benchmark sets utilizing the popular platforms PicoCTF and OverTheWire. These benchmarks include two hundred challenges across diverse domains and difficulties, providing a standardized framework for evaluating LLM-based penetration testing agents.
Le numéro de confiance pour vérifier les faits.
Un seul numéro (gratuit) pour contrer la désinformation et apaiser le débat public.
MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools.
Related contents:
- Model Context Protocol (MCP) Course @ Hugging Face.
- Introducing the Model Context Protocol @ Anthropic.
- LLM Agent Assisted Coding @ stafford williams.
- Everyone is talking about MCP @ Adam Silverman's X.
- What is Model Context Protocol (MCP)? How it simplifies AI integrations compared to APIs @ Norah Sakal.
- What is MCP by Anthropic?(Model Context Protocol) @ Toward AI.
- A Deep Dive Into MCP and the Future of AI Tooling @ Andreessen Horowitz.
- What is MCP (Model Context Protocol)? @ daily.dev.
- 230. MCP - It's Hot, But Will It Win? @ Hardcore Software by Steven Sinofsky.
- MCP (Model Context Protocol): Simply explained in 5 minutes @ High Growth Engineer.
- The “S” in MCP Stands for Security @ Elena Cross' Medium.
- Everything Wrong with MCP @ Shrivu's Substack.
- MCPs, Gatekeepers, and the Future of AI @ I Am Charlie Graham.
- A Critical Look at MCP @ Raz Blog.
- MCP vs API @ Glama.
- MCP explained without hype or fluff @ nilenso.
- What are MCP Servers? @ The Fly Blog.
- Block's Playbook for Designing MCP Servers @ Block Engineering Blog.
- MCP is eating the world—and it's here to stay @ Stainless.
- MCP: An (Accidentally) Universal Plugin System @ Works on My Machine.
- MCP is eating the world—and it's here to stay @ Stainless.
- MCP Vulnerabilities Every Developer Should Know @ Composio.
- Your MCP Doesn’t Need 30 Tools: It Needs Code @ Armin Ronacher's Thoughts and Writings.
- The State of MCP Security @ Pynt.
- MCPs Are Just Other People's Prompts Pointing to Other People's Code @ Daniel Miessler.
- you need to learn MCP RIGHT NOW!! (Model Context Protocol) @ NetworkChuck's YouTube.
- Episode #112: A new direction for AI developer tooling @ Changelog & Friends.
Opensource, self-hosted AI coding assistant. Secure, flexible, and transparent AI coding.
Tabby is a self-hosted AI coding assistant, offering an open-source and on-premises alternative to GitHub Copilot.
A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.
Use LLMs and LLM Vision (OCR) to handle paperless-ngx - Document Digitalization powered by AI.
paperless-gpt seamlessly pairs with paperless-ngx to generate AI-powered document titles and tags, saving you hours of manual sorting. While other tools may offer AI chat features, paperless-gpt stands out by supercharging OCR with LLMs—ensuring high accuracy, even with tricky scans. If you’re craving next-level text extraction and effortless document organization, this is your solution.
Open-Source, Free, and AI-Powered News in Short.
Your open-source, AI-powered news companion that redefines how you consume news. Get bite-sized summaries from trusted sources worldwide, personalized to your interests. Stay informed without the overwhelm.
An automated document analyzer for Paperless-ngx using OpenAI API and Ollama (Mistral, llama, phi 3, gemma 2) to automatically analyze and tag your documents.
It features: Automode, Manual Mode, Ollama and OpenAI, a Chat function to query your documents with AI, a modern and intuitive Webinterface.
Storyteller is a self-hosted platform for creating and reading ebooks with synced narration. It's made of three components: the API server, the web interface, and the mobile apps. Together, these components allow you to take audiobooks and ebooks that you already own and automatically synchronize them, as well as read or listen to (or both!) the resulting synced books.
Related contents:
Built to keep you in flow state
The first agentic IDE, and then some. The Windsurf Editor is where the work of developers and AI truly flow together, allowing for a coding experience that feels like literal magic.
Open-Source LLM-Friendly Web Crawler & Scraper.
Crawl4AI delivers blazing-fast, AI-ready web crawling tailored for large language models, AI agents, and data pipelines. Fully open source, flexible, and built for real-time performance, Crawl4AI empowers developers with unmatched speed, precision, and deployment ease.
Your AI Second Brain. Ask anything, understand documents, create new content.
Khoj is a personal AI app to extend your capabilities. It smoothly scales up from an on-device personal AI to a cloud-scale enterprise AI.
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
Related contents:
Open-Source Data Movement for LLMs. AI Platform. Data integration platform for ELT pipelines from APIs, databases & files to databases, warehouses & lakes.
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
AI Code Generator.
An open source Claude Artifacts – generate small apps with one prompt. Powered by Llama 3 405B & Together.ai.
⚡ Build context-aware reasoning applications ⚡
LangChain is a framework for developing applications powered by large language models (LLMs).
Simple frontend for LLMs built in react-native.
ChatterUI is a native mobile frontend for LLMs.
Run LLMs on device or connect to various commercial or open source APIs. ChatterUI aims to provide a mobile-friendly interface with fine-grained control over chat structuring.
Related contents:
A course on neural networks that starts all the way at the basics. The course is a series of YouTube videos where we code and train neural networks together. The Jupyter notebooks we build in the videos are then captured here inside the lectures directory. Every lecture also has a set of exercises included in the video description. (This may grow into something more respectable).
MMAudio generates synchronized audio given video and/or text inputs.
Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis.
Related contents: