llm
OpenTelemetry-native GenAI and LLM Application Observability.
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.
Related contents:
Open-source observability for your LLM application, based on OpenTelemetry.
OpenLLMetry is a set of extensions built on top of OpenTelemetry that gives you complete observability over your LLM application. Because it uses OpenTelemetry under the hood, it can be connected to your existing observability solutions - Datadog, Honeycomb, and others.
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator.
ONNX Runtime inference can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc.
Related contents:
An app that brings language models directly to your phone.
PocketPal AI is a pocket-sized AI assistant powered by small language models (SLMs) that run directly on your phone. Designed for both iOS and Android, PocketPal AI lets you interact with various SLMs without the need for an internet connection.
Related contents:
Cursor for Designers.
The power of Cursor for your own React website. Onlook lets you visually edit your React website and write your changes back to code in real-time.
The open source Cursor for Designers. Design directly in your live React app and publish your changes to code.
Orange Intelligence is a powerful, fully customizable productivity tool for macOS. With its elegant floating window interface, you can capture, process, and replace text seamlessly across any application. Whether you're running basic text processing, leveraging the power of large language models (LLMs) like OpenAI or local LLaMA, or creating complex agent systems, Orange Intelligence empowers you to work smarter, faster, and better.
Apple Intelligence is closed, limited, and inflexible. Orange Intelligence brings the power of customization and open source innovation to macOS, making it the perfect productivity tool for developers, researchers, and AI enthusiasts.
Easy, fast, and cheap LLM serving for everyone.
vLLM is a fast and easy-to-use library for LLM inference and serving.
Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evloved into a community-driven project with contributions from both academia and industry.
Related contents:
- How to serve LLMs with vLLM and OVHcloud AI Deploy @ OVHcloud.
- Episode 616: From Boston to bootc @ Linux Unplugged.
- What is vLLM @ RedHat.
- Faire tourner un LLM localement sur votre ordinateur @ Quoi de neuf les devs ? :fr:.
- Inside vLLM: Anatomy of a High-Throughput LLM Inference System @ Aleksa Gordić blog.
- vLLM : Maîtriser l'Inference Haute Performance pour les LLM @ DevSecOps :fr:.
- Docker Model Runner now supports vLLM on Windows @ Docker.
- Running a self-hosted LLM in Kubernetes with vLLM @ CNCF.
Open Repository of Web Crawl Data.
Common Crawl maintains a free, open repository of web crawl data that can be used by anyone.
Related contents:
15 trillion tokens of the finest data the 🌐 web has to offer.
The 🍷 FineWeb dataset consists of more than 15T tokens of cleaned and deduplicated english web data from CommonCrawl. The data processing pipeline is optimized for LLM performance and ran on the 🏭 datatrove library, our large scale data processing library.
🍷 FineWeb was originally meant to be a fully open replication of 🦅 RefinedWeb, with a release of the full dataset under the ODC-By 1.0 license. However, by carefully adding additional filtering steps, we managed to push the performance of 🍷 FineWeb well above that of the original 🦅 RefinedWeb, and models trained on our dataset also outperform models trained on other commonly used high quality web datasets (like C4, Dolma-v1.6, The Pile, SlimPajama, RedPajam2) on our aggregate group of benchmark tasks.
Related contents:
The Annual Conference on Neural Information Processing Systems.
Related content:
Multi-modal modular data ingestion and retrieval.
DataBridge is an open source library for natural language search and management of multi-modal data. Get started by installing databridge now!
DataBridge is a powerful document processing and retrieval system designed for building intelligent document-based applications. It provides a robust foundation for semantic search, document processing, and AI-powered document interactions.
Run AI with an API. Run and fine-tune open-source models. Deploy custom models at scale. All with one line of code.
Thousands of models contributed by our community. All the latest open-source models are on Replicate. They’re not just demos — they all actually work and have production-ready APIs.
AI shouldn’t be locked up inside academic papers and demos. Make it real by pushing it to Replicate.
Related contents:
Pack your codebase into AI-friendly formats.
📦 Repomix (formerly Repopack) is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, and Gemini.
Discover, download, and run local LLMs.
Related contents:
- #104 Développer des projets IA - introduction @ Double Slash :fr:.
- Faire tourner un LLM localement sur votre ordinateur @ Quoi de neuf les devs ? :fr:.
- Drames et dramas d’août @ Le RDV Tech podcast.
- LM Studio : Faire tourner son IA (LLM) facilement (Chat, Developpement, ...) @ Adrien Linuxtricks' YouTube :fr:.
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrated remarkable performance on reasoning. With RL, DeepSeek-R1-Zero naturally emerged with numerous powerful and interesting reasoning behaviors. However, DeepSeek-R1-Zero encounters challenges such as endless repetition, poor readability, and language mixing. To address these issues and further enhance reasoning performance, we introduce DeepSeek-R1, which incorporates cold-start data before RL.
Related contents:
A fast tool to read text-based files in a repository or directory, chunk them, and serialize them for LLM consumption.
Explore Agent Recipes
Explore common agent recipes with ready to copy code to improve your LLM applications.
Related contents:
This is a simple demonstration of more advanced, agentic patterns built on top of the Realtime API. - openai/openai-realtime-agents
Open-source document processing platform built for knowledge workers.
Rowfill helps extract, analyze, and process data from complex documents, images, PDFs and more with advanced AI capabilities.
structured-logprobs is an open-source Python library that enhances OpenAI's structured outputs by providing detailed information about token log probabilities.
This library is designed to offer valuable insights into the reliability of an LLM's structured outputs. It works with OpenAI's Structured Outputs, a feature that ensures the model consistently generates responses adhering to a supplied JSON Schema. This eliminates concerns about missing required keys or hallucinating invalid values.
Automate and clean your inbox. Clean Up Your Inbox In Minutes.
Bulk unsubscribe from newsletters, automate your emails with AI, block cold emails, and view your analytics. Open-source.
Everything to Markdown.
E2M converts various file types (doc, docx, epub, html, htm, url, pdf, ppt, pptx, mp3, m4a) into Markdown. It’s easy to install, with dedicated parsers and converters, supporting custom configs. E2M offers an all-in-one, flexible, and open-source solution.
Open weights LLM for French, English, German, Spanish and Italian.
Related contents:
LLM génératifs ouverts et performants.
Le Consortium OpenLLM France réunit 17 acteurs qui se sont rassemblés dans le prolongement de la création de la communauté OpenLLM France qui fédère à ce jour un écosystème de près de 200 entités (laboratoires publics de recherche, fournisseurs potentiels de données, acteurs technologiques spécialisés, fournisseurs de cas d'usage...). Ces acteurs échangent de manière publique et transparente depuis le début de l’été 2023 sur le serveur Discord de la communauté.
A unified interface for LLMs. Better prices, better uptime, no subscription.
Related contents:
🐢 Open-Source Evaluation & Testing for AI & LLM systems.
The testing framework dedicated to ML models, from tabular to LLMs. Control risks of performance, bias and security issues in AI systems.
Fetch an entire site and save it as a text file (to be used with AI models).
LLM Agent and Evaluation Framework for Autonomous Penetration Testing.
We introduce HackSynth, a novel Large Language Model (LLM)-based agent capable of autonomous penetration testing. HackSynth's dual-module architecture includes a Planner and a Summarizer, which enable it to generate commands and process feedback iteratively. To benchmark HackSynth, we propose two new Capture The Flag (CTF)-based benchmark sets utilizing the popular platforms PicoCTF and OverTheWire. These benchmarks include two hundred challenges across diverse domains and difficulties, providing a standardized framework for evaluating LLM-based penetration testing agents.
Le numéro de confiance pour vérifier les faits.
Un seul numéro (gratuit) pour contrer la désinformation et apaiser le débat public.
MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools.
Related contents:
- Model Context Protocol (MCP) Course @ Hugging Face.
- Introducing the Model Context Protocol @ Anthropic.
- LLM Agent Assisted Coding @ stafford williams.
- Everyone is talking about MCP @ Adam Silverman's X.
- What is Model Context Protocol (MCP)? How it simplifies AI integrations compared to APIs @ Norah Sakal.
- What is MCP by Anthropic?(Model Context Protocol) @ Toward AI.
- A Deep Dive Into MCP and the Future of AI Tooling @ Andreessen Horowitz.
- What is MCP (Model Context Protocol)? @ daily.dev.
- 230. MCP - It's Hot, But Will It Win? @ Hardcore Software by Steven Sinofsky.
- MCP (Model Context Protocol): Simply explained in 5 minutes @ High Growth Engineer.
- The “S” in MCP Stands for Security @ Elena Cross' Medium.
- Everything Wrong with MCP @ Shrivu's Substack.
- MCPs, Gatekeepers, and the Future of AI @ I Am Charlie Graham.
- A Critical Look at MCP @ Raz Blog.
- MCP vs API @ Glama.
- MCP explained without hype or fluff @ nilenso.
- What are MCP Servers? @ The Fly Blog.
- Block's Playbook for Designing MCP Servers @ Block Engineering Blog.
- MCP is eating the world—and it's here to stay @ Stainless.
- MCP: An (Accidentally) Universal Plugin System @ Works on My Machine.
- MCP is eating the world—and it's here to stay @ Stainless.
- MCP Vulnerabilities Every Developer Should Know @ Composio.
- Your MCP Doesn’t Need 30 Tools: It Needs Code @ Armin Ronacher's Thoughts and Writings.
- The State of MCP Security @ Pynt.
- MCPs Are Just Other People's Prompts Pointing to Other People's Code @ Daniel Miessler.
- you need to learn MCP RIGHT NOW!! (Model Context Protocol) @ NetworkChuck's YouTube.
- Episode #112: A new direction for AI developer tooling @ Changelog & Friends.
- What is special about MCP? @ Jessitron.
- Code execution with MCP: Building more efficient agents @ Engineering @ Anthropic.
- MCP Apps: Extending servers with interactive user interfaces @ mcp blog.
- Model Context Protocol (MCP) Tutorial: Build Your First MCP Server in 6 Steps @ towards data science.
- MCP is dead. Long live the CLI @ Eric Holmes.
Opensource, self-hosted AI coding assistant. Secure, flexible, and transparent AI coding.
Tabby is a self-hosted AI coding assistant, offering an open-source and on-premises alternative to GitHub Copilot.
A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.
Use LLMs and LLM Vision (OCR) to handle paperless-ngx - Document Digitalization powered by AI.
paperless-gpt seamlessly pairs with paperless-ngx to generate AI-powered document titles and tags, saving you hours of manual sorting. While other tools may offer AI chat features, paperless-gpt stands out by supercharging OCR with LLMs—ensuring high accuracy, even with tricky scans. If you’re craving next-level text extraction and effortless document organization, this is your solution.
Open-Source, Free, and AI-Powered News in Short.
Your open-source, AI-powered news companion that redefines how you consume news. Get bite-sized summaries from trusted sources worldwide, personalized to your interests. Stay informed without the overwhelm.
An automated document analyzer for Paperless-ngx using OpenAI API and Ollama (Mistral, llama, phi 3, gemma 2) to automatically analyze and tag your documents.
It features: Automode, Manual Mode, Ollama and OpenAI, a Chat function to query your documents with AI, a modern and intuitive Webinterface.
Storyteller is a self-hosted platform for creating and reading ebooks with synced narration. It's made of three components: the API server, the web interface, and the mobile apps. Together, these components allow you to take audiobooks and ebooks that you already own and automatically synchronize them, as well as read or listen to (or both!) the resulting synced books.
Related contents:
Built to keep you in flow state
The first agentic IDE, and then some. The Windsurf Editor is where the work of developers and AI truly flow together, allowing for a coding experience that feels like literal magic.
Open-Source LLM-Friendly Web Crawler & Scraper.
Crawl4AI delivers blazing-fast, AI-ready web crawling tailored for large language models, AI agents, and data pipelines. Fully open source, flexible, and built for real-time performance, Crawl4AI empowers developers with unmatched speed, precision, and deployment ease.
Your AI Second Brain. Ask anything, understand documents, create new content.
Khoj is a personal AI app to extend your capabilities. It smoothly scales up from an on-device personal AI to a cloud-scale enterprise AI.
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
Related contents:
Open-Source Data Movement for LLMs. AI Platform. Data integration platform for ELT pipelines from APIs, databases & files to databases, warehouses & lakes.
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
AI Code Generator.
An open source Claude Artifacts – generate small apps with one prompt. Powered by Llama 3 405B & Together.ai.
Simple frontend for LLMs built in react-native.
ChatterUI is a native mobile frontend for LLMs.
Run LLMs on device or connect to various commercial or open source APIs. ChatterUI aims to provide a mobile-friendly interface with fine-grained control over chat structuring.
Related contents:
A course on neural networks that starts all the way at the basics. The course is a series of YouTube videos where we code and train neural networks together. The Jupyter notebooks we build in the videos are then captured here inside the lectures directory. Every lecture also has a set of exercises included in the video description. (This may grow into something more respectable).
MMAudio generates synchronized audio given video and/or text inputs.
Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis.
Related contents:
API Parrot is the tool specifically designed to reverse engineer the HTTP APIs of any website. Making life easier for developers looking to automate, integrate or scrape websites without public APIs.
Power Your AI with Live Data.
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
A web browser for AI agents & applications. An AI web browsing framework focused on simplicity and extensibility.
Python tool for converting files and office documents to Markdown. MarkItDown is a utility for converting various files to Markdown (e.g., for indexing, text analysis, etc).
Related contents:
Convert Anything into Structured Actionable Data.
Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks.
OmniParse is a platform that ingests and parses any unstructured data into structured, actionable data optimized for GenAI (LLM) applications. Whether you are working with documents, tables, images, videos, audio files, or web pages, OmniParse prepares your data to be clean, structured, and ready for AI applications such as RAG, fine-tuning, and more
Your AI Meeting Copilot (Early Preview)
Take the hassle out of meetings with Amurex, the ultimate Chrome extension that transforms how you manage and stay productive during meetings.
Amurex is your simple yet powerful AI meeting assistant that seamlessly integrates into your workflow. Built with cutting-edge AI, Amurex ensures you never miss a detail, always stay on top of action items, and make every meeting more productive.
Prompt, run, edit, and deploy full-stack web applications using any LLM you want!
bolt.diy allows you to choose the LLM that you use for each prompt! Currently, you can use OpenAI, Anthropic, Ollama, OpenRouter, Gemini, LMStudio, Mistral, xAI, HuggingFace, DeepSeek, or Groq models - and it is easily extended to use any other model supported by the Vercel AI SDK! See the instructions below for running this locally and extending it to include more models.
OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference.
OpenVINO is an open-source toolkit for optimizing and deploying deep learning models from cloud to edge. It accelerates deep learning inference across various use cases, such as generative AI, video, audio, and language with models from popular frameworks like PyTorch, TensorFlow, ONNX, and more. Convert and optimize models, and deploy across a mix of Intel® hardware and environments, on-premises and on-device, in the browser or in the cloud.
Low code LLM Apps Builder. Build LLM Apps Easily.
Open source low-code tool for developers to build customized LLM orchestration flow & AI agents.
Related contents:
Agent Framework / shim to use Pydantic with LLMs.
PydanticAI is a Python Agent Framework designed to make it less painful to build production grade applications with Generative AI.
Related contents:
Your Terminal Assistant.
BashSenpai is a command-line tool that utilizes the power of ChatGPT, bringing it straight to your terminal. You can ask questions and receive insightful responses related to shell scripting, making it an indispensable asset for both beginners and seasoned users alike.
Turn websites into LLM - ready data.
DataFuel API scrapes entire websites and knowledge bases in a single query. Get clean, markdown-structured web data instantly for your RAG systems and AI models. No complex scraping code needed.
A smarter web fuzzing tool that combines local LLM models and ffuf to optimize directory and file discovery.
This tool enhances traditional web fuzzing by using local AI language models (via Ollama) to generate intelligent guesses for potential paths and filenames.
A research project to add some brrrrrr to Burp.
"burpference" started as a research idea of offensive agent capabilities and is a fun take on Burp Suite and running inference. The extension is open-source and designed to capture in-scope HTTP requests and responses from Burp's proxy history and ship them to a remote LLM API in JSON format. It's designed with a flexible approach where you can configure custom system prompts, store API keys and select remote hosts from numerous model providers as well as the ability for you to create your own API configuration. The idea is for an LLM to act as an agent in an offensive web application engagement to leverage your skills and surface findings and lingering vulnerabilities. By being able to create your own configuration and model provider allows you to also host models locally via Ollama to prevent potential high inference costs and potential network delays or rate limits.