<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <title>inference</title>
    <link rel="self" type="application/atom+xml" href="https://links.biapy.com/guest/tags/129/feed"/>
    <updated>2026-08-26T05:49:03+00:00</updated>
    <id>https://links.biapy.com/guest/tags/129/feed</id>
            <entry>
            <id>https://links.biapy.com/links/13126</id>
            <title type="text"><![CDATA[Modelplane]]></title>
            <link rel="alternate" href="https://modelplane.ai/" />
            <link rel="via" type="application/atom+xml" href="https://links.biapy.com/links/13126"/>
            <author>
                <name><![CDATA[Biapy]]></name>
            </author>
            <summary type="text">
                <![CDATA[The open source control plane for AI inference.

Modelplane is software you install and run in your own environment to orchestrate models, the serving stack, and the infrastructure underneath across cloud, neocloud, and on-premise. It runs any model on any engine on any infrastructure, from a single GPU to disaggregated, multi-node deployments. Built on Crossplane, it is an active system that continuously reconciles your fleet toward the state you declare: provisioning inference clusters, scheduling deployments onto compatible clusters, scaling replicas, caching weights, and routing traffic.

- [Modelplane @ GitHub](https://github.com/modelplaneai/modelplane).

Related contents:

- [Building Modelplane on Crossplane @ Crossplane Blog](https://blog.crossplane.io/building-modelplane/).]]>
            </summary>
            <updated>2026-06-26T11:50:41+00:00</updated>
        </entry>
            <entry>
            <id>https://links.biapy.com/links/12910</id>
            <title type="text"><![CDATA[oMLX]]></title>
            <link rel="alternate" href="https://omlx.ai/" />
            <link rel="via" type="application/atom+xml" href="https://links.biapy.com/links/12910"/>
            <author>
                <name><![CDATA[Biapy]]></name>
            </author>
            <summary type="text">
                <![CDATA[LLM inference, optimized for your Mac.
Local AI, no more waiting on your Mac.

 LLM inference server with continuous batching &amp;amp; SSD caching for Apple Silicon — managed from the macOS menu bar.

macOS-native MLX server with smart caching. Claude Code, OpenClaw, and Cursor respond in 5 seconds, not 90. 

- [oMLX  @ GitHub](https://github.com/jundot/omlx).

Related contents:

- [\#133 - News Juin 2026, Bun passe à Rust, npm verrouille les scripts et SEO pour l&amp;#039;IA @ Double Slash :fr:](https://double-slash.dev/podcasts/news-jun26/).
- [oMLX – Faites tourner vos agents IA en local sur votre Mac @ Korben :fr:](https://korben.info/omlx-serveur-llm-local-apple-silicon.html).]]>
            </summary>
            <updated>2026-08-20T06:58:32+00:00</updated>
        </entry>
            <entry>
            <id>https://links.biapy.com/links/12109</id>
            <title type="text"><![CDATA[BitNet]]></title>
            <link rel="alternate" href="https://github.com/microsoft/BitNet" />
            <link rel="via" type="application/atom+xml" href="https://links.biapy.com/links/12109"/>
            <author>
                <name><![CDATA[Biapy]]></name>
            </author>
            <summary type="text">
                <![CDATA[Official inference framework for 1-bit LLMs.

bitnet.cpp is the official inference framework for 1-bit LLMs (e.g., BitNet b1.58). It offers a suite of optimized kernels, that support fast and lossless inference of 1.58-bit models on CPU and GPU (NPU support will coming next).]]>
            </summary>
            <updated>2026-03-13T13:00:23+00:00</updated>
        </entry>
            <entry>
            <id>https://links.biapy.com/links/42</id>
            <title type="text"><![CDATA[llm-d]]></title>
            <link rel="alternate" href="https://llm-d.ai/" />
            <link rel="via" type="application/atom+xml" href="https://links.biapy.com/links/42"/>
            <author>
                <name><![CDATA[Biapy]]></name>
            </author>
            <summary type="text">
                <![CDATA[a Kubernetes-native high-performance distributed LLM inference framework.

llm-d is a Kubernetes-native distributed inference serving stack, providing well-lit paths for anyone to serve large generative AI models at scale, with the fastest time-to-value and competitive performance per dollar for most models across most hardware accelerators.

- [llm-d @ GitHub](https://github.com/llm-d/llm-d).

Related contents:

- [Episode 616: From Boston to bootc @ Linux Unplugged](https://linuxunplugged.com/616).
- [Kubernetes as AI Infrastructure: Google Cloud, llm-d, and the CNCF @ Google Cloud Blog](https://cloud.google.com/blog/products/containers-kubernetes/llm-d-officially-a-cncf-sandbox-project/).]]>
            </summary>
            <updated>2026-04-03T16:45:21+00:00</updated>
        </entry>
            <entry>
            <id>https://links.biapy.com/links/358</id>
            <title type="text"><![CDATA[Mirai]]></title>
            <link rel="alternate" href="https://trymirai.com/" />
            <link rel="via" type="application/atom+xml" href="https://links.biapy.com/links/358"/>
            <author>
                <name><![CDATA[Biapy]]></name>
            </author>
            <summary type="text">
                <![CDATA[The future of on device AI.
Deploy high-performance AI directly in your app — with zero latency, full data privacy, and no inference costs.

Uzu is a high-performance inference engine for AI models on Apple Silicon.

- [uzu @ GitHub](https://github.com/trymirai/uzu).]]>
            </summary>
            <updated>2026-01-23T15:18:53+00:00</updated>
        </entry>
            <entry>
            <id>https://links.biapy.com/links/1240</id>
            <title type="text"><![CDATA[NVIDIA Dynamo]]></title>
            <link rel="alternate" href="https://developer.nvidia.com/dynamo" />
            <link rel="via" type="application/atom+xml" href="https://links.biapy.com/links/1240"/>
            <author>
                <name><![CDATA[Biapy]]></name>
            </author>
            <summary type="text">
                <![CDATA[A Datacenter Scale Distributed Inference Serving Framework.

NVIDIA Dynamo is a high-throughput low-latency inference framework designed for serving generative AI and reasoning models in multi-node distributed environments. Dynamo is designed to be inference engine agnostic (supports TRT-LLM, vLLM, SGLang or others) and captures LLM-specific capabilities.

- [Dynamo @ GitHub](https://github.com/ai-dynamo/dynamo).

Related contents:

- [A closer look at Dynamo, Nvidia&amp;#039;s &amp;#039;operating system&amp;#039; for AI inference @ The register](https://www.theregister.com/2025/03/23/nvidia_dynamo/).]]>
            </summary>
            <updated>2025-08-28T19:23:04+00:00</updated>
        </entry>
            <entry>
            <id>https://links.biapy.com/links/1769</id>
            <title type="text"><![CDATA[Cerebras]]></title>
            <link rel="alternate" href="https://cerebras.ai/" />
            <link rel="via" type="application/atom+xml" href="https://links.biapy.com/links/1769"/>
            <author>
                <name><![CDATA[Biapy]]></name>
            </author>
            <summary type="text">
                <![CDATA[Cerebras Inference
The world’s fastest inference -70x faster than GPU clouds,128K context, 16-bit precision.

Cerebras Inference Llama 3.3 70B runs at 2,200 tokens/s and Llama 3.1 405B at 969 tokens/s – over 70x faster than GPU clouds. Get instant responses to code-gen, summarization, and agentic tasks.

Related contents:

- [105 - les news web dev pour janvier 2025 @ Double Slash :fr:](https://double-slash.dev/podcasts/news-janv25/).]]>
            </summary>
            <updated>2025-08-28T20:51:52+00:00</updated>
        </entry>
    </feed>
