orangu 2.0.0

mnemosyne systems is happy to announce orangu 2.0.0 - the Open Source AI-powered coding environment.

1.4.0 taught the server to draw. 2.0.0 is a major release: a new architecture for the coordinator and the server, a model that is too large for one machine served by several, clips with sound and speech alongside text and pictures, and the Anthropic Claude API next to the OpenAI one.

orangu remains a complete, self-contained stack that runs entirely on your own machine - a terminal environment, an on-demand model manager, and a native GGUF inference server, written end to end in Rust . No llama.cpp, no ggml, no Python. No API keys, no telemetry, and no code leaving your machine.

New architecture

The coordinator and the server have been reworked around how a coding agent actually uses them - several requests at once, conversations that grow turn by turn, and models that come and go.

• Several models resident. With resident above one, orangu-coordinator keeps profiles that have a place of their own - their own device, or the CPU beside the card - running side by side, so moving between them swaps nothing. When the set is full, the least recently used profile is the one stopped
• Swaps that lose nothing. A swap lets every response still streaming finish first, saves the outgoing model’s conversations to disk, and loads them back on the model’s next start, so a conversation picks up where it was instead of reading its whole history again
• A plan before every start. The coordinator asks orangu-server plan --deep --json whether a model can run here - its shards are present, the architecture is known, and its weights and KV cache fit beside what is already running. A start that does not fit is refused with what it would take, and every running profile keeps serving
• Swap progress. A streamed request that waits for a swap gets its response straight away, with a progress line every two seconds that OpenAI-compatible clients ignore, and the same progress is available from GET /v1/coordinator

Massive performance improvements

• One forward pass per step. Decoding requests share a single forward pass, so four agents get about twice the tokens of one; requests that arrive together with the same new prompt build it once; and a long prompt makes way for the others’ decode steps between its layers
• A shared KV pool. The cache is shared rather than split between slots, and a request is promised its prompt and then each step’s rows as it reaches them - not its whole max_tokens - so a large answer cap keeps nobody waiting
• Speculation everywhere. Prompt-lookup speculation is on by default and runs for greedy, sampled, and constrained requests alike, with output identical to plain decoding. Multi-token-prediction heads and draft models are fetched with orangu-server download --with-drafter
• Engine work across every model family, the backends, and the tuning profiles, which are now remembered per device, model, and build so a restart skips the measurement
• orangu-bench measures what an agent does. --chat replays a coding session turn by turn with its system prompt, tools, and prefix cache, --interference measures one request’s prefill against another’s decode, --kl checks that a change has not moved the model’s answers, and --chart-grid lays the results out side by side

Workers

A model too large for one machine can now be served by several. With a [workers] section, orangu-server nodes form a pyramid: the node clients talk to runs some of the model’s layers and hands the rest to its workers, and a worker can hand part of its share to workers of its own. The top-level node samples and streams the reply, exactly as a single server does.

[workers]
host = all
port = 8400
workers = beta:8400, gamma:8400
secret = a-long-random-string

Each node holds in memory only the layers it runs, and shares follow each node’s measured speed as far as its memory holds. A worker does not even need the whole model - download = range fetches only the layers it is assigned. GET /v1/workers shows the plan and every link’s round trip and bandwidth, and orangu-bench --workers compare measures the pyramid against a single node. The new Workers chapter of the manual covers it end to end.

Video and speech

Initial support for two new kinds of output, served the same way as text and pictures:

• Video - /v1/videos with LTX-2.5, Lightricks’ 22-billion-parameter audio-video model: MP4 clips of AV1 video with a FLAC sound track, drawn together so a street at night comes with its traffic, made without an external encoder
• Speech - /v1/audio/speech with Qwen3-TTS: text spoken in a voice from a voice pack, as WAV, FLAC, or raw PCM

orangu-server download unsloth/LTX-2.5-GGUF:Q4_K_M
orangu-server --video

Anthropic Claude API

orangu-server now speaks Anthropic’s API as well as OpenAI’s, so Anthropic SDKs and tools can use your own models unchanged. /v1/messages supports streaming, content blocks and PDF documents, tools, thinking, and stop_sequences, alongside token counting, Message Batches, and the Files and Skills APIs.

New models

• DeepSeek 4.1 Flash
• Mistral 4 - Mistral Large 4, 1T parameters with 52B active
• Muse Spark - 816B parameters with 42B active
• MiniMax 3.1 Flash
• Qwen-Image 2.1 Turbo
• FLUX.2 [klein] 4B - Black Forest Labs’ compact text-to-image model
• Gemma Embedding 2

Improved PDF support

PDFs are read as context - with local OCR for scanned pages, written in Rust, and tables kept intact with their rows, columns, and merged cells - and orangu writes searchable PDFs. Both are community contributions.

Also in this release

• A container image of the whole serving stack, built from your coordinator configuration with Podman or Docker - a community contribution
• A live Monitor pane in the web console with CPU, memory, GPU, NPU, power, and thermals - a community contribution
• /issue creates an issue on GitHub or GitLab - a community contribution
• /graph diff, the subgraph a branch changed with inline diffs - a community contribution
• /comment all and an improved /export pr|issue
• [web] download gates model downloads in the web console, the build’s Git hash is reported by --version and over HTTP, and orangu-server download accepts a Hugging Face URL - community contributions
• Chunked downloads, and orangu asks for a GitHub or GitLab password when it needs one
• Image generation on x86
• Windows fixes
• The complete manual and an updated cheatsheet ship with the release as PDF and HTML, and the manual is embedded in the binary for offline reading with /manual

Our thanks to everyone who filed an issue, opened a discussion, or sent a patch.

Upgrading

The one-liner installer puts the whole stack on the machine in a single step, on Linux, macOS, and Windows:

curl -fsSL https://mnemosyne-systems.github.io/orangu/install.sh | sh

Availability

orangu 2.0.0 is available now for Linux, macOS, and Windows on x86_64 and aarch64, with source and checksums published alongside the binaries. The backends are CPU, Vulkan, Metal, CUDA, ROCm, OpenCL, and NPU. Everything is Open Source under the GNU General Public License v3.0 .

Commercial support for orangu is available from mnemosyne systems - contact sales .

We would genuinely like your feedback - especially from anyone spreading a model over several machines with workers, and from anyone pointing an Anthropic SDK at orangu-server.

Be part of the community, and be proud of your contributions ! Try it, star it, fork it, break it, tell us what’s missing.

Thanks for your time !

Overview: https://www.mnemosyne-systems.ai/products/orangu
Repository: https://github.com/mnemosyne-systems/orangu
Release: https://github.com/mnemosyne-systems/orangu/releases/tag/2.0.0