orangu

orangu is a terminal-based, locally-hosted AI coding environment - and a complete, self-contained AI coding stack, not just a client. It ships the coding environment, the model coordinator, and a native GGUF inference server, written end to end in Rust , so every line of your code stays on your own machine.

With zero telemetry and no cloud uploads, orangu is built for developers in regulated or air-gapped environments - and eliminates the per-token costs of cloud-based assistants by running your own self-hosted models. After orangu-server and your models are downloaded, no Internet connection is required.

It is named after the Orangutan - the smartest ape.

A complete stack from one vendor

Most local AI setups are a patchwork: one tool for the editor, a separate engine for inference, and glue to manage which model is loaded. orangu is the whole stack in one project - three cooperating programs, one language, one license, one installer:

• orangu - the workspace-aware coding environment you drive: the terminal interface, local and Git/forge tools, /review and /auto_review, the knowledge graph, semantic /search, and the context-compression engine
• orangu-coordinator - an optional companion HTTP proxy that starts and stops the server on demand and swaps to whichever model each request needs, so a single-GPU machine can use a different model per role - or, with room for more, keeps several models resident side by side
• orangu-server - the inference engine itself: GGUF loading, tokenization, the transformer forward pass, sampling, and request scheduling implemented directly in Rust, with no llama.cpp, no ggml, and no Python - plus the model inventory and the web console, from the same binary

Every layer speaks the OpenAI-compatible API, so the pieces stay cleanly separated - yet they build, ship, install, and run as one.

A fourth binary, orangu-gguf, builds the model files the server runs: it pretrains a model from scratch out of a JSON manifest of permissively-licensed repositories - cloning, tokenizer, corpus packing, training, and GGUF export in one binary - and rewrites an existing model at a smaller weight format. With that, the stack covers the whole path from source repositories to a served model without leaving the project.

A fifth binary, orangu-bench, measures the stack on your own hardware and your own models instead of asking you to take our word for it. Each measurement point gets a fresh server, and warmups settle the server’s own tuning probes before anything is timed. It also runs head-to-head comparisons against other inference engines, and orangu-bench --image sweeps picture sizes for image generation. The same benchmarks run from the web console.

Key Features

• Complete privacy - Your code stays on your machine; zero telemetry, no cloud uploads
• Built-in code review - Interactive (/review) and automated LLM-driven (/auto_review) review with category-based findings and confidence scoring to filter out false positives
• Train your own model - orangu-gguf pretrains a GGUF model from random weights out of a manifest of permissively-licensed repositories, with no Python and no training framework, and quantizes existing models down to Q2_K
• Native inference server - orangu-server runs GGUF models on CPU, GPU (Vulkan, Metal, CUDA, ROCm, OpenCL), or NPU with continuous batching, a shared KV pool, prefix caching, speculative decoding, and both an OpenAI-compatible and an Anthropic-compatible API, and doubles as the machine’s model inventory
• Runs the big models - Support for the large model families - DeepSeek 4.1, GLM 5.3, Kimi K3, Mistral 4, Muse Spark, MiniMax 3.1, Inkling-Small, Ling, NVIDIA Nemotron, Ornith 1.5, Qwen 3.8, and Ternary Bonsai 2 - with multi-card support so a large model can be spread across the GPUs you already have, and multi-token-prediction draft heads attached automatically for the models that ship them
• Workers - A model too large for one machine is served by several: orangu-server nodes form a pyramid, each holding only the layers it runs, with shares that follow each node’s measured speed
• Image, video, and speech generation - orangu-server generates and edits images with Qwen-Image 2.1 and FLUX.2 [klein], makes video clips with sound with LTX-2.5, and speaks text with Qwen3-TTS - all served from GGUF like any language model, on the CPU as well as the GPU
• Container image - The whole serving stack, built from your coordinator configuration as one Podman or Docker image
• Prometheus metrics - Slot and queue gauges, latency histograms, and request and token counters on a dedicated metrics port, plus a /ready probe for load balancers
• Benchmarking - orangu-bench measures prefill, decode, and image generation on your own hardware and models, with fresh servers per point, proper warmups, and head-to-head comparisons against other engines - from the command line or the web console
• Workflow language - Describe multi-step, multi-project jobs in YAML over the same slash commands you drive by hand, including a code-and-review loop that runs until the goal is verified, all validated up front with --dry-run
• NPU support - Rockchip RKNPU matmul offload on boards like the RK3588, alongside detection and reporting for Arm China’s Zhouyi AIPU
• MCP support - Connect external tools over the Model Context Protocol, with local MCP server management, refresh, and reconnect handled inside orangu
• Built-in web console - A browser chat UI served from the same binary, with markdown and syntax highlighting, LaTeX, Mermaid and PlantUML diagrams, live tokens/second, model management, benchmarking with orangu-bench, and persistent history you can prune - no build step, no external assets
• Git-centric workflow - Full Git plus GitHub and GitLab support, driven straight from the prompt, with natural-language aliases for nearly every command - including /add_repository for pulling in a contributor’s branch and /revert for backing out a pushed commit
• Workspace-aware tools - Read, edit, list, search, and fetch files and run shell commands, with project and git context
• Builds your project - /build detects the toolchain (Cargo, CMake, Autotools, Meson, Maven, Python, Go, plain make) and runs format, lint, build, and test as one reported pipeline
• Knowledge graph & visualization - Tree-sitter powered dependency graph of every function, class, and call, with an interactive HTML visualization (/graph)
• Semantic code search - Find code by meaning, not just text, using offline embeddings (/search)
• Code duplication report - Detect duplicate code across 20+ languages using AST similarity scoring (/duplicates)
• Agent skills & memory - Discovers reusable skills and merges persistent AGENTS.md instructions and memory across sessions
• Developer and Committer mode - /developer and /committer shape orangu’s behaviour around what you are doing right now: writing the change, or getting it committed
• License header support - Keeps license headers consistent across a codebase as you work in it
• Multi-workspace support - Open multiple projects as tabs in a single instance
• Scriptable and schedulable - Run a single prompt or command and exit (-p), and let the built-in cron-style scheduler run jobs unattended
• Single native binary - Written in Rust for fast startup with no runtime dependencies
• Offline documentation - Complete embedded manual with full-text search
• Advanced context compression - AST-aware file downsampling, intelligent Git diffs, and secret redaction to protect the LLM’s context window

Code review

Code review is orangu’s standout feature, and it runs entirely on your own hardware. Both reviewers work on the changes on your current branch - committed work plus what is still in the working tree - measured against the merge base with the default branch.

• /review - a full-screen, two-pane view with the file checklist on the right and the selected file’s diff on the left. Approve or reject each file, comment on any diff line under a chosen category (Overall, Code, Security, Memory, Performance, Test Suite, Documentation), ask the model about the selected file on demand, and open any file in your $EDITOR without leaving the view
• /auto_review - the model reviews the whole change and each file on its own, sorts findings into the same seven categories, and ends with an orangu approves/rejects this patch verdict. Per-file depth cycles Normal → Deep → Ignore, and a rigorous rubric combined with confidence scoring (0-100) filters out false positives and pedantic nitpicks, so only high-confidence findings are flagged
• /create_patch - turns the findings of the last /review or /auto_review into changes in the working tree: the model verifies each finding against the current code, implements the valid fixes, updates tests, runs focused validation, and reports what it left alone. It never commits or pushes. The same command resolves and stages the conflicts a stopped merge, rebase, or cherry-pick leaves behind
• Share the result - /export review writes a PDF to the workspace root, and /comment 42 with auto review posts the report onto a GitHub or GitLab issue or pull request. No gh or glab needed

Use Cases

• Privacy-first development - Your code never leaves your workspace
• Air-gapped and regulated environments - A tightly integrated review workflow with no code sent to the cloud
• Cost control - No per-token cloud fees; run the models you already host
• One supplier for the whole stack - Environment, coordinator, and inference engine from a single project, with no integration to own yourself
• Repository-aware coding - Tools understand your project structure and git history
• Codebase exploration - Navigate unfamiliar code with the knowledge graph and semantic search
• Unattended, repeatable work - Describe a job once in a workflow file and run it across every project that needs it, with a code-and-review loop that keeps going until the goal is verified
• A model of your own - Train a small, domain-specific model from permissively-licensed sources with orangu-gguf, and serve the result from the same stack
• Offline development - No internet connection required after initial setup

Installation

The one-liner installer puts the whole stack - orangu, orangu-coordinator, and orangu-server - on the machine in a single step, on Linux, macOS, and Windows:

curl -fsSL https://mnemosyne-systems.github.io/orangu/install.sh | sh

Platforms

orangu is available for Linux, macOS, and Windows on x86_64 and aarch64, with source and checksums published alongside the binaries. orangu-server runs on CPU, Vulkan, Metal, CUDA, ROCm, OpenCL, and NPU backends.

Open Source

orangu is licensed under the GNU General Public License v3.0 , and that applies to the entire stack - the environment, the coordinator, the inference engine, and orangu-gguf. There is no open core, no proprietary engine behind the Open Source front end, and no feature held back for a paid tier.

Latest release

orangu 2.0.0 is out - a new architecture for the coordinator and the server with massive performance improvements, and workers that spread one model over several machines. This release also adds initial video and speech generation, the Anthropic Claude API, improved PDF support, a container image, and support for DeepSeek 4.1, Mistral 4, Muse Spark, MiniMax 3.1, FLUX.2 [klein], and Gemma Embedding 2.

• Announcement
• Release notes

Web site

• Web site

Documentation

• Getting Started Guide
• Manual
• Configuration
• Inference server
• Coordinator
• Workflows
• Building a model
• Image generation
• Video generation
• Speech generation
• Workers
• Container image
• Releases and Downloads

GitHub

• GitHub repository

Commercial support

• Contact sales