magnitudedev/magnitudePublic

What is magnitude?

An open-source inference server that seamlessly connects AI coding agents to locally hosted models

What does magnitude do?

Magnitude is a localized inference server designed to bridge the gap between popular AI coding agents (like Claude Code, OpenCode, and Pi) and locally hosted language models. It acts as a smart middleware that profiles the user's host hardware, recommends the optimal models that fit within local memory constraints, and automatically handles downloading and tuning. Once running, it exposes a standardized endpoint that existing agent software can plug into seamlessly. This enables developers to use powerful, automated agent tools entirely offline and privately, without relying on costly or privacy-invasive cloud APIs.

Who is magnitude for?

Magnitude is built for software engineers, security-conscious organizations, and AI enthusiasts who want to leverage modern AI coding agents using their own hardware. It is ideal for users with capable local machines who prioritize data privacy and zero-cost inference.

How do I get started with magnitude?

npm i -g @magnitudedev/cli

How popular is magnitude on GitHub?

magnitudedev/magnitude has 6,274 stars and 425 forks on GitHub, and gained 1,172 stars in the last 7 days.

What license does magnitude use?

magnitudedev/magnitude is released under the Apache-2.0 license.

• Breakout launch

  6,274 stars in 114 days

• Rising fast

  +1,172 stars this week

• Very active

  979 commits in 52 weeks

• Permissive license

  Apache-2.0

• Continuous integration

  Automated checks passing

• Repeat trending

  16 trending appearances

Magnitude is a localized inference server designed to bridge the gap between popular AI coding agents (like Claude Code, OpenCode, and Pi) and locally hosted language models. It acts as a smart middleware that profiles the user's host hardware, recommends the optimal models that fit within local memory constraints, and automatically handles downloading and tuning. Once running, it exposes a standardized endpoint that existing agent software can plug into seamlessly. This enables developers to use powerful, automated agent tools entirely offline and privately, without relying on costly or privacy-invasive cloud APIs.

Magnitude is built for software engineers, security-conscious organizations, and AI enthusiasts who want to leverage modern AI coding agents using their own hardware. It is ideal for users with capable local machines who prioritize data privacy and zero-cost inference.

• Hardware profiling: automatically analyzes system resources to recommend models that fit within available RAM and VRAM.
• Seamless agent integration: provides drop-in compatibility with popular coding agents like Claude Code, OpenClaw, and Hermes.
• Automated model management: handles the downloading, tuning, and execution of local models without manual configuration.
• Offline execution: ensures all inference and data processing stays entirely on the local machine for maximum privacy.
• Standardized API endpoint: exposes a uniform interface that mimics cloud providers, allowing existing tools to connect without modification.

Getting started: npm i -g @magnitudedev/cli

1. @magnitudedev/cli@0.2.4@magnitudedev/cli@0.2.4Oct 2, 20263.1K downloads

   ## 0.2.4 ### Patch Changes - [`f0498ce`](https://github.com/magnitudedev/magnitude/commit/f0498ce285e4e97815ba16dd74044ce5efebb63a) Thanks [@anerli](https://github.com/anerli)! - - Keep first-load kernel tuning within its minute for large models too, such as Gemma 4 26B: preparing each kernel's test data now counts against the same time, is done once instead of twice, and is skipped for kernels whose share of the minute cannot cover it, which keep their default configuration. - [`e38af0e`](https://github.com/magnitudedev/magnitude/commit/e38af0e9a232293dca12c5bad3a91b96e58ca12e) Thanks [@thrgreenwald](https://github.com/thrgreenwald)! - - Fix models failing to load on M1 and M2 Macs when a kernel's default configuration needs more threads than the chip allows for it. Tuning now starts from the nearest configuration that runs. - [`cef7f3b`](https://github.com/magnitudedev/magnitude/commit/cef7f3bb9c2dd06d96aeb59dde8039b6f83386a5) Thanks [@thrgreenwald](https://github.com/thrgreenwald)! - - Fix requests that fail partway through, for example when memory runs short during a long conversation, returning an empty reply that looked like success. They now return a 503 with `Retry-Afte

2. @magnitudedev/cli@0.2.3@magnitudedev/cli@0.2.3Oct 1, 20264K downloads

   ## 0.2.3 ### Patch Changes - [`5b1aba8`](https://github.com/magnitudedev/magnitude/commit/5b1aba81185ac8b07ba363ba2a11427c3447d62a) Thanks [@thrgreenwald](https://github.com/thrgreenwald)! - - Fix the Windows installer failing on Windows 10 with "An interrupted installation could not be recovered (code 4395)". Setup now installs normally and publishes the `magnitude` command to PATH, which also failed on Windows 10 once installation got past that error.

3. @magnitudedev/cli@0.2.2@magnitudedev/cli@0.2.2Oct 1, 2026364 downloads

   ## 0.2.2 ### Patch Changes - [`1588451`](https://github.com/magnitudedev/magnitude/commit/1588451a88f979a70c1f6385599d50de029064f7) Thanks [@anerli](https://github.com/anerli)! - - Fix the app getting stuck on "Assessing models" on some hardware: model speed is now estimated from the device's memory bandwidth instead of running kernels on the GPU, which could hang or fail. - [`3940408`](https://github.com/magnitudedev/magnitude/commit/394040878bd6cf10bf5a9addd9a3791d8eb2ef9f) Thanks [@anerli](https://github.com/anerli)! - - Fix Codex hanging after its first tool call over the Responses WebSocket: follow-up requests now continue from the previous response's output, and request errors end the request instead of leaving it waiting. - Fix requests that repeat the same image, in one message or across turns, failing with a 400. A repeated image is now encoded once. - Fix tool call IDs repeating across turns (every turn's first call was `call_0`), which made Claude Code drop tool calls and loop. - Fix Anthropic token usage counting cached tokens twice in responses and reporting none when streaming, and `count_tokens` requiring `max_tokens`. - Fix large system prompts being re-r

4. @magnitudedev/cli@0.2.1@magnitudedev/cli@0.2.1Sep 30, 20261.5K downloads

   ## 0.2.1 ### Patch Changes - [`0476b71`](https://github.com/magnitudedev/magnitude/commit/0476b71a0fb772245328529312a3392d77b630aa) Thanks [@anerli](https://github.com/anerli)! - - Fix the local model hanging forever when a request arrived while another was generating, as with Qwen3.6 35B-A3B: every later request waited without a response while the model still reported Ready, and the engine held a CPU core at 100%. Admission no longer waits on the running generation, and the engine can no longer wait on work only it could release. - Fix speculative-decoding models failing mid-request with "only a blocked last page is relocated" during long prompts.

5. @magnitudedev/cli@0.2.0@magnitudedev/cli@0.2.0Sep 30, 2026346 downloads

   ## 0.2.0 ### Minor Changes - [`f0cd67e`](https://github.com/magnitudedev/magnitude/commit/f0cd67ed900fe76022273081490be0939908df70) Thanks [@anerli](https://github.com/anerli)! - Replace the llama.cpp-based inference engine with Magnitude's own engine which automatically optimizes itself for any hardware and has efficient kernels for several open-weight model families. ### Patch Changes - [`9b929cf`](https://github.com/magnitudedev/magnitude/commit/9b929cfef434742e0f43043dfbf212f7801d5ba5) Thanks [@anerli](https://github.com/anerli)! - - Fix models that load and run, such as Gemma 4 26B-A4B and Qwen3.6 35B-A3B on Apple Silicon, being reported as unable to run on this computer. - A model is reported as unsupported only when Magnitude cannot actually run it. A gap in the device's speed measurements now shows "Speed estimate unavailable" instead of hiding the model. - [`482e6ab`](https://github.com/magnitudedev/magnitude/commit/482e6abbb94f4081c19e223fef09263ac559df14) Thanks [@anerli](https://github.com/anerli)! - - Make DFlash, DSpark, and DFlash2 speculative decoding faster than plain decoding on Apple Silicon (Qwen3.6-35B-A3B at 65k tokens: 10.7% faster, previously 7% slow

• ultraworkers/claw-code

  An agent-managed museum exhibit, built in Rust with Gajae-Code / LazyCodex — developed and maintained with no human intervention.

  195.2K stars · Rust

• farion1231/cc-switch

  A cross-platform desktop All-in-One assistant for Claude Code, Codex, OpenCode, OpenClaw, Grok Build & Hermes Agent. Only official website: ccswitch.io

  140K stars · Rust

• openai/codex

  Lightweight coding agent that runs in your terminal

  127.8K stars · Rust

• denoland/deno

  A modern runtime for JavaScript and TypeScript.

  108.6K stars · Rust

• ruvnet/RuView

  π RuView turns commodity WiFi signals into real-time spatial intelligence, vital sign monitoring, and presence detection — all without a single pixel of video.

  96.4K stars · Rust

• oven-sh/bun

  Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one

  96.1K stars · Rust