Osmantic/ODSPublic

ODS installs and wires together everything you need to run AI locally, so you do not have to assemble Ollama, Open WebUI, n8n, ComfyUI, and privacy tools by hand:

• Local model inference — run open models on your own hardware
• ChatGPT-style web UI — talk to your models from any browser
• Control dashboard — manage models, services, setup, GPU status, and extensions from one place
• Voice, agents, and workflows — build automations that can listen, speak, call tools, and get work done
• RAG and search — connect local documents, private search, and retrieval workflows
• Image generation — run local image tools without sending prompts to a hosted API
• Privacy and ops — keep service auth, secrets, observability, and diagnostics in one local stack

No cloud required. No subscriptions required. Your prompts and data stay on your machine unless you choose otherwise. Cloud and hybrid API modes are optional when you want them.

Release validation: Operational changes are checked with a release-grade fleet and distro lab: zero-prereq bootstrap, fresh installs, product flows, full-model capabilities, lifecycle recovery, and the final User Green gate. See Release Validation for what a green run proves.

ODS V3 Pre-Release: V3 is in public testing and refinement ahead of its official launch. Try it, share feedback, and help us improve the experience. The pinned source snapshot (v3.0.0) is available for reproducibility. Full fleet qualification is incomplete; see the V3 Pre-Release notes and the promotion record for known task limitations, available evidence, and remaining release gates.

Repo layout: the repository root holds the public README, installers, security policy, GitHub workflows, and project coordination docs. The ods/ directory is the product runtime: services, installer phases, compose overlays, dashboard, CLI, tests, and operator docs.

Release consumption: v3.0.0 is the latest published source release, presented as V3 Pre-Release during public testing and refinement. Its GitHub Latest designation does not mark the official V3 launch or establish full fleet qualification. main continues receiving fixes; pin a tag or audited commit and retain its validation receipt when reproducibility matters. V3 fixes land on main; release/2.6.x is the older 2.6 maintenance lane. See Release Channels, Installer Trust, and Forkability.

September main update: the quickstarts below follow development main, including the September Portal/platform promotion. That merge is not a new stable release or proof of complete fleet qualification. Native Pixel source-update and backup/recovery limits are documented in Source Updates. Use a pinned release or audited commit when reproducibility is required.

Release verification is being qualified: the commands below still use development main; they do not provide signed-release provenance. The verified installer preview is kept separate until a signed immutable release passes end-to-end testing. This security update does not switch the public installation channel prematurely.

Choose your system, copy the block, run it in a normal terminal. ODS installs the stack, picks a model for your hardware, starts the services, and gives you the local web UI.

Linux or macOS

curl -fsSL https://install.osmantic.com/ods.sh | bash

Windows PowerShell — guided Ubuntu/WSL2 setup with Pixel/Portal

$ProgressPreference = "SilentlyContinue"
$odsSrc = Join-Path $env:TEMP ("ods-install-" + [guid]::NewGuid().ToString("N"))
$odsZip = Join-Path $odsSrc "ods-main.zip"
New-Item -ItemType Directory -Path $odsSrc | Out-Null
Invoke-WebRequest "https://github.com/Osmantic/ODS/archive/refs/heads/main.zip" -OutFile $odsZip
Expand-Archive -LiteralPath $odsZip -DestinationPath $odsSrc -Force
cd (Get-ChildItem -LiteralPath $odsSrc -Directory | Select-Object -First 1).FullName
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1

Linux and macOS: Docker must be installed and running.

Windows: open a normal PowerShell window (not "Run as administrator"), paste the block, and answer the prompts. Nothing else needs to be installed first. The installer:

1. Checks free disk space (40 GB) and that hardware virtualization is on.
2. Offers to enable WSL2 and install Docker Desktop with winget. Windows asks for administrator permission, then one restart; setup continues by itself after you sign in again.
3. Offers to download Ubuntu 24.04 and asks you, in PowerShell, for a new Ubuntu username and password.
4. Starts Docker Desktop and checks that it is connected to Ubuntu. If not, it shows the one setting to turn on in Docker Desktop and continues as soon as it works.
5. Installs ODS inside Ubuntu with --pixel --no-hermes --no-openclaw. When Ubuntu asks for your [sudo] password, type the Ubuntu password; nothing appears while you type.
6. Verifies Pixel and Portal, then opens Portal in your browser and adds an ODS Portal shortcut to your desktop.

Each step asks before changing anything and stops with instructions if it cannot finish; rerun the same command after fixing it. There is no fallback to Hermes or the native Windows installer. On NVIDIA machines, update the Windows driver to 570 or newer first. To use an existing distribution, add -Distro <name> (names from wsl -l -v).

Existing native Windows installations are not automatically migrated or deleted; see Windows Quickstart before switching.

If another device runs your ODS model gateway and this Windows PC only needs image generation, use the standalone ComfyUI installer. It keeps its Docker project and data separate from a full ODS installation.

The hosted Linux/macOS endpoint proxies the current bootstrap from repository main. Reviewed merges reach it automatically after edge-cache refresh. ODS_REF selects a compatible repository checkout. See Installer Trust to inspect the script or install a stable release or audited commit manually.

Windows users should not run the curl ... | bash command from PowerShell. The PowerShell block above downloads the public ODS source ZIP and delegates installation to Ubuntu/WSL2. For more detail, see the Windows Quickstart.

After the installer completes successfully, Portal opens at http://localhost:3001/pixel (the Windows installer opens it for you and prints the exact URL). http://localhost:3000 is Open WebUI, a separate interface. Verify that Portal is available and send a message; a loaded dashboard alone does not prove Pixel is ready. If installation fails or Portal is degraded, follow the Windows Quickstart checks before proceeding.

WSL GPU access must be checked separately. NVIDIA needs a supported Windows driver and GPU access inside WSL/Docker. On AMD, Windows setup runs Lemonade through an ODS task bound to the selected WSL installation. Once that ownership is verified, Dashboard Models supports compatible GGUF downloads (including Hugging Face), activation, context changes, and unload/resume. An independently configured Lemonade service remains externally managed. Older ODS tasks without the installation binding require an installer rerun; see the Windows Quickstart and WSL2 GPU guide.

For Linux, macOS, or the recommended Windows/WSL installation, uninstall from the matching Linux/macOS terminal (open Ubuntu on Windows):

cd ~/ods
./ods-uninstall.sh --force

For a native Windows installation only:

$installDir = "$env:USERPROFILE\ods"
cd $installDir
.\ods.ps1 uninstall --force

Windows recovery note: if the runtime folder is partial and .\ods.ps1 is missing, run the same command from a source checkout as .\ods\installers\windows\ods.ps1 uninstall --force. It verifies the containers' Compose installation directory before removing resources. A shared ods project name does not authorize removing another Windows or WSL installation. Unattached volumes with no verifiable owner are preserved, with an error naming the resource; --force does not bypass this check.

API endpoint: Linux Docker installs expose llama-server on http://localhost:11434 by default (OLLAMA_PORT) while containers use llama-server:8080. macOS native Metal and Windows native/Lemonade paths use http://localhost:8080 unless overridden. Open WebUI stays on http://localhost:3000.

No GPU? ODS also runs in cloud mode — same full stack, powered by OpenAI/Anthropic/Together APIs instead of local inference:

./install.sh --cloud

Port conflicts? Every port is configurable via environment variables. See .env.example for the full list, or override at install time:

WEBUI_PORT=9090 ./install.sh

New here? Read the Friendly Guide or listen to the audio version — a complete walkthrough of what ODS is, how it works, and how to make it your own. No technical background needed.

Current Platform Support

Platform	Status
Linux (NVIDIA + AMD Strix Halo)	Supported — see the hardware and distro limits in the support matrix
Linux + Intel Arc (SYCL)	Experimental / Tier C — validation is hardware-specific
Windows (NVIDIA + AMD)	Supported — install and run today
macOS (Apple Silicon)	Supported — install and run today

Tested Linux distros: Ubuntu 26.04/24.04/22.04, Debian 12, Linux Mint 21.3, Fedora 41+, Rocky Linux 9, Arch Linux, Manjaro, CachyOS, and openSUSE Tumbleweed. Other distros using apt, dnf, pacman, or zypper should also work — open an issue if yours doesn't.

Release validation: Operational changes run through a release-grade gate that covers zero-prereq bootstrap, clean installs, product behavior, full-model capabilities, lifecycle recovery, and User Green. See Release Validation and the Validation Matrix.

Windows: Requires Docker Desktop with WSL2 backend. NVIDIA GPUs use Docker GPU passthrough; AMD Strix Halo runs through the platform-specific accelerated path documented in the Windows installer and support matrix.

macOS: Requires Apple Silicon (M1+) and Docker Desktop. llama-server uses native Metal acceleration; Portal's gateway and managed host helpers also run natively. The UI, ingress, sandbox and supporting services run in Docker. See the macOS Quickstart.

See the Support Matrix for supported platform claims and the Validation Matrix for the layered test surface used to test those claims.

A handful of companies control the vast majority of global AI traffic — and with it, your data, your costs, and your uptime. Every query you send to a centralized provider is business intelligence you don’t own, running on infrastructure you don’t control, priced on terms you can’t negotiate.

If AI is becoming critical infrastructure, it shouldn’t be rented. Self-hosting local AI should be a sovereign human right, not a career choice.

Because running your own AI shouldn't require a CS degree and a weekend of debugging CUDA drivers. Right now, setting up local AI means stitching together a dozen projects, writing Docker configs from scratch, and praying everything talks to each other. Most people give up and go back to paying OpenAI.

We built ODS so you don't have to.

• One command — detects your GPU, picks the right model, generates credentials, launches everything
• Chatting in under 2 minutes — bootstrap mode gives you a working model instantly while your full model downloads in the background
• Full service stack, pre-wired — chat, agents, voice, workflows, search, RAG, image generation, privacy tools, observability, and developer tools. All talking to each other out of the box
• Fully moddable — every service is an extension. Drop in a folder, run ods enable, done

git clone https://github.com/Osmantic/ODS.git
cd ODS/ods
./install.sh

$ProgressPreference = "SilentlyContinue"
$odsSrc = Join-Path $env:TEMP ("ods-install-" + [guid]::NewGuid().ToString("N"))
$odsZip = Join-Path $odsSrc "ods-main.zip"
New-Item -ItemType Directory -Path $odsSrc | Out-Null
Invoke-WebRequest "https://github.com/Osmantic/ODS/archive/refs/heads/main.zip" -OutFile $odsZip
Expand-Archive -LiteralPath $odsZip -DestinationPath $odsSrc -Force
cd (Get-ChildItem -LiteralPath $odsSrc -Directory | Select-Object -First 1).FullName
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1

git clone https://github.com/Osmantic/ODS.git
cd ODS/ods
./install.sh

• Open WebUI — full-featured chat interface with conversation history, web search, document upload, and 30+ languages
• llama-server — high-performance LLM inference with continuous batching, auto-selected for your GPU; Linux Docker host API defaults to localhost:11434, native macOS/Windows paths use localhost:8080, and container API runs on 8080
• LiteLLM — API gateway supporting local/cloud/hybrid modes
• TEI Embeddings — text embedding service for RAG and search workflows

• Whisper — speech-to-text
• Kokoro — text-to-speech

• Portal — bundled core conversational assistant on Apple Silicon macOS and qualified Ubuntu 24.04/26.04 or Debian 12 systemd hosts, including qualified WSL2 installations through the Linux installer. No private repository access or separate license flag is required; available in the Dashboard and through a compatible Open WebUI model route. The native PowerShell installer does not install the Portal host runtime.
• Hermes Agent — independent general-purpose agent, available alongside Portal; includes memory, skills, and a proxy with optional owner-card gating; direct access by default
• OpenClaw — deprecated legacy autonomous agent, still opt-in during the migration window
• n8n — workflow automation with 400+ integrations (Slack, email, databases, APIs)
• APE — Agent Policy Engine for auditing and governing autonomous tool calls
• OpenCode — browser-based AI coding assistant wired to the local stack
• Memory Shepherd — host/systemd helper for agent memory lifecycle management

• Qdrant — vector database for retrieval-augmented generation (RAG)
• SearXNG — self-hosted web search (no tracking)
• Perplexica — deep research engine
• Brave Search — optional paid Brave Search API integration

• ComfyUI — node-based image generation

• Privacy Shield — PII scrubbing proxy for API calls
• Dashboard — real-time GPU metrics, service health, model management
• Dashboard API — service health, setup, status, metrics, and management API behind the dashboard
• Token Spy — token usage monitor for local and proxied LLM traffic
• Langfuse — optional LLM observability and tracing

The installer detects your GPU and first assigns a deterministic hardware tier. Linux and macOS then run the versioned catalog selector (ods/scripts/select-model.py), while Windows uses the PowerShell catalog selector in ods/installers/windows/lib/tier-map.ps1; both read ods/config/model-library.json to choose the best installable GGUF for the detected memory envelope. The final choice is written to .env as LLM_MODEL, GGUF_FILE, MAX_CONTEXT, and MODEL_RECOMMENDATION_*.

MODEL_PROFILE=qwen is the default non-Gemma catalog profile, so the effective pick can be Qwen, Phi, or DeepSeek depending on what fits best. MODEL_PROFILE=gemma4 forces Gemma 4 where available, and MODEL_PROFILE=auto uses Gemma 4 on NVIDIA, Apple Silicon, and Intel Arc tiers. Override tier selection with ./install.sh --tier 3; override the model family with MODEL_PROFILE=gemma4 ./install.sh or MODEL_PROFILE=auto ./install.sh.

When the Hermes fallback is enabled, installers keep the first-run bootstrap model at a 64K context floor and promote the full local model context to 128K where the selected model supports it. That avoids Hermes's hard 64K minimum while preserving the under-2-minute first chat experience. The examples below are current catalog-selector outputs for common hardware envelopes; exact installs can differ with detected VRAM/RAM, host architecture, existing downloads, or explicit profile overrides. Throughput still needs a local benchmark after first launch.

The selector routes unified-memory hosts away from Qwen3 Coder Next when that model would otherwise be selected, because current repo policy documents correctness issues on those backends.

Gemma 4 profile tiers remain in the installer tier maps: E2B on entry hardware, E4B on midrange hardware, 26B-A4B on pro hardware, and 31B on large/ultra hardware.

No waiting for large downloads. ODS uses bootstrap mode by default:

1. Downloads a tiny 1.5B model in under a minute
2. You start chatting immediately
3. The full model downloads in the background
4. Hot-swap to the full model when it's ready — zero downtime

The bootstrap model starts with a 64K context window so Hermes can work during the first session. After the background download finishes, ODS swaps to the full model and restores the Hermes/full-model context target.

Skip bootstrap: ./install.sh --no-bootstrap

The installer picks a model for your hardware, but you can switch anytime:

ods model current              # What's running now?
ods model list                 # Show all available tiers
ods model swap T3              # Switch to a different tier

If the new model isn't downloaded yet, pre-fetch it first:

./scripts/pre-download.sh --tier 3    # Download before switching
ods model swap T3                    # Then swap (restarts llama-server)

Already have a GGUF you want to use? Drop the single .gguf file in data/models/, then open Dashboard -> Models and load the local entry. For older installs or headless maintenance, update GGUF_FILE and LLM_MODEL in .env, then restart with the CLI:

ods restart llm

Or restart the container directly from the installed ods directory:

docker compose restart llama-server

Rollback is automatic — if a new model fails to load, ODS reverts to your previous model.

ODS is designed to be modded. Every service is an extension — a folder with a manifest.yaml and a compose.yaml. The dashboard, CLI, health checks, and compose stack all discover extensions automatically.

extensions/services/
  my-service/
    manifest.yaml      # Metadata: name, port, health endpoint, GPU backends
    compose.yaml       # Docker Compose fragment (auto-merged into the stack)

ods enable my-service     # Enable it
ods disable my-service    # Disable it
ods list                  # See everything

The installer itself is modular — 19 library modules, a shared service registry, and 13 ordered phases. Want to add a hardware tier, swap a default model, or skip a phase? Start with the installer architecture map so you update the Linux, macOS, Windows, upgrade, and host-agent writers together.

Full extension guide | Installer architecture

The ods CLI manages your entire stack:

ods status                # Health checks + GPU status
ods list                  # All services and their state
ods logs llm              # Tail logs (aliases: llm, stt, tts)
ods restart [service]     # Restart one or all services
ods start / stop          # Start or stop the stack

ods mode cloud            # Switch to cloud APIs via LiteLLM
ods mode local            # Switch back to local inference
ods mode hybrid           # Local primary, cloud fallback

ods model swap T3         # Switch to a different hardware tier
ods enable n8n            # Enable an extension
ods disable whisper       # Disable one

ods config show           # View .env (secrets masked)
ods preset save gaming    # Snapshot current config
ods preset load gaming    # Restore it

Other tools get you part of the way. ODS gets you the whole way.

ODS is built by a growing group of contributors across installers, GPU support, dashboard, security, extensions, docs, and release validation. The README keeps the product overview focused; the long-form credits, upstream acknowledgements, and contributor history live in CONTRIBUTORS.md.

ODS has been recognized by the local AI and developer community, including AMD Featured Developer recognition, selection as a May 2026 AMD Lemonade Developer Challenge winner, and a feature at (Co)nnect: Philly's AI Ecosystem Summit at Pennovation Works.

ODS code is Apache-2.0 except the bundled Pixel source, which has a separate ODS-only use and distribution grant. See Licensing, LICENSE, and Pixel's license.