Welcome to the Local LLM Server Manager (v3.17.0) User Guide. This document provides a complete guide to operating the dashboard, configuring local AI engines, and using generative studio tools.
The desktop application organizes capabilities into dedicated workspaces using a collapsible Activity Rail (56 px collapsed / 200 px expanded) on the left flank and a compact 34 px Telemetry Ribbon in the titlebar:
flowchart TD
App["Local LLM Server Manager (Port 5246)"]
App --> Rail["Activity Rail\n(56px collapsed / 200px expanded)"]
Rail --> Dom1["⚡ Studio\nSticker Studio, Images, Video, Audio & 3D Mesh"]
Rail --> Dom2["📦 Models\nInstalled Models, Hugging Face Hub & CivitAI"]
Rail --> Dom3["💻 Hardware Fit\nCan I Run It Sizing & Memory Offloading"]
Rail --> Dom4["⚙️ Settings\nAuto-Discovery, Daemon Ports & Feature Packs"]
Rail -.-> CompL["📖 Documentation\n(Companion Window / Drawer)"]
Rail -.-> CompR["🤖 AI Assist\n(Companion Window / Drawer)"]
The Models workspace monitors active hardware metrics and manages local model weights.
- Live VRAM Bar: Displays total, used, and free GPU memory in real time via NVML CUDA telemetry.
- Model Capability Badges: Identifies model capabilities (e.g.,
Coding & General,Reasoning,Math). - Interactive KV Cache Estimator: Drag the context length slider (up to 32,768 tokens) to preview memory consumption before loading models.
- VRAM Orchestrator: Automatically frees GPU memory before heavy diffusion or 3D tasks start.
- Unload All VRAM Button: Releases all active models from GPU memory with a single click.
Tip
Read the complete Engines & VRAM Guide and Ollama Engine Guide.
Download models directly without opening a web browser or using terminal commands.
Search community repositories, compare quantization levels (Q4_K_M, Q8_0), filter by input/output modalities, and stream downloads to disk.
Search diffusion checkpoints, LoRA style adapters, and VAE models with real-time download counters and hardware compatibility badges.
Tip
Read the Model Hubs & Downloads Guide and LoRA Art Styles Guide.
The Can I Run It workspace estimates whether an AI model fits within your system memory before downloading files.
- Live GPU Detection: Queries your graphics card and system memory automatically.
- Multi-Modality Sizing: Calculates memory consumption for Text LLMs, Diffusion Images, Video, Audio, and 3D Mesh models.
- Visual Allocation Bar: Color-coded breakdown of Model Weights, Context/KV Cache, CUDA Overhead, and Free Headroom.
- Layer Offloading Calculation: Predicts the exact number of transformer layers that fit in GPU VRAM versus CPU RAM.
- Performance Throughput: Provides real-time token per second estimates for your hardware.
Tip
Read the dedicated Can I Run It Hardware Fit Guide.
The Workflows workspace provides generation pipelines across five creative modalities:
| Modality | Supported Models | Output Formats | Dedicated Guide |
|---|---|---|---|
| Sticker Studio | SDXL, SD 1.5, FLUX | PNG (32-bit Transparent) | Sticker Studio Guide |
| Image Generation | FLUX.1, SDXL, SD 1.5 | PNG, WebP | Image Generation Guide |
| Video Generation | Wan 2.2, LTX-Video 2.5, HunyuanVideo | MP4 | Video Generation Guide |
| Audio & Speech | Kokoro TTS, Stable Audio Open, YuE | WAV, MP3 | Audio & Music Guide |
| 3D Mesh | TRELLIS V2, Hunyuan3D v2 | GLB, OBJ | 3D Mesh Guide |
The Sticker Studio creates die-cut vinyl stickers from image references or text descriptions:
- Click ⚡ Studio in the Activity Rail and select Sticker Studio.
- Drop a reference image (PNG, JPG, WebP) into the Input Deck drop zone, or type a custom prompt.
- Select an art style preset chip (Die-Cut Vinyl, Holographic, Chibi Anime, 80s Retro, Pop Art, or Watercolor).
- Verify that Auto-Cutout is checked to remove backgrounds automatically.
- Set the Die-Cut Border Width slider (0–24 px).
- Click ✨ Generate Sticker.
- Monitor progress through the 4-stage pipeline tracker:
Diffusion Generation➔Subject Isolation➔Contour Application➔Ready.
- Click 📋 Copy PNG to copy the 32-bit transparent image to the clipboard, or click 💾 Save File to write to disk.
Tip
Read the complete Sticker Studio Guide for full style token references and VRAM requirements.
Before starting complex renders, use the Test Flight control panel in the studio header:
- Select your target modality (Text, Image, Video, or Audio).
- Choose a starter prompt.
- Click 🚀 Launch Test Flight.
- The system validates network readiness and GPU memory in seconds.
Tip
Read the Real Engine Test Flight Guide.
The AI Assistant workspace provides interactive guidance, screenshot diagnostics, and app control without consuming local GPU memory.
flowchart LR
LeftCompanion["Documentation Window\n(Left Flank)"] <-->|Magnetic Proximity Snap| MainWindow["Main Dashboard Window\n(Center)"]
MainWindow <-->|Magnetic Proximity Snap| RightCompanion["AI Assist Window\n(Right Flank)"]
- External Gateway: Connects to LiteLLM or Vertex AI Gemini Flash to keep local GPU memory free for generation.
- Multimodal Attachments: Paste screenshots (Ctrl+V) to diagnose ComfyUI errors or review outputs.
- Magnetic Companion Windows: Detach the assistant into a floating window that docks magnetically to the right flank.
- Lockstep Movement: Moving the main window moves docked companion windows automatically.
Tip
Read the AI Chat Assistant Guide and Magnetic Companion Windows Guide.
The Settings workspace centralizes engine paths, port bindings, and optional component management.
- Auto-Detect Installed Tools: Scans system drives to locate Ollama, ComfyUI, Forge, and Kokoro TTS automatically.
- Modular Feature Packs: Install or remove
ext_videoandext_audiopackages on demand. - Network Endpoints: Displays auto-detected LAN IP addresses and remote MCP connection URLs.
Tip
Read the First-Time Configuration Guide and Remote Access Guide.
- Getting Started:
- Engines & Models:
- Multimodal Studio:
- AI & MCP Automation:
- Technical Reference: