Local LLM Setup Guide

Trade-offs

• Slower than cloud APIs (5-10x longer processing)
• Requires decent hardware (GPU with 8GB+ VRAM recommended)

Ollama is the easiest way to run local LLMs. It just works, runs as a lightweight background service, and takes up almost no resources until you actually need it.

Step 1: Download and Install

1. Visit ollama.com
2. Download Ollama for Windows
3. Install - Ollama starts automatically as a background service

Step 2: Download a Model

Option A: Install from Loki Studio (Easiest)

1. Open Loki Studio
2. Go to Settings > Application Settings > LLM Providers
3. Select Ollama as provider
4. Click Install next to any model (llama3, mistral, qwen, etc.)
5. Wait for download to complete (progress shown)
6. Click Refresh to see installed models

Option B: Install via Command Line

Open Command Prompt or PowerShell and run:

# Best balance of quality and speed (8GB VRAM)
ollama pull llama3:8b

# Faster, smaller footprint (4GB VRAM)
ollama pull llama3.2:3b

# Good alternative (8GB VRAM)
ollama pull mistral:7b

Step 3: Configure Loki Studio

1. Open Loki Studio
2. Go to Settings > Application Settings > LLM Providers
3. Set AI Provider to Ollama
4. Select your model from the dropdown (auto-detected)
5. Settings auto-save!

That's it! Ollama runs automatically in the background whenever you need it.

LM Studio is a desktop application with a nice UI and model browser. It gives you more control over model settings.

Step 1: Download and Install

1. Visit lmstudio.ai
2. Download LM Studio for Windows
3. Install and launch the application

Step 2: Download a Model

1. Open LM Studio
2. Go to the Search tab (magnifying glass icon)
3. Search for and download a model:
   • Qwen2.5-7B-Instruct - Best balance (~8GB VRAM)
   • Llama-3.2-3B-Instruct - Faster (~4GB VRAM)
   • Mistral-7B-Instruct - Good alternative (~8GB VRAM)
   • Phi-3-mini - Very fast (~2GB VRAM)
4. Wait for download to complete

Step 3: Start the Server (Every Time!)

1. Go to the Local Server tab (leftmost icon)
2. Select your downloaded model from the dropdown
3. Click Start Server
4. Keep LM Studio running while using Loki Studio

Server runs at: http://localhost:1234

Step 4: Configure Loki Studio

1. Open Loki Studio
2. Go to Application Settings
3. Set AI Provider to LM Studio
4. Set LM Studio Endpoint to http://localhost:1234
5. Select your model from the dropdown
6. Click Save Settings

Pro Tip

Process videos overnight with local LLMs. Set up a batch before bed and wake up to fully processed content - all at zero cost. After ~50-100 videos, local LLMs have paid for themselves!