feat: add support for local LLM providers (Ollama & LM Studio)
Some checks failed
CI / test (3.10) (push) Has been cancelled
CI / test (3.11) (push) Has been cancelled
CI / test (3.12) (push) Has been cancelled
CI / lint (push) Has been cancelled

Implement local LLM inference support for Ollama and LM Studio:

New Clients:
- OllamaClient: Interface to Ollama API (default: localhost:11434)
- LMStudioClient: Interface to LM Studio API (default: localhost:1234)

Factory Updates:
- Added OLLAMA and LMSTUDIO to LLMProvider enum
- Updated create_client() to instantiate local clients
- Updated list_available_providers() with is_local flag

Configuration:
- Added ollama_base_url and lmstudio_base_url settings
- Local providers return configured for API key check

Tests:
- Comprehensive test suite (250+ lines)
- Tests for client initialization and invocation
- Factory integration tests

Documentation:
- Added LLM Providers section to SKILL.md
- Documented setup for Ollama and LM Studio
- Added usage examples and configuration guide

Usage:
  provider: ollama, model: llama3.2
  provider: lmstudio, model: local-model
This commit is contained in:
Luca Sacchi Ricciardi
2026-04-06 18:28:21 +02:00
parent 13c9bd5029
commit 0b33cd1619
4 changed files with 549 additions and 5 deletions

154
SKILL.md
View File

@@ -602,7 +602,113 @@ curl http://localhost:8000/api/v1/notebooks -H "X-API-Key: your-key"
---
**Skill Version:** 1.2.0
## LLM Providers
DocuMente supporta molteplici provider LLM, inclusi quelli locali tramite **Ollama** e **LM Studio**.
### Provider Cloud
| Provider | API Key Richiesta | Default Model |
|----------|------------------|---------------|
| **OpenAI** | ✅ `OPENAI_API_KEY` | gpt-4o-mini |
| **Anthropic** | ✅ `ANTHROPIC_API_KEY` | claude-3-sonnet |
| **Google** | ✅ `GOOGLE_API_KEY` | gemini-pro |
| **Mistral** | ✅ `MISTRAL_API_KEY` | mistral-medium |
| **Azure** | ✅ `AZURE_API_KEY` | gpt-4 |
| **OpenRouter** | ✅ `OPENROUTER_API_KEY` | openai/gpt-4o-mini |
| **Z.AI** | ✅ `ZAI_API_KEY` | zai-large |
| **OpenCode Zen** | ✅ `OPENCODE_ZEN_API_KEY` | zen-1 |
### Provider Locali
| Provider | URL Default | Configurazione |
|----------|-------------|----------------|
| **Ollama** | http://localhost:11434 | `OLLAMA_BASE_URL` |
| **LM Studio** | http://localhost:1234 | `LMSTUDIO_BASE_URL` |
#### Setup Ollama
```bash
# 1. Installa Ollama
# macOS/Linux
curl -fsSL https://ollama.com/install.sh | sh
# 2. Scarica un modello
ollama pull llama3.2
ollama pull mistral
ollama pull qwen2.5
# 3. Avvia Ollama (in un terminale separato)
ollama serve
# 4. Verifica che sia in esecuzione
curl http://localhost:11434/api/tags
```
**Uso con DocuMente:**
```bash
# Query con Ollama
curl -X POST http://localhost:8000/api/v1/query \
-H "Content-Type: application/json" \
-d '{
"question": "Spiega l\'intelligenza artificiale",
"provider": "ollama",
"model": "llama3.2"
}'
```
#### Setup LM Studio
```bash
# 1. Scarica LM Studio da https://lmstudio.ai/
# 2. Avvia LM Studio e carica un modello
# 3. Attiva il server locale (Settings > Local Server)
# Default URL: http://localhost:1234
# 4. Verifica che sia in esecuzione
curl http://localhost:1234/v1/models
```
**Uso con DocuMente:**
```bash
# Query con LM Studio
curl -X POST http://localhost:8000/api/v1/query \
-H "Content-Type: application/json" \
-d '{
"question": "Cosa sono i notebook?",
"provider": "lmstudio",
"model": "local-model"
}'
```
#### Configurazione URL Personalizzato
Per usare Ollama/LM Studio su un'altra macchina nella rete:
```env
# .env
OLLAMA_BASE_URL=http://192.168.1.100:11434
LMSTUDIO_BASE_URL=http://192.168.1.50:1234
```
#### Vantaggi dei Provider Locali
- 🔒 **Privacy**: I dati non lasciano il tuo computer/rete
- 💰 **Gratuito**: Nessun costo per API call
-**Offline**: Funziona senza connessione internet
- 🔧 **Controllo**: Scegli tu quali modelli usare
#### Limitazioni
- Richiedono hardware adeguato (RAM, GPU consigliata)
- I modelli locali sono generalmente meno potenti di GPT-4/Claude
- Tempo di risposta più lungo su hardware consumer
---
**Skill Version:** 1.3.0
**API Version:** v1
**Last Updated:** 2026-04-06
@@ -667,3 +773,49 @@ curl http://localhost:8000/api/v1/notebooks -H "X-API-Key: your-key"
- ✅ Created docs/integration.md with full guide
- ✅ Updated SKILL.md with new capabilities
- ✅ API examples and best practices
---
## Changelog Sprint 3
### 2026-04-06 - Local LLM Providers (Ollama & LM Studio)
**Implemented:**
-`OllamaClient` - Support for Ollama local inference
-`LMStudioClient` - Support for LM Studio local inference
- ✅ Added `ollama` and `lmstudio` to `LLMProvider` enum
- ✅ Updated `LLMClientFactory` to create local provider clients
- ✅ Added configuration options `OLLAMA_BASE_URL` and `LMSTUDIO_BASE_URL`
- ✅ Local providers marked with `is_local: true` in provider list
**Features:**
- OpenAI-compatible API endpoints (/v1/chat/completions)
- Configurable base URLs for network deployments
- Longer timeouts (120s) for local inference
- No API key required for local providers
- Support for all Ollama models (llama3.2, mistral, qwen, etc.)
- Support for any model loaded in LM Studio
**Configuration:**
```env
# Optional: Custom URLs
OLLAMA_BASE_URL=http://localhost:11434
LMSTUDIO_BASE_URL=http://localhost:1234
```
**Usage:**
```bash
curl -X POST http://localhost:8000/api/v1/query \
-H "Content-Type: application/json" \
-d '{
"question": "Explain AI",
"provider": "ollama",
"model": "llama3.2"
}'
```
**Tests:**
- ✅ 250+ lines of tests for local providers
- ✅ Unit tests for OllamaClient and LMStudioClient
- ✅ Integration tests for factory creation
- ✅ Configuration tests