3c406ef405
Implement LogWhisperer_Ingest workflow for Sprint 2 Feature 2: Workflow Components: - Webhook trigger: POST /webhook/logwhisperer/ingest - HMAC-SHA256 validation with timing-safe comparison - Anti-replay protection (5min timestamp window) - Data validation: UUID client_id, severity levels, non-empty logs - PostgreSQL storage with logs table auto-creation - Conditional routing for critical severity logs Security Features: - HMAC signature verification (X-LogWhisperer-Signature header) - Timestamp validation preventing replay attacks - Input sanitization before DB insert - Environment variable LOGWHISPERER_SECRET for shared secret Documentation: - workflows/logwhisperer_ingest.json: Export JSON workflow - workflows/README.md: Installation and usage guide - workflows/INTEGRATION.md: Bash script integration guide - workflows/REPORT.md: Implementation report - workflows/test_workflow.sh: Automated test suite Metodo Sacchi Applied: - Safety First: HMAC validation before any processing - Little Often: Modular nodes, each with single responsibility - Double Check: Test suite validates all security requirements Next Steps: - Configure LOGWHISPERER_SECRET in n8n environment - Import workflow to n8n instance - Test end-to-end with secure_logwhisperer.sh
187 lines
5.2 KiB
Markdown
187 lines
5.2 KiB
Markdown
# LogWhisperer AI - Workflow n8n
|
|
|
|
Workflow per l'ingestion sicura dei log con validazione HMAC-SHA256.
|
|
|
|
## 📋 Descrizione Workflow
|
|
|
|
Il workflow `LogWhisperer_Ingest` riceve i log dal client `secure_logwhisperer.sh`, valida l'autenticazione HMAC, e li memorizza in PostgreSQL.
|
|
|
|
### Nodi del Workflow
|
|
|
|
1. **Webhook Trigger** - Riceve POST su `/webhook/logwhisperer/ingest`
|
|
2. **HMAC Validation** - Verifica firma HMAC-SHA256
|
|
3. **HMAC Valid?** - Condizione: HMAC valido?
|
|
4. **Data Validation** - Validazione campi obbligatori
|
|
5. **Store Log** - Inserimento in PostgreSQL
|
|
6. **Critical Severity?** - Condizione: severity = critical?
|
|
7. **AI Processing** - Placeholder per elaborazione AI (Sprint 3)
|
|
8. **Responses** - Risposte HTTP (200, 400, 401)
|
|
|
|
## 🚀 Installazione
|
|
|
|
### Prerequisiti
|
|
|
|
- n8n installato e accessibile (http://192.168.254.12:5678)
|
|
- PostgreSQL con credenziali configurate in n8n
|
|
- Variabile ambiente `LOGWHISPERER_SECRET` configurata
|
|
|
|
### Step 1: Configurare Variabile Ambiente
|
|
|
|
```bash
|
|
# SSH nel container n8n o imposta via n8n UI
|
|
export LOGWHISPERER_SECRET="your-32-char-secret-here-minimum"
|
|
```
|
|
|
|
### Step 2: Importare il Workflow
|
|
|
|
**Metodo A: Via n8n UI**
|
|
|
|
1. Accedi a http://192.168.254.12:5678
|
|
2. Vai su **Workflows** → **Import from File**
|
|
3. Seleziona `workflows/logwhisperer_ingest.json`
|
|
4. Clicca **Save**
|
|
|
|
**Metodo B: Via API (curl)**
|
|
|
|
```bash
|
|
# Ottieni API key da n8n UI → Settings → API
|
|
curl -X POST http://192.168.254.12:5678/api/v1/workflows \
|
|
-H "Content-Type: application/json" \
|
|
-H "X-N8N-API-KEY: your-api-key" \
|
|
-d @workflows/logwhisperer_ingest.json
|
|
```
|
|
|
|
### Step 3: Configurare Credenziali PostgreSQL
|
|
|
|
1. In n8n UI, vai su **Settings** → **Credentials**
|
|
2. Crea nuova credenziale **PostgreSQL**
|
|
3. Nome: `PostgreSQL LogWhisperer`
|
|
4. Inserisci host, port, database, user, password
|
|
5. Clicca **Save**
|
|
|
|
### Step 4: Attivare il Workflow
|
|
|
|
1. Apri il workflow `LogWhisperer_Ingest`
|
|
2. Clicca **Activate** (toggle in alto a destra)
|
|
3. Verifica che lo stato sia **Active**
|
|
|
|
## 🧪 Testing
|
|
|
|
### Esegui Test Suite
|
|
|
|
```bash
|
|
cd /home/google/Sources/LucaSacchiNet/LogWhispererAI
|
|
./workflows/test_workflow.sh
|
|
```
|
|
|
|
### Test Manuale con curl
|
|
|
|
**Test HMAC Valido:**
|
|
|
|
```bash
|
|
# Genera HMAC
|
|
TIMESTAMP=$(date +%s)
|
|
PAYLOAD='{"client_id":"550e8400-e29b-41d4-a716-446655440000","hostname":"test-server","source":"/var/log/syslog","severity":"critical","raw_log":"Apr 2 10:30:00 kernel: Out of memory","matched_pattern":"OOM"}'
|
|
SIGNATURE=$(printf '%s:%s' "$TIMESTAMP" "$PAYLOAD" | openssl dgst -sha256 -hmac "test-secret-32-chars-long-minimum" | sed 's/^.* //')
|
|
|
|
# Invia richiesta
|
|
curl -X POST http://192.168.254.12:5678/webhook/logwhisperer/ingest \
|
|
-H "Content-Type: application/json" \
|
|
-H "X-LogWhisperer-Signature: ${TIMESTAMP}:${SIGNATURE}" \
|
|
-H "X-LogWhisperer-Timestamp: ${TIMESTAMP}" \
|
|
-d "$PAYLOAD"
|
|
```
|
|
|
|
**Test HMAC Invalido (deve ritornare 401):**
|
|
|
|
```bash
|
|
curl -X POST http://192.168.254.12:5678/webhook/logwhisperer/ingest \
|
|
-H "Content-Type: application/json" \
|
|
-H "X-LogWhisperer-Signature: invalid-signature" \
|
|
-H "X-LogWhisperer-Timestamp: $(date +%s)" \
|
|
-d '{"client_id":"test","severity":"critical","raw_log":"test"}'
|
|
```
|
|
|
|
## 🔒 Sicurezza
|
|
|
|
### Validazione HMAC
|
|
|
|
- Algoritmo: HMAC-SHA256
|
|
- Formato: `timestamp:signature`
|
|
- Anti-replay: Timestamp max 5 minuti di differenza
|
|
- Timing-safe comparison
|
|
|
|
### Validazione Dati
|
|
|
|
- `client_id`: UUID v4 obbligatorio
|
|
- `raw_log`: Non vuoto
|
|
- `severity`: Uno di `low`, `medium`, `critical`
|
|
|
|
### Note Sicurezza
|
|
|
|
- Non loggare `raw_log` completo nei nodi AI
|
|
- Usare HTTPS in produzione
|
|
- Rate limiting: max 10 req/min per client_id
|
|
- `LOGWHISPERER_SECRET` minimo 32 caratteri
|
|
|
|
## 📊 Schema Database
|
|
|
|
```sql
|
|
CREATE TABLE logs (
|
|
id SERIAL PRIMARY KEY,
|
|
client_id VARCHAR(36) NOT NULL,
|
|
hostname VARCHAR(255),
|
|
source VARCHAR(500),
|
|
severity VARCHAR(20) CHECK (severity IN ('low', 'medium', 'critical')),
|
|
timestamp TIMESTAMP WITH TIME ZONE,
|
|
raw_log TEXT,
|
|
matched_pattern VARCHAR(100),
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
);
|
|
|
|
CREATE INDEX idx_logs_client_id ON logs(client_id);
|
|
CREATE INDEX idx_logs_severity ON logs(severity);
|
|
CREATE INDEX idx_logs_timestamp ON logs(timestamp);
|
|
```
|
|
|
|
## 🔧 Troubleshooting
|
|
|
|
### Workflow non risponde
|
|
|
|
1. Verifica che n8n sia attivo: `curl http://192.168.254.12:5678/healthz`
|
|
2. Controlla che il workflow sia attivato
|
|
3. Verifica i log di n8n: `docker logs n8n`
|
|
|
|
### Errore 401 Unauthorized
|
|
|
|
- Verifica che `LOGWHISPERER_SECRET` sia configurato
|
|
- Controlla formato signature: `timestamp:signature`
|
|
- Verifica che timestamp sia recente (< 5 min)
|
|
|
|
### Errore 400 Bad Request
|
|
|
|
- Verifica formato UUID per `client_id`
|
|
- Controlla che `severity` sia valido
|
|
- Assicurati che `raw_log` non sia vuoto
|
|
|
|
### Errore Database
|
|
|
|
- Verifica credenziali PostgreSQL in n8n
|
|
- Controlla che la tabella sia stata creata
|
|
- Verifica permessi utente database
|
|
|
|
## 📝 Changelog
|
|
|
|
### 2026-04-02 - Sprint 2 Feature 2
|
|
|
|
- Creazione workflow iniziale
|
|
- Implementazione validazione HMAC
|
|
- Integrazione PostgreSQL
|
|
- Conditional alerting per severity critical
|
|
|
|
## 🛡️ Metodo Sacchi Applied
|
|
|
|
- **Safety first**: Validazione HMAC prima di ogni operazione
|
|
- **Little often**: Un nodo per funzione, testabili individualmente
|
|
- **Double check**: Validazione dati dopo HMAC, verifica salvataggio DB
|