LogWhispererAI/workflows/INTEGRATION.md

Guida Integrazione: Script Bash ↔ Workflow n8n

Questa guida descrive come integrare secure_logwhisperer.sh con il workflow n8n LogWhisperer_Ingest.

🔄 Flusso di Dati

┌─────────────────┐     HMAC-SHA256      ┌──────────────────┐
│ secure_logwhis- │ ───────────────────> │ Webhook n8n      │
│ perer.sh        │    POST /ingest      │ LogWhisperer_    │
│                 │                      │ Ingest           │
└─────────────────┘                      └──────────────────┘
                                                │
                                                ▼
                                        ┌──────────────────┐
                                        │ PostgreSQL       │
                                        │ Table: logs      │
                                        └──────────────────┘

⚙️ Configurazione

1. Configurare il Client (Script Bash)

Crea un file config.env nella directory del progetto:

# config.env
CLIENT_ID="550e8400-e29b-41d4-a716-446655440000"
CLIENT_SECRET="your-secret-32-chars-long-minimum-here"
WEBHOOK_URL="https://192.168.254.12:5678/webhook/logwhisperer/ingest"
MAX_LINE_LENGTH=2000
OFFSET_DIR="/var/lib/logwhisperer"

Requisiti:

• CLIENT_ID: UUID v4 valido
• CLIENT_SECRET: Minimo 32 caratteri, no spazi
• WEBHOOK_URL: Deve usare HTTPS in produzione

2. Configurare il Server (n8n)

Impostare Variabile Ambiente

Docker Compose:

services:
  n8n:
    image: n8nio/n8n
    environment:
      - LOGWHISPERER_SECRET=your-secret-32-chars-long-minimum-here
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=password

Docker Run:

docker run -d \
  --name n8n \
  -p 5678:5678 \
  -e LOGWHISPERER_SECRET="your-secret-32-chars-long-minimum-here" \
  -v ~/.n8n:/home/node/.n8n \
  n8nio/n8n

Configurare Credenziali PostgreSQL

1. Accedi a n8n UI: http://192.168.254.12:5678
2. Vai su Settings → Credentials
3. Clicca Add Credential
4. Seleziona PostgreSQL
5. Configura:
   • Name: PostgreSQL LogWhisperer
   • Host: postgres (o IP del tuo DB)
   • Port: 5432
   • Database: logwhisperer
   • User: logwhisperer
   • Password: your-password
   • SSL: disable (per test locale)

3. Verificare Segreto Condiviso

Il segreto DEVE essere identico su client e server:

Client (Bash):

# Genera firma di test
./scripts/secure_logwhisperer.sh --generate-hmac '{"test":"data"}' 'test-secret-32-chars-long-minimum' 1234567890
# Output: 1234567890:abc123...

Server (n8n Code Node):

Il nodo HMAC Validation calcola la firma con lo stesso algoritmo:

const expectedSignature = crypto
  .createHmac('sha256', secret)
  .update(`${timestamp}:${payload}`)
  .digest('hex');

🚀 Esempio di Uso Completo

Step 1: Validare Configurazione

cd /home/google/Sources/LucaSacchiNet/LogWhispererAI

# Verifica dipendenze
./scripts/secure_logwhisperer.sh --check-deps

# Valida configurazione
./scripts/secure_logwhisperer.sh --validate-config

Step 2: Test Ingezione Singola

# Sanitizza una linea di log
SANITIZED=$(./scripts/secure_logwhisperer.sh --sanitize-line "Apr 2 10:30:00 kernel: password=secret123 Out of memory")
echo "$SANITIZED"
# Output: Apr 2 10:30:00 kernel: password=*** Out of memory

# Genera payload JSON
PAYLOAD=$(./scripts/secure_logwhisperer.sh --encode-json '{
    "client_id": "550e8400-e29b-41d4-a716-446655440000",
    "hostname": "web-server-01",
    "source": "/var/log/syslog",
    "severity": "critical",
    "raw_log": "Apr 2 10:30:00 kernel: Out of memory",
    "matched_pattern": "OOM"
}')

# Genera firma HMAC
TIMESTAMP=$(date +%s)
SIGNATURE=$(./scripts/secure_logwhisperer.sh --generate-hmac "$PAYLOAD" "$CLIENT_SECRET" "$TIMESTAMP")

# Invia a n8n
curl -X POST "$WEBHOOK_URL" \
    -H "Content-Type: application/json" \
    -H "X-LogWhisperer-Signature: $SIGNATURE" \
    -H "X-LogWhisperer-Timestamp: $TIMESTAMP" \
    -d "$PAYLOAD"

Step 3: Verifica Salvataggio

# Connettiti al database PostgreSQL
psql -h localhost -U logwhisperer -d logwhisperer

# Query per verificare inserimento
SELECT * FROM logs ORDER BY created_at DESC LIMIT 5;

# Esci
\q

🔐 Sicurezza End-to-End

HMAC Signature Format

Header: X-LogWhisperer-Signature: <timestamp>:<signature>
Header: X-LogWhisperer-Timestamp: <timestamp>

Dove:
- timestamp: Unix epoch seconds
- signature: HMAC-SHA256(timestamp:payload, secret)

Esempio Calcolo HMAC (Bash)

#!/bin/bash

payload='{"client_id":"550e8400-e29b-41d4-a716-446655440000","severity":"critical","raw_log":"test"}'
timestamp=$(date +%s)
secret="test-secret-32-chars-long-minimum"

# Calcola HMAC
signature=$(printf '%s:%s' "$timestamp" "$payload" | \
    openssl dgst -sha256 -hmac "$secret" | \
    sed 's/^.* //')

echo "Timestamp: $timestamp"
echo "Signature: $signature"
echo "Full: ${timestamp}:${signature}"

Esempio Calcolo HMAC (JavaScript/n8n)

const crypto = require('crypto');

const payload = '{"client_id":"550e8400-e29b-41d4-a716-446655440000","severity":"critical","raw_log":"test"}';
const timestamp = Math.floor(Date.now() / 1000);
const secret = "test-secret-32-chars-long-minimum";

const signature = crypto
  .createHmac('sha256', secret)
  .update(`${timestamp}:${payload}`)
  .digest('hex');

console.log(`Timestamp: ${timestamp}`);
console.log(`Signature: ${signature}`);
console.log(`Full: ${timestamp}:${signature}`);

🧪 Test di Integrazione

Test 1: Validazione Completa

./workflows/test_workflow.sh

Test 2: Flusso Completo

# 1. Crea una linea di log di test
echo "Apr 2 12:00:00 kernel: FATAL: Out of memory: Kill process 1234" > /tmp/test_critical.log

# 2. Processa con secure_logwhisperer.sh
# (Assumendo che lo script legga da file e invii a webhook)
# TODO: Implementare modalità daemon nel prossimo sprint

# 3. Verifica in database
psql -h localhost -U logwhisperer -c "SELECT * FROM logs WHERE severity='critical' ORDER BY created_at DESC LIMIT 1;"

📊 Monitoraggio

Log n8n

# Visualizza log in tempo reale
docker logs -f n8n

# Cerca errori specifici
docker logs n8n 2>&1 | grep -i "logwhisperer\|error\|unauthorized"

Metriche

• Richieste totali: Conteggio righe in tabella logs
• Errori 401: Webhook chiamate rifiutate (HMAC invalido)
• Errori 400: Validazione dati fallita
• Latency: Tempo medio di risposta del webhook

🐛 Troubleshooting Comuni

"Invalid signature" (401)

Causa: Segreti diversi tra client e server

Soluzione:

# Verifica segreto sul client
echo "CLIENT_SECRET: $CLIENT_SECRET"

# Verifica segreto sul server (n8n container)
docker exec n8n echo "$LOGWHISPERER_SECRET"

# Devono essere identici!

"Request timestamp too old" (401)

Causa: Clock skew tra client e server

Soluzione:

# Sincronizza orario
sudo ntpdate pool.ntp.org

# O su container n8n
docker exec n8n date
docker exec n8n sh -c "date -s '@$(date +%s)'"

Database connection error

Causa: Credenziali PostgreSQL errate o database non raggiungibile

Soluzione:

# Test connessione
docker exec n8n pg_isready -h postgres -p 5432

# Verifica credenziali
docker exec n8n psql -h postgres -U logwhisperer -d logwhisperer -c "SELECT 1;"

🔄 Workflow CI/CD

Per test automatici in CI/CD:

# .github/workflows/integration-test.yml
name: Integration Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_DB: logwhisperer
          POSTGRES_USER: logwhisperer
          POSTGRES_PASSWORD: test
        ports:
          - 5432:5432
      n8n:
        image: n8nio/n8n
        env:
          LOGWHISPERER_SECRET: test-secret-32-chars-long-minimum
        ports:
          - 5678:5678
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Import Workflow
        run: |
          curl -X POST http://localhost:5678/api/v1/workflows \
            -H "Content-Type: application/json" \
            -d @workflows/logwhisperer_ingest.json
      
      - name: Run Tests
        run: ./workflows/test_workflow.sh

📝 Checklist Pre-Deploy

• CLIENT_SECRET configurato su client (min 32 chars)
• LOGWHISPERER_SECRET configurato su server (identico al client)
• Credenziali PostgreSQL configurate in n8n
• Workflow importato e attivato
• Tabella logs creata (automatizzato dal workflow)
• Test suite passati (./workflows/test_workflow.sh)
• HTTPS abilitato (in produzione)
• Rate limiting configurato
• Monitoring e alerting attivo