lucasacchi/LogWhispererAI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
29 Commits 1 Branch 0 Tags
3d24dfdeaf8997f3c766ea614c77935897d357d8
Commit Graph

29 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Luca Sacchi Ricciardi
3d24dfdeaf feat: complete Sprint 2 with Telegram notifications integration 
Add Telegram Bot notification node to n8n workflow:

New Features:
- Telegram notification node for critical severity logs
- Italian message template with emoji and MarkdownV2 formatting
- Smart routing: Telegram only for critical logs
- Error handling: continueOnFail prevents workflow interruption
- Environment-based configuration (TELEGRAM_CHAT_ID)

Message Template Includes:
- 🚨 Alert header with severity
- 📍 Server hostname identification
- 📝 AI-generated problem summary
- 💡 Suggested bash command in code block
- ⚠️ Confirmation required flag
- 📝 Additional notes from AI
- 📊 AI processing status
- 🤖 Model used (openai/gpt-4o-mini)
- ⏰ Localized Italian timestamp

Workflow Flow:
Webhook → HMAC → Validation → PostgreSQL → OpenRouter → Critical? → Telegram → Response
                                                      ↓ FALSE
                                               Success Response

Configuration Required:
1. Create Telegram Bot via @BotFather
2. Get TELEGRAM_BOT_TOKEN
3. Get TELEGRAM_CHAT_ID via @userinfobot
4. Configure credentials in n8n UI
5. Set TELEGRAM_CHAT_ID environment variable

Documentation:
- docs/telegram_setup.md: Setup instructions
- .env.example: Environment variables template
- .gitignore: Protect sensitive telegram_setup.md
- docs/prd.md: Updated Sprint 2 completion status

Sprint 2 Complete:
✅ Secure log ingestion (bash)
✅ n8n webhook workflow
✅ OpenRouter AI integration
✅ PostgreSQL storage
✅ HMAC authentication
✅ Telegram notifications

Refs: docs/specs/ai_pipeline.md, docs/specs/bash_ingestion_secure.md
 2026-04-02 19:47:57 +02:00
Luca Sacchi Ricciardi
5aab19626f feat: implement OpenRouter AI integration in n8n workflow 
Add 'Call OpenRouter' node to LogWhisperer_Ingest workflow:

New Node Features:
- Model: openai/gpt-4o-mini via OpenRouter API
- System prompt with Metodo Sacchi (Safety First, Little Often, Double Check)
- Timeout: 10 seconds with AbortController
- Log truncation: max 2000 characters
- Required headers: Authorization, HTTP-Referer, X-Title
- Error handling with graceful fallback response
- Output: JSON with ai_analysis, ai_status, ai_timestamp, ai_model

Workflow Flow:
Webhook → HMAC Validation → Data Validation → Store Log → Call OpenRouter → Critical Severity Check → Response

Test Suite (workflows/test_openrouter.js):
- 10 comprehensive tests covering:
  - Input/output structure validation
  - Log truncation logic
  - OpenRouter API payload format
  - Required HTTP headers
  - AI response structure
  - Fallback error handling
  - Timeout configuration
  - Dangerous command patterns
  - System Prompt Metodo Sacchi validation
  - Workflow connections

Environment Variables Required:
- OPENROUTER_API_KEY
- OPENROUTER_SITE_URL (optional, defaults to https://logwhisperer.ai)
- OPENROUTER_APP_NAME (optional, defaults to LogWhispererAI)

Next Steps:
1. Configure environment variables in n8n
2. Import updated workflow to n8n instance
3. Configure PostgreSQL credentials
4. Test with sample log payload

Refs: docs/specs/ai_pipeline.md (section 4.1)
 2026-04-02 19:40:42 +02:00
Luca Sacchi Ricciardi
d0f530f757 docs: update AI Pipeline spec to use OpenRouter instead of OpenAI direct 
Migrated from OpenAI direct API to OpenRouter for multiple advantages:

Cost Optimization:
- Reduced cost by ~25%: /bin/bash.00015/call (vs /bin/bash.0002/call OpenAI)
- Monthly estimate: ~/bin/bash.15 for 1000 calls (vs /bin/bash.20)

Provider Benefits:
- Access to 300+ AI models (OpenAI, Anthropic, Google, etc.)
- Automatic fallback between providers for 99.9% uptime
- Single API key for all providers
- Unified API interface

Technical Changes:
- Updated endpoint: api.openai.com → openrouter.ai/api/v1
- Model format: gpt-4o-mini → openai/gpt-4o-mini
- Added required headers: HTTP-Referer, X-Title
- Environment variables: OPENAI_API_KEY → OPENROUTER_API_KEY
  + OPENROUTER_SITE_URL (for OpenRouter ranking)
  + OPENROUTER_APP_NAME

Added Sections:
- 3.1 Why OpenRouter? (benefits comparison)
- 3.5 Model Fallback (automatic provider switching)
- Updated all code examples with OpenRouter integration
- Updated test cases (openai_client → openrouter_client)
- Updated cost estimates throughout document

References updated to OpenRouter documentation.

Refs: OpenRouter docs https://openrouter.ai/docs
 2026-04-02 19:31:04 +02:00
Luca Sacchi Ricciardi
96ba01b349 docs: add AI Pipeline technical specification 
Create comprehensive spec for Sprint 2 Feature 2:

Content:
- System prompt with Metodo Sacchi integration (Safety First, Little Often, Double Check)
- OpenAI GPT-4o-mini configuration (cost: ~/bin/bash.0002/call)
- Complete n8n Code Node JavaScript implementation
- JSON output schema with severity mapping
- Error handling with circuit breaker pattern
- Security guidelines (data sanitization, rate limiting)
- 10 acceptance criteria defined
- 5 test scenarios with expected input/output
- 5 real-world examples (OOM, disk full, connection refused, etc.)
- Implementation checklist for developers

Key Features:
- Safe command generation (destructive commands blacklisted)
- Incremental actions (Little Often principle)
- Confirmation required for uncertain situations (Double Check)
- Fallback when OpenAI unavailable
- Response time < 5s target
- Cost-optimized with GPT-4o-mini

Next: @python-developer implements OpenAI integration in n8n workflow

Refs: workflows/logwhisperer_ingest.json (already created)
 2026-04-02 19:15:02 +02:00
Luca Sacchi Ricciardi
3c406ef405 feat: create n8n workflow for secure log ingestion 
Implement LogWhisperer_Ingest workflow for Sprint 2 Feature 2:

Workflow Components:
- Webhook trigger: POST /webhook/logwhisperer/ingest
- HMAC-SHA256 validation with timing-safe comparison
- Anti-replay protection (5min timestamp window)
- Data validation: UUID client_id, severity levels, non-empty logs
- PostgreSQL storage with logs table auto-creation
- Conditional routing for critical severity logs

Security Features:
- HMAC signature verification (X-LogWhisperer-Signature header)
- Timestamp validation preventing replay attacks
- Input sanitization before DB insert
- Environment variable LOGWHISPERER_SECRET for shared secret

Documentation:
- workflows/logwhisperer_ingest.json: Export JSON workflow
- workflows/README.md: Installation and usage guide
- workflows/INTEGRATION.md: Bash script integration guide
- workflows/REPORT.md: Implementation report
- workflows/test_workflow.sh: Automated test suite

Metodo Sacchi Applied:
- Safety First: HMAC validation before any processing
- Little Often: Modular nodes, each with single responsibility
- Double Check: Test suite validates all security requirements

Next Steps:
- Configure LOGWHISPERER_SECRET in n8n environment
- Import workflow to n8n instance
- Test end-to-end with secure_logwhisperer.sh
 2026-04-02 19:01:40 +02:00
Luca Sacchi Ricciardi
9de40fde2d feat: implement secure bash log ingestion script (Sprint 2) 
Implement secure_logwhisperer.sh resolving HIGH severity vulnerabilities:

Security Features:
- Path traversal prevention: validate_log_source() enforces /var/log/ only
- Command injection protection: no eval, array-based commands
- JSON injection fix: jq-based encoding (no manual escaping)
- DLP masking: passwords, emails, API keys, IPs redacted
- HMAC-SHA256 webhook authentication with timestamps
- Atomic file operations preventing race conditions
- HTTPS enforcement for webhook URLs

New Functions:
- validate_log_source(): whitelist /var/log paths, symlink validation
- sanitize_log_line(): DLP + control char removal + truncation
- encode_json_payload(): safe JSON via jq
- generate_hmac_signature(): HMAC-SHA256 for auth
- atomic_write_offset(): tmp+mv atomic writes
- dispatch_webhook_secure(): authenticated HTTPS POST

CLI Commands:
--validate-source, --sanitize-line, --check-deps
--validate-config, --generate-hmac, --atomic-write
--read-offset, --encode-json

Test Results:
- 27/27 security tests passing
- 4/4 integration tests skipped (require webhook)
- All SEC-* requirements met

Documentation:
- Technical spec in docs/specs/bash_ingestion_secure.md
- Test suite in tests/test_secure_logwhisperer.py (31 tests)

Security Audit: Passes all OWASP guidelines
Breaking Changes: Requires jq, openssl dependencies
 2026-04-02 18:52:02 +02:00
Luca Sacchi Ricciardi
3a455e48d2 docs: update documentation with MCP configuration and agent catalog 
- Add MCP servers documentation (n8n, context7, sequential-thinking)
- Update README.md with complete project structure and requirements.txt
- Transform agents.md into comprehensive agent staff catalog (9 agents)
- Update CHANGELOG.md with [Unreleased] MCP entries
- Fix ingestion_script.md acceptance criteria checkboxes
- Add .opencode/opencode.json to .gitignore for security
- Include new agent configs: n8n_specialist_agent, context_auditor_agent
- Include new skill playbooks: n8n_automation, context7_documentation

Security: API credentials in .opencode/opencode.json are now gitignored
 2026-04-02 18:21:21 +02:00
Luca Sacchi Ricciardi
9205c4ef32 docs: update git-history.md with Sprint 1 Project Review 
- Add latest commit 88cfe9a to recent commits section

- Update Version 0.1.1 entry with Project Review details

- Update cronologia completa table with new commit

- Update Sprint 1 status to Completed and Approved

- Update statistics: 21 total commits

- Add Project Review reference and Go/No-Go decision
 2026-04-02 17:27:47 +02:00
Luca Sacchi Ricciardi
88cfe9af50 docs: update all documentation for Sprint 1 completion 
- Update README.md with badges, project status, and improved structure
- Update CHANGELOG.md with Project Review Sprint 1 and version 0.1.1
- Update PRD status to reflect Sprint 1 completion
- Update ingestion script spec status to Completed with review link
- Update Sprint 1 verification report with Project Review reference
- Add comprehensive Sprint 1 Project Review document

Refs: Project Review conducted by agent staff (Product Manager, Tech Lead, Security Auditor)
 2026-04-02 17:25:29 +02:00
Luca Sacchi Ricciardi
844294f7b2 docs: update changelog with AGENTS.md documentation entry 2026-04-02 17:04:11 +02:00
Luca Sacchi Ricciardi
b565895df1 docs: update AGENTS.md with complete agent rules and workflow 2026-04-02 16:53:12 +02:00
Luca Sacchi Ricciardi
ffb396a5a5 refactor: reorganize agent configuration structure 
- Create individual agent config files in .opencode/agents/
- Add detailed configurations for all 7 agents:
  * tech-lead.md - Architecture validation
  * product-manager.md - Value alignment
  * python-developer.md - TDD Python implementation
  * bash-expert.md - Bash scripting specialist
  * security-auditor.md - Security auditing
  * qa-engineer.md - Testing and coverage
  * documentation-agent.md - Documentation maintenance
- Simplify and optimize docs/1.setup_procedure/agents.md
- Add workflow diagram and structure overview
- Reference detailed configs from main agents.md
 2026-04-02 16:48:37 +02:00
Luca Sacchi Ricciardi
8c37e4bc50 docs: add agent staff configuration document 
- Define 7 AI agent roles and responsibilities
- Document tools and focus areas for each agent
- Include operational workflow guidelines
- Configure for Spec-Driven and Safety First workflow
 2026-04-02 16:40:56 +02:00
Luca Sacchi Ricciardi
a510d5fc68 docs: add git history tracking document 
- Create comprehensive git history documentation
- Track all commits with dates, authors, and types
- Include sprint history section
- Add statistics and update instructions
- Maintainable format for future updates
 2026-04-02 16:30:44 +02:00
Luca Sacchi Ricciardi
702f50cc30 chore: add project configuration and agent skills 
- Add requirements.txt with pytest and requests dependencies
- Add TDD Python Specialist skill for test-driven workflow
- Add Git & Changelog skill for conventional commits
- Configure .opencode/skills/ directory structure
 2026-04-02 16:27:30 +02:00
Luca Sacchi Ricciardi
7def30702f docs: add Sprint 1 verification report 
- Create comprehensive verification document for Sprint 1
- Document test results (12/12 tests passed)
- Include acceptance criteria checklist
- Add security audit and code quality assessment
- Update CHANGELOG with verification report entry
 2026-04-02 16:26:36 +02:00
Luca Sacchi Ricciardi
69f475ec78 feat(ingestion): implement log monitoring script with webhook integration 
- Add logwhisperer.sh script for tailing and monitoring system logs
- Implement pattern matching for critical errors (FATAL, ERROR, OOM, segfault)
- Add JSON payload generation with severity levels
- Implement rate limiting and offset tracking per log source
- Add install.sh with interactive configuration and systemd support
- Create comprehensive test suite with pytest
- Add technical specification documentation
- Update CHANGELOG.md following Common Changelog standard

All 12 tests passing. Follows Metodo Sacchi (Safety first, little often, double check).
 2026-04-02 16:10:20 +02:00
lucasacchi
34dbba1201 Update README.md 
chores
 2026-04-02 15:45:39 +02:00
lucasacchi
40abdd5348 Update README.md 
chores
 2026-04-02 15:44:52 +02:00
lucasacchi
d907738403 Update README.md 
chores
 2026-04-02 15:43:53 +02:00
lucasacchi
c669273aeb Update README.md 
chores
 2026-04-02 15:43:00 +02:00
lucasacchi
7b2b7fcc82 Update README.md 
chores
 2026-04-02 15:42:20 +02:00
lucasacchi
ed6185b2e4 Update README.md 
refactor on readme.md
 2026-04-02 15:42:06 +02:00
Luca Sacchi Ricciardi
adfdec41a6 chores on README.md 2026-04-02 15:40:14 +02:00
lucasacchi
7a3f08499b Update LICENSE.md 
added legal infos
 2026-04-02 15:29:41 +02:00
Luca Sacchi Ricciardi
7da30cbc93 added license.md 2026-04-02 15:27:42 +02:00
Luca Sacchi Ricciardi
df75e51478 added venv, created and documented setup's procedure, added agents.md 2026-04-02 15:25:52 +02:00
Luca Sacchi Ricciardi
e1fc873700 chores, add prompt for project management. 2026-04-02 15:11:24 +02:00
Luca Sacchi Ricciardi
1cf076ba6d added prd.md 2026-04-02 15:07:05 +02:00