Commit Graph

23 Commits

Author SHA1 Message Date
Luca Sacchi Ricciardi
13c9bd5029 feat(frontend): add notebook selection and management UI
Some checks failed
CI / test (3.10) (push) Has been cancelled
CI / test (3.11) (push) Has been cancelled
CI / test (3.12) (push) Has been cancelled
CI / lint (push) Has been cancelled
Implement complete frontend integration for NotebookLM + RAG:

New Components:
- Notebooks.tsx: Full notebook management page
  - View indexed notebooks with stats
  - Sync available notebooks from NotebookLM
  - Delete notebook indexes
  - Tab-based interface (Indexed/Available)

Updated Components:
- Chat.tsx: Enhanced with notebook selector
  - Collapsible notebook selection panel
  - Multi-notebook selection with checkboxes
  - 'Include documents' toggle
  - Visual indicators for notebook sources in chat
  - Shows selected notebook count in header

New Store:
- notebookStore.ts: Zustand store for notebook state
  - Manage selected notebooks
  - Track sync status
  - Fetch indexed/available notebooks
  - Handle sync/delete operations

API Client Updates:
- Added notebook-related endpoints:
  - getNotebooks(), getIndexedNotebooks()
  - syncNotebook(), deleteNotebookIndex()
  - getSyncStatus(), queryWithNotebooks()

Type Definitions:
- Notebook, IndexedNotebook, SyncStatus
- SyncResponse, NotebookLMSource
- Extended Source types for notebook metadata

UI/UX:
- Added Notebooks to navigation menu
- Stats cards showing indexed/total sources/chunks
- Color-coded source badges (blue for notebooks)
- Warning when no notebooks indexed
- Loading states for all operations

Closes frontend integration requirement
2026-04-06 18:20:06 +02:00
Luca Sacchi Ricciardi
568489cae4 docs: comprehensive documentation for NotebookLM-RAG integration
Some checks failed
CI / test (3.10) (push) Has been cancelled
CI / test (3.11) (push) Has been cancelled
CI / test (3.12) (push) Has been cancelled
CI / lint (push) Has been cancelled
Update documentation to reflect new integration features:

README.md:
- Add 'Integrazione NotebookLM + RAG' section after Overview
- Update DocuMente component section with new endpoints
- Add notebooklm_sync.py and notebooklm_indexer.py to architecture
- Add integration API examples
- Add link to docs/integration.md

SKILL.md:
- Add RAG Integration to Capabilities table
- Update Autonomy Rules with new endpoints
- Add RAG Integration section to Quick Reference
- Add Sprint 2 changelog with integration features
- Update Skill Version to 1.2.0

docs/integration.md (NEW):
- Complete integration guide with architecture diagram
- API reference for all sync and query endpoints
- Usage examples and workflows
- Best practices and troubleshooting
- Performance considerations and limitations
- Roadmap for future features

All documentation now accurately reflects the unified
NotebookLM + RAG agent capabilities.
2026-04-06 18:01:50 +02:00
Luca Sacchi Ricciardi
a5029aef20 test: add comprehensive tests for NotebookLM-RAG integration
Some checks failed
CI / test (3.10) (push) Has been cancelled
CI / test (3.11) (push) Has been cancelled
CI / test (3.12) (push) Has been cancelled
CI / lint (push) Has been cancelled
Add test coverage for new integration components:

New Test Files:
- test_notebooklm_indexer.py: Unit tests for NotebookLMIndexerService
  * test_sync_notebook_success: Verify successful notebook sync
  * test_sync_notebook_not_found: Handle non-existent notebooks
  * test_extract_source_content_success/failure: Content extraction
  * test_delete_notebook_index_success/failure: Index management
  * test_end_to_end_sync_flow: Integration verification

- test_notebooklm_sync.py: API route tests
  * test_sync_notebook_endpoint: POST /notebooklm/sync/{id}
  * test_list_indexed_notebooks_endpoint: GET /notebooklm/indexed
  * test_delete_notebook_index_endpoint: DELETE /notebooklm/sync/{id}
  * test_get_sync_status_endpoint: GET /notebooklm/sync/{id}/status
  * test_query_with_notebook_ids: Query with notebook filters
  * test_query_notebooks_endpoint: POST /query/notebooks

All tests use mocking to avoid external dependencies.
2026-04-06 17:21:06 +02:00
Luca Sacchi Ricciardi
e3bacbc0a4 feat: integrate NotebookLM with RAG system
Some checks failed
CI / test (3.10) (push) Has been cancelled
CI / test (3.11) (push) Has been cancelled
CI / test (3.12) (push) Has been cancelled
CI / lint (push) Has been cancelled
Add complete integration between NotebookLM Agent and DocuMente RAG:

New Components:
- NotebookLMIndexerService: Syncs NotebookLM content to Qdrant vector store
- notebooklm_sync API routes: Manage notebook indexing (/api/v1/notebooklm/*)

Enhanced Components:
- RAGService: Added notebook_ids filter and query_notebooks() method
- VectorStoreService: Added filter support for metadata queries
- DocumentService: Added ingest_notebooklm_source() method
- Query routes: Added /query/notebooks endpoint for notebook-only queries
- Main API: Integrated new routes and updated to v2.1.0

Features:
- Sync NotebookLM notebooks to local vector store
- Query across documents and/or notebooks
- Filter RAG queries by specific notebook IDs
- Manage indexed notebooks (list, sync, delete)
- Track sync status and metadata

API Endpoints:
- POST /api/v1/notebooklm/sync/{notebook_id}
- GET /api/v1/notebooklm/indexed
- DELETE /api/v1/notebooklm/sync/{notebook_id}
- GET /api/v1/notebooklm/sync/{notebook_id}/status
- POST /api/v1/query/notebooks

Closes integration request for unified NotebookLM + RAG agent
2026-04-06 17:18:33 +02:00
Luca Sacchi Ricciardi
67ba5bc2dd docs: reorganize README to reflect dual-system architecture
Some checks failed
CI / test (3.10) (push) Has been cancelled
CI / test (3.11) (push) Has been cancelled
CI / test (3.12) (push) Has been cancelled
CI / lint (push) Has been cancelled
Restructure README to clearly present both systems:
- NotebookLM Agent: API for Google NotebookLM automation
- DocuMente: Multi-provider RAG system with web UI

Changes:
- Add detailed architecture diagrams for both systems
- Separate configuration and usage instructions
- Update project structure to show both src/notebooklm_agent and src/agentic_rag
- Add API examples for both systems
- Reorganize documentation section by system
2026-04-06 17:08:40 +02:00
Luca Sacchi Ricciardi
cfb6d9052a chore: add uploads folder to .gitignore
Some checks failed
CI / test (3.10) (push) Has been cancelled
CI / test (3.11) (push) Has been cancelled
CI / test (3.12) (push) Has been cancelled
CI / lint (push) Has been cancelled
- Remove test upload files from git tracking
- Add uploads/ directory to .gitignore
- Keep uploads folder local-only
2026-04-06 13:55:18 +02:00
Luca Sacchi Ricciardi
b7ef07dd34 refactor: rename project from AgenticRAG to DocuMente
## Changes
- Update all references from AgenticRAG to DocuMente
- Update README.md with new project description and structure
- Update LICENSE with new project name
- Update API title and descriptions in main.py
- Update frontend components (Layout, Login, Dashboard, Settings)
- Update static HTML page
- Update all documentation files (prd-v2.md, frontend-plan.md, etc.)
- Update test files with new project name
- Update docker-compose.yml, Dockerfile, requirements.txt

## SEO Benefits
- DocuMente combines 'Documento' and 'Mente' (Italian for Document and Mind)
- Memorable and brandable name
- Reflects the core functionality: AI-powered document intelligence

🎉 Project officially renamed to DocuMente!
2026-04-06 13:54:57 +02:00
Luca Sacchi Ricciardi
527adec599 chore(license): update to proprietary license
- Remove MIT License
- Add proprietary license with All Rights Reserved
- Update README with ⚖️ Licenza e Note Legali section
- Specify Luca Sacchi Ricciardi as copyright holder
- Designate Foro di Milano as exclusive jurisdiction

⚖️ All Rights Reserved - Luca Sacchi Ricciardi 2026
2026-04-06 13:19:44 +02:00
Luca Sacchi Ricciardi
0ee402b5fa docs: add comprehensive test coverage report
- Document 280 passing tests
- Coverage breakdown by module
- Explanation of uncovered API routes
- Path to 95% coverage calculation
2026-04-06 13:12:40 +02:00
Luca Sacchi Ricciardi
f6638d5406 test(agentic-rag): add comprehensive unit tests for core, services, and API
## Added
- conftest.py: Shared fixtures and mocks
- test_core/test_config.py: 35 tests for Settings
- test_core/test_logging.py: 15 tests for logging
- test_api/test_chat.py: 27 tests for chat endpoints
- test_api/test_health.py: 27 tests for health endpoints
- test_services/test_document_service.py: 38 tests
- test_services/test_rag_service.py: 66 tests
- test_services/test_vector_store.py: 32 tests

## Coverage
- auth.py: 100%
- config.py: 100%
- logging.py: 100%
- chat.py: 100%
- health.py: 100%
- document_service.py: 96%
- rag_service.py: 100%
- vector_store.py: 100%

Total: 240 tests passing, 64% coverage

🧪 Core functionality fully tested
2026-04-06 13:11:09 +02:00
Luca Sacchi Ricciardi
ee13751a72 docs(frontend): add comprehensive README with setup instructions
- Installation and development guide
- Project structure documentation
- API endpoints reference
- Troubleshooting section
2026-04-06 11:46:12 +02:00
Luca Sacchi Ricciardi
74cefd3366 feat(frontend): implement complete React/Vite frontend
## Features
- React 18 + TypeScript + Vite setup
- Tailwind CSS + shadcn/ui components
- Complete authentication flow (API Key)
- Dashboard with stats and recent documents
- Document upload (drag & drop) and management
- Chat interface with RAG support
- Settings page (theme, provider selection)
- Responsive design with mobile support

## Components
- Layout with sidebar navigation
- Button, Card, Input, Label, Separator (shadcn)
- Protected and public routes

## State Management
- Zustand stores: auth, chat, settings
- Persisted to localStorage

## API Integration
- Axios client with interceptors
- All API endpoints integrated
- Error handling and loading states

## Pages
- Login: API key authentication
- Dashboard: Overview and stats
- Documents: Upload, list, delete
- Chat: Conversational interface with sources
- Settings: Theme and provider config

🎨 Production-ready build (339KB gzipped)
2026-04-06 11:44:46 +02:00
Luca Sacchi Ricciardi
2566f712a2 docs: add comprehensive frontend development plan
## Added
- frontend-plan.md: Complete React/Vite frontend development plan

## Contents
- Project architecture and folder structure
- Tech stack (React 18, Vite, TypeScript, Tailwind, shadcn/ui)
- Design system with colors and components
- Page-by-page feature specifications
- API integration details
- 8-phase implementation roadmap (3-4 weeks)
- UI mockups for Dashboard and Chat
- Acceptance criteria

🎨 Ready for frontend development
2026-04-06 11:29:38 +02:00
Luca Sacchi Ricciardi
437a484b1c test(agentic-rag): add comprehensive unit tests for auth, llm_factory, and providers
## Added
- test_auth.py: 19 tests for JWT, API Key, password hashing, and auth flow
- test_llm_factory.py: 21 tests for all 8 LLM providers
- test_providers.py: API route tests for provider management

## Coverage
- Password hashing with bcrypt
- JWT token creation/validation/expiration
- API key verification (admin key)
- Dual-mode authentication (API key + JWT)
- Z.AI, OpenCode Zen, OpenRouter client implementations
- Factory pattern for all providers
- Client caching mechanism

All 40+ tests passing 
2026-04-06 11:27:39 +02:00
Luca Sacchi Ricciardi
f2408b2b88 feat(agentic-rag): add multi-provider LLM, auth, and Docker support
## Added
- Multi-provider LLM support with factory pattern (8 providers):
  * OpenAI, Z.AI, OpenCode Zen, OpenRouter, Anthropic, Google, Mistral, Azure
- Authentication system: JWT + API Key dual-mode
- Provider management API (/api/v1/providers)
- Docker containerization (Dockerfile + docker-compose.yml)
- Updated documentation in main.py

## Modified
- Documents API: added authentication
- Query API: support for provider/model selection
- RAG service: dynamic LLM provider selection
- Config: multi-provider settings

## Infrastructure
- Qdrant vector store integration
- Redis support (optional)
- Health check endpoints

🚀 Ready for production deployment
2026-04-06 11:22:26 +02:00
Luca Sacchi Ricciardi
2aa1f66227 feat: implement AgenticRAG system with datapizza-ai
Major refactoring from NotebookLM API to Agentic Retrieval System:

## New Features
- AgenticRAG backend powered by datapizza-ai framework
- Web interface for document upload and chat
- REST API with Swagger/OpenAPI documentation
- Document processing pipeline (Docling, chunking, embedding)
- Qdrant vector store integration
- Multi-provider LLM support (OpenAI, Google, Anthropic)

## New Components
- src/agentic_rag/api/ - FastAPI REST API
  - Documents API (upload, list, delete)
  - Query API (RAG queries)
  - Chat API (conversational interface)
  - Swagger UI at /api/docs
- src/agentic_rag/services/
  - Document service with datapizza-ai pipeline
  - RAG service with retrieval + generation
  - Vector store service (Qdrant)
- static/index.html - Web UI (upload + chat)

## Dependencies
- datapizza-ai (core framework)
- datapizza-ai-clients-openai
- datapizza-ai-embedders-openai
- datapizza-ai-vectorstores-qdrant
- FastAPI, Pydantic, Qdrant

## API Endpoints
- POST /api/v1/documents - Upload documents
- GET /api/v1/documents - List documents
- POST /api/v1/query - Query knowledge base
- POST /api/v1/chat - Chat interface
- GET /api/docs - Swagger documentation
- GET / - Web UI

🏁 Ready for testing and deployment!
2026-04-06 10:56:43 +02:00
Luca Sacchi Ricciardi
f1016f94ca feat(api): add webhook system (Sprint 5 - FINAL)
Implement Sprint 5: Webhook System - FINAL SPRINT

- Add WebhookService with registration, listing, deletion
- Add POST /api/v1/webhooks - Register webhook
- Add GET /api/v1/webhooks - List webhooks
- Add GET /api/v1/webhooks/{id} - Get webhook
- Add DELETE /api/v1/webhooks/{id} - Delete webhook
- Add POST /api/v1/webhooks/{id}/test - Test webhook

Features:
- HMAC-SHA256 signature verification support
- Event filtering (8 event types supported)
- Retry logic with exponential backoff (3 retries)
- HTTPS-only URL validation
- In-memory webhook storage (use DB in production)

Models:
- WebhookRegistrationRequest (url, events, secret)
- Webhook (registration details)
- WebhookEventPayload (event data)

Tests:
- 17 unit tests for WebhookService
- 10 integration tests for webhooks API
- 26/27 tests passing

🏁 FINAL SPRINT COMPLETE - API v1.0.0 READY!
2026-04-06 10:26:18 +02:00
Luca Sacchi Ricciardi
83fd30a2a2 feat(api): add content generation endpoints (Sprint 4)
Implement Sprint 4: Content Generation

- Add ArtifactService with generation methods for 9 content types
- Add POST /generate/audio - Generate podcast
- Add POST /generate/video - Generate video
- Add POST /generate/slide-deck - Generate slides
- Add POST /generate/infographic - Generate infographic
- Add POST /generate/quiz - Generate quiz
- Add POST /generate/flashcards - Generate flashcards
- Add POST /generate/report - Generate report
- Add POST /generate/mind-map - Generate mind map (instant)
- Add POST /generate/data-table - Generate data table
- Add GET /artifacts - List artifacts
- Add GET /artifacts/{id}/status - Check artifact status

Models:
- AudioGenerationRequest, VideoGenerationRequest
- QuizGenerationRequest, FlashcardsGenerationRequest
- SlideDeckGenerationRequest, InfographicGenerationRequest
- ReportGenerationRequest, DataTableGenerationRequest
- Artifact, GenerationResponse, ArtifactList

Tests:
- 13 unit tests for ArtifactService
- 6 integration tests for generation API
- 19/19 tests passing

Related: Sprint 4 - Content Generation
2026-04-06 01:58:47 +02:00
Luca Sacchi Ricciardi
081f3f0d89 feat(api): add chat functionality (Sprint 3)
Implement Sprint 3: Chat Functionality

- Add ChatService with send_message and get_history methods
- Add POST /api/v1/notebooks/{id}/chat - Send message
- Add GET /api/v1/notebooks/{id}/chat/history - Get chat history
- Add ChatRequest model (message, include_references)
- Add ChatResponse model (message, sources[], timestamp)
- Add ChatMessage model (id, role, content, timestamp, sources)
- Add SourceReference model (source_id, title, snippet)
- Integrate chat router with main app

Features:
- Send messages to notebook chat
- Get AI responses with source references
- Retrieve chat history
- Support for citations in responses

Tests:
- 14 unit tests for ChatService
- 11 integration tests for chat API
- 25/25 tests passing

Related: Sprint 3 - Chat Functionality
2026-04-06 01:48:19 +02:00
Luca Sacchi Ricciardi
3991ffdd7f test(sources): add comprehensive tests for Sprint 2
- Add 28 unit tests for SourceService (TEST-004)
  - Test all CRUD operations
  - Test validation logic
  - Test error handling
  - Test research functionality

- Add 13 integration tests for sources API (TEST-005)
  - Test POST /sources endpoint
  - Test GET /sources endpoint with filters
  - Test DELETE /sources endpoint
  - Test POST /sources/research endpoint

- Fix ValidationError signatures in SourceService
- Fix NotebookLMError signatures
- Fix status parameter shadowing in sources router

Coverage: 28/28 unit tests pass, 13/13 integration tests pass
2026-04-06 01:42:07 +02:00
Luca Sacchi Ricciardi
d869ab215c feat(api): add source management endpoints (Sprint 2)
Implement Sprint 2: Source Management

- Add SourceService with create, list, delete, research methods
- Add POST /api/v1/notebooks/{id}/sources - Add source (URL, YouTube, Drive)
- Add GET /api/v1/notebooks/{id}/sources - List sources with filtering
- Add DELETE /api/v1/notebooks/{id}/sources/{source_id} - Delete source
- Add POST /api/v1/notebooks/{id}/sources/research - Web research
- Add ResearchRequest model for research parameters
- Integrate sources router with main app

Endpoints:
- POST /sources - 201 Created
- GET /sources - 200 OK with pagination
- DELETE /sources/{id} - 204 No Content
- POST /sources/research - 202 Accepted

Technical:
- Support for url, youtube, drive source types
- Filtering by source_type and status
- Validation for research mode (fast/deep)
- Error handling with standardized responses

Related: Sprint 2 - Source Management
2026-04-06 01:26:59 +02:00
Luca Sacchi Ricciardi
fe88bf2ca1 refactor: fix linting issues and code quality
- Fix import ordering in __init__.py
- Remove unused imports from dependencies.py
- Fix import sorting across multiple files
- Apply ruff auto-fixes

No functional changes
2026-04-06 01:19:38 +02:00
Luca Sacchi Ricciardi
4b7a419a98 feat(api): implement notebook management CRUD endpoints
Implement Sprint 1: Notebook Management CRUD

- Add NotebookService with full CRUD operations
- Add POST /api/v1/notebooks (create notebook)
- Add GET /api/v1/notebooks (list with pagination)
- Add GET /api/v1/notebooks/{id} (get by ID)
- Add PATCH /api/v1/notebooks/{id} (partial update)
- Add DELETE /api/v1/notebooks/{id} (delete)
- Add Pydantic models for requests/responses
- Add custom exceptions (ValidationError, NotFoundError, NotebookLMError)
- Add comprehensive unit tests (31 tests, 97% coverage)
- Add API integration tests (26 tests)
- Fix router prefix duplication
- Fix JSON serialization in error responses

BREAKING CHANGE: None
2026-04-06 01:13:13 +02:00