mockupAWS/frontend/e2e/TEST-PLAN-v050.md

mockupAWS v0.5.0 Testing Strategy

Overview

This document outlines the comprehensive testing strategy for mockupAWS v0.5.0, focusing on the new authentication, API keys, and advanced filtering features.

Test Period: 2026-04-07 onwards
Target Version: v0.5.0
QA Engineer: @qa-engineer

---

Test Objectives

1. Authentication System - Verify JWT-based authentication flow works correctly
2. API Key Management - Test API key creation, revocation, and access control
3. Advanced Filters - Validate filtering functionality on scenarios list
4. E2E Regression - Ensure v0.4.0 features work with new auth requirements

---

Test Suite Overview

Test Suite	File	Test Count	Priority
QA-AUTH-019	auth.spec.ts	18+	P0 (Critical)
QA-APIKEY-020	apikeys.spec.ts	20+	P0 (Critical)
QA-FILTER-021	scenarios.spec.ts	24+	P1 (High)
QA-E2E-022	regression-v050.spec.ts	15+	P1 (High)

---

QA-AUTH-019: Authentication Tests

File: frontend/e2e/auth.spec.ts

Test Categories

1. Registration Tests

Test Case	Description	Expected Result
REG-001	Register new user successfully	Redirect to dashboard, token stored
REG-002	Duplicate email registration	Error message displayed
REG-003	Password mismatch	Validation error shown
REG-004	Invalid email format	Validation error shown
REG-005	Weak password	Validation error shown
REG-006	Missing required fields	Validation errors displayed
REG-007	Navigate to login from register	Login page displayed

2. Login Tests

Test Case	Description	Expected Result
LOG-001	Login with valid credentials	Redirect to dashboard
LOG-002	Login with invalid credentials	Error message shown
LOG-003	Login with non-existent user	Error message shown
LOG-004	Invalid email format	Validation error shown
LOG-005	Navigate to register from login	Register page displayed
LOG-006	Navigate to forgot password	Password reset page displayed

3. Protected Routes Tests

Test Case	Description	Expected Result
PROT-001	Access /scenarios without auth	Redirect to login
PROT-002	Access /profile without auth	Redirect to login
PROT-003	Access /settings without auth	Redirect to login
PROT-004	Access /settings/api-keys without auth	Redirect to login
PROT-005	Access /scenarios with auth	Page displayed
PROT-006	Auth persistence after refresh	Still authenticated

4. Logout Tests

Test Case	Description	Expected Result
OUT-001	Logout redirects to login	Login page displayed
OUT-002	Clear tokens on logout	localStorage cleared
OUT-003	Access protected route after logout	Redirect to login

5. Token Management Tests

Test Case	Description	Expected Result
TOK-001	Token refresh mechanism	New tokens issued
TOK-002	Store tokens in localStorage	Tokens persisted

---

QA-APIKEY-020: API Keys Tests

File: frontend/e2e/apikeys.spec.ts

Test Categories

1. Create API Key (UI)

Test Case	Description	Expected Result
CREATE-001	Navigate to API Keys page	Settings page loaded
CREATE-002	Create new API key	Modal with full key displayed
CREATE-003	Copy API key to clipboard	Success message shown
CREATE-004	Key appears in list after creation	Key visible in table
CREATE-005	Validate required fields	Error message shown

2. Revoke API Key (UI)

Test Case	Description	Expected Result
REVOKE-001	Revoke API key	Key removed from list
REVOKE-002	Confirm before revoke	Confirmation dialog shown

3. API Access with Key (API)

Test Case	Description	Expected Result
ACCESS-001	Access API with valid key	200 OK
ACCESS-002	Access /auth/me with key	User info returned
ACCESS-003	Access with revoked key	401 Unauthorized
ACCESS-004	Access with invalid key format	401 Unauthorized
ACCESS-005	Access with non-existent key	401 Unauthorized
ACCESS-006	Access without key header	401 Unauthorized
ACCESS-007	Respect API key scopes	Operations allowed per scope
ACCESS-008	Track last used timestamp	Timestamp updated

4. API Key Management (API)

Test Case	Description	Expected Result
MGMT-001	List all API keys	Keys returned without full key
MGMT-002	Key prefix in list	Prefix visible, full key hidden
MGMT-003	Create key with expiration	Expiration date set
MGMT-004	Rotate API key	New key issued, old revoked

5. API Key List View (UI)

Test Case	Description	Expected Result
LIST-001	Display keys table	All columns visible
LIST-002	Empty state	Message shown when no keys
LIST-003	Display key prefix	Prefix visible in table

---

QA-FILTER-021: Filters Tests

File: frontend/e2e/scenarios.spec.ts

Test Categories

1. Region Filter

Test Case	Description	Expected Result
REGION-001	Apply us-east-1 filter	Only us-east-1 scenarios shown
REGION-002	Apply eu-west-1 filter	Only eu-west-1 scenarios shown
REGION-003	No region filter	All scenarios shown

2. Cost Filter

Test Case	Description	Expected Result
COST-001	Apply min cost filter	Scenarios above min shown
COST-002	Apply max cost filter	Scenarios below max shown
COST-003	Apply cost range	Scenarios within range shown

3. Status Filter

Test Case	Description	Expected Result
STATUS-001	Filter by draft status	Only draft scenarios shown
STATUS-002	Filter by running status	Only running scenarios shown

4. Combined Filters

Test Case	Description	Expected Result
COMBINE-001	Combine region + status	Both filters applied
COMBINE-002	URL sync with filters	Query params updated
COMBINE-003	Parse filters from URL	Filters applied on load
COMBINE-004	Multiple regions in URL	All regions filtered

5. Clear Filters

Test Case	Description	Expected Result
CLEAR-001	Clear all filters	Full list restored
CLEAR-002	Clear individual filter	Specific filter removed
CLEAR-003	Clear on refresh	Filters reset

6. Search by Name

Test Case	Description	Expected Result
SEARCH-001	Search by exact name	Matching scenario shown
SEARCH-002	Partial name match	Partial matches shown
SEARCH-003	Non-matching search	Empty results or message
SEARCH-004	Combine search + filters	Both applied
SEARCH-005	Clear search	All results shown

7. Date Range Filter

Test Case	Description	Expected Result
DATE-001	Filter by from date	Scenarios after date shown
DATE-002	Filter by date range	Scenarios within range shown

---

QA-E2E-022: E2E Regression Tests

File: frontend/e2e/regression-v050.spec.ts

Test Categories

1. Scenario CRUD with Auth

Test Case	Description	Expected Result
CRUD-001	Display scenarios list	Table with headers visible
CRUD-002	Navigate to scenario detail	Detail page loaded
CRUD-003	Display scenario metrics	All metrics visible
CRUD-004	404 for non-existent scenario	Error message shown

2. Log Ingestion with Auth

Test Case	Description	Expected Result
INGEST-001	Start scenario and ingest logs	Logs accepted, metrics updated
INGEST-002	Persist metrics after refresh	Metrics remain visible

3. Reports with Auth

Test Case	Description	Expected Result
REPORT-001	Generate PDF report	Report created successfully
REPORT-002	Generate CSV report	Report created successfully

4. Navigation with Auth

Test Case	Description	Expected Result
NAV-001	Navigate to dashboard	Dashboard loaded
NAV-002	Navigate via sidebar	Routes work correctly
NAV-003	404 for invalid routes	Error page shown
NAV-004	Maintain auth on navigation	User stays authenticated

5. Comparison with Auth

Test Case	Description	Expected Result
COMPARE-001	Compare 2 scenarios	Comparison data returned
COMPARE-002	Compare 3 scenarios	Comparison data returned

6. API Authentication Errors

Test Case	Description	Expected Result
AUTHERR-001	Access API without token	401 returned
AUTHERR-002	Access with invalid token	401 returned
AUTHERR-003	Access with malformed header	401 returned

---

Test Execution Plan

Phase 1: Prerequisites Check

• Backend auth endpoints implemented (BE-AUTH-003)
• Frontend auth pages implemented (FE-AUTH-009, FE-AUTH-010)
• API Keys endpoints implemented (BE-APIKEY-005)
• API Keys UI implemented (FE-APIKEY-011)
• Filters UI implemented (FE-FILTER-012)

Phase 2: Authentication Tests

1. Execute auth.spec.ts tests
2. Verify all registration scenarios
3. Verify all login scenarios
4. Verify protected routes behavior
5. Verify logout flow

Phase 3: API Keys Tests

1. Execute apikeys.spec.ts tests
2. Verify key creation flow
3. Verify key revocation
4. Verify API access with keys
5. Verify key rotation

Phase 4: Filters Tests

1. Execute scenarios.spec.ts tests
2. Verify region filters
3. Verify cost filters
4. Verify status filters
5. Verify combined filters
6. Verify search functionality

Phase 5: Regression Tests

1. Execute regression-v050.spec.ts tests
2. Verify v0.4.0 features with auth
3. Check pass rate on Chromium

---

Test Environment

Requirements

• Backend: Running on http://localhost:8000
• Frontend: Running on http://localhost:5173
• Database: Migrated with v0.5.0 schema
• Browsers: Chromium (primary), Firefox, WebKit

Configuration

# Run specific test suite
npx playwright test auth.spec.ts
npx playwright test apikeys.spec.ts
npx playwright test scenarios.spec.ts
npx playwright test regression-v050.spec.ts

# Run all v0.5.0 tests
npx playwright test auth.spec.ts apikeys.spec.ts scenarios.spec.ts regression-v050.spec.ts

# Run with HTML report
npx playwright test --reporter=html

---

Expected Results

Pass Rate Targets

• Chromium: >80%
• Firefox: >70%
• WebKit: >70%

Critical Path (Must Pass)

1. User registration
2. User login
3. Protected route access control
4. API key creation
5. API key access authorization
6. Scenario list filtering

---

Helper Utilities

auth-helpers.ts

Provides authentication utilities:

• registerUser() - Register via API
• loginUser() - Login via API
• loginUserViaUI() - Login via UI
• registerUserViaUI() - Register via UI
• logoutUser() - Logout via UI
• createAuthHeader() - Create Bearer header
• createApiKeyHeader() - Create API key header
• generateTestEmail() - Generate test email
• generateTestUser() - Generate test user data

test-helpers.ts

Updated with auth support:

• createScenarioViaAPI() - Now accepts accessToken
• deleteScenarioViaAPI() - Now accepts accessToken
• startScenarioViaAPI() - Now accepts accessToken
• stopScenarioViaAPI() - Now accepts accessToken
• sendTestLogs() - Now accepts accessToken

---

Known Limitations

1. API Availability: Tests will skip if backend endpoints return 404
2. Timing: Some tests include wait times for async operations
3. Cleanup: Test data cleanup may fail silently
4. Visual Tests: Visual regression tests not included in v0.5.0

---

Success Criteria

• All P0 tests passing on Chromium
• >80% overall pass rate on Chromium
• No critical authentication vulnerabilities
• API keys work correctly for programmatic access
• Filters update list in real-time
• URL sync works correctly
• v0.4.0 features still functional with auth

---

Reporting

Test Results Format

Test Suite: QA-AUTH-019
Total Tests: 18
Passed: 16 (89%)
Failed: 1
Skipped: 1

Test Suite: QA-APIKEY-020
Total Tests: 20
Passed: 18 (90%)
Failed: 1
Skipped: 1

Test Suite: QA-FILTER-021
Total Tests: 24
Passed: 20 (83%)
Failed: 2
Skipped: 2

Test Suite: QA-E2E-022
Total Tests: 15
Passed: 13 (87%)
Failed: 1
Skipped: 1

Overall Pass Rate: 85%

---

Appendix: Test Data

Test Users

• Email pattern: user.{timestamp}@test.mockupaws.com
• Password: TestPassword123!
• Full Name: Test User {timestamp}

Test Scenarios

• Name pattern: E2E Test {timestamp}
• Regions: us-east-1, eu-west-1, ap-southeast-1, us-west-2, eu-central-1
• Status: draft, running, completed

Test API Keys

• Name pattern: Test API Key {purpose}
• Scopes: read:scenarios, write:scenarios, read:reports
• Format: mk_ + 32 random characters

---

Document Version: 1.0
Last Updated: 2026-04-07
Prepared by: @qa-engineer