move docs to docs/ folder, merge architecture files, update references

2026-01-19 09:02:11 +01:00
parent 1b34b0e974
commit 010e11cf6b
68 changed files with 1741 additions and 1350 deletions
--- a/docs/MILESTONE-2.md
+++ b/docs/MILESTONE-2.md
@@ -0,0 +1,177 @@
+# Milestone 2: Testing Framework
+
+**Status**: 100% Complete
+**Goal**: Automated UI testing for rapid iteration and AI agent verification.
+
+---
+
+## Overview
+
+The testing framework enables automated validation of UI behavior through:
+- UI hierarchy inspection (JSON dump)
+- Input simulation and recording
+- Log verification for navigation events
+- Visual regression testing via screenshot diff
+- JSON test results compatible with CI/CD
+
+---
+
+## Current State
+
+### Completed Components
+
+| Component | File | Status |
+|-----------|------|--------|
+| Action Types | `designer/src/testing/action_types.h` | Complete |
+| Action Recorder | `designer/src/testing/action_recorder.h/.cpp` | Complete |
+| Action Player | `designer/src/testing/action_player.h/.cpp` | Complete |
+| UI Inspector | `designer/src/testing/ui_inspector.cpp` | Complete |
+| Visual Capture | `designer/src/testing/visual_capture.cpp` | Complete |
+| Test Runner | `designer-test/src/test_runner.cpp` | Complete |
+| CLI Integration | `designer/src/main.cpp` | Complete |
+
+### Already Working
+
+1. **Action Types** (`action_types.h`)
+   - TapAction, SwipeAction, LongPressAction
+   - ButtonAction, WaitAction, KeyAction
+   - ActionSequence container with metadata
+
+2. **Action Recorder** (`action_recorder.h/.cpp`)
+   - Mouse down/up/move tracking
+   - Automatic gesture classification (tap vs swipe vs long press)
+   - JSON serialization/deserialization
+   - Configurable thresholds
+
+3. **Action Player** (`action_player.h/.cpp`)
+   - Timestamp-based playback
+   - RmlUi event injection
+   - Pause/resume/stop controls
+   - Progress tracking
+
+4. **UI Inspector** (`ui_inspector.cpp`)
+   - Full element tree traversal
+   - JSON export with bounds, IDs, classes
+   - Continuous hierarchy dumping mode
+
+5. **Test Runner** (`designer-test/`)
+   - WindowController (Windows SendInput)
+   - HierarchyReader (element lookup)
+   - LogParser (navigation verification)
+   - All 5 navigation tests passing
+
+---
+
+## Recently Completed
+
+### Screenshot Diff Implementation (Complete)
+
+**File**: `designer/src/testing/visual_capture.cpp`
+
+Implemented real PNG pixel-by-pixel comparison:
+- `LoadPNG()` helper function loads PNG files using libpng
+- `PixelsMatch()` compares pixels with configurable tolerance (default: 2)
+- `CompareImages()` returns difference ratio (0.0 = identical, 1.0 = completely different)
+
+### Recording/Playback CLI (Complete)
+
+**File**: `designer/src/main.cpp`
+
+Added CLI options and keyboard controls:
+- `--record <file>` - Enable recording mode
+- `--playback <file>` - Play back recorded actions
+- F5 - Start/stop recording (saves to specified file)
+- F6 - Pause/resume playback
+
+**Usage**:
+```bash
+# Record interactions
+mosis-designer.exe home.rml --record my-test.json
+# Press F5 to start recording, interact, press F5 again to save
+
+# Play back
+mosis-designer.exe home.rml --playback my-test.json
+```
+
+---
+
+## Completed: GLFW Input Hooks for Recording
+
+### Task 2.3: GLFW Input Hooks
+
+**Status**: Complete
+
+**Solution Implemented**:
+Forked the RmlUi backend files into `designer/src/backend/` with input recording hooks:
+
+1. **RmlUi_Backend.h** - Added callback type definitions:
+   - `MouseButtonCallback` - Called on mouse button press/release
+   - `MouseMoveCallback` - Called on mouse movement
+   - `KeyCallback` - Called on key press/release
+
+2. **RmlUi_Backend_GLFW_GL3.cpp** - Modified GLFW callbacks to:
+   - Track mouse position in framebuffer coordinates
+   - Call recording callbacks before forwarding to RmlUi
+   - Support all three callback types
+
+3. **main.cpp** - Connected callbacks to ActionRecorder:
+   - Mouse button events trigger `RecordMouseDown`/`RecordMouseUp`
+   - Mouse move events trigger `RecordMouseMove`
+   - Key events trigger `RecordKey`
+
+**Files Added**:
+- `designer/src/backend/RmlUi_Backend.h`
+- `designer/src/backend/RmlUi_Backend_GLFW_GL3.cpp`
+- `designer/src/backend/RmlUi_Platform_GLFW.h`
+- `designer/src/backend/RmlUi_Platform_GLFW.cpp`
+
+---
+
+## File Changes Summary
+
+| File | Changes | Status |
+|------|---------|--------|
+| `designer/src/testing/visual_capture.cpp` | Real PNG pixel comparison | Done |
+| `designer/src/main.cpp` | --record/--playback CLI, F5/F6 keys | Done |
+| `designer/src/RmlUi_Backend.h` | Expose GLFW window for recording | Future |
+
+---
+
+## Test Recording Format
+
+```json
+{
+  "name": "Navigate to contacts and back",
+  "description": "Test navigation flow",
+  "screen_width": 540,
+  "screen_height": 960,
+  "initial_screen": "apps/home/home.rml",
+  "actions": [
+    {"type": "tap", "x": 413, "y": 1174, "timestamp": 0},
+    {"type": "wait", "duration": 1000, "timestamp": 100},
+    {"type": "tap", "x": 40, "y": 28, "timestamp": 1100},
+    {"type": "swipe", "x1": 100, "y1": 500, "x2": 100, "y2": 200, "duration": 300, "timestamp": 2000}
+  ]
+}
+```
+
+---
+
+## Acceptance Criteria
+
+- [x] `CompareImages()` returns accurate pixel difference ratio
+- [ ] `--record <file>` captures all mouse/key events to JSON (needs GLFW hooks)
+- [x] `--playback <file>` replays recorded actions with correct timing
+- [x] Recording stops gracefully on F5 or window close
+- [x] Playback shows progress in console
+- [x] Screenshot diff with 2-pixel tolerance per channel
+
+---
+
+## Future Enhancements (Not This Milestone)
+
+- Visual diff output (highlight changed pixels)
+- Parallel test execution
+- Android action recording/playback
+- Cross-platform test runner
+- Coverage reporting