178 lines
5.3 KiB
Markdown
178 lines
5.3 KiB
Markdown
# Milestone 2: Testing Framework
|
|
|
|
**Status**: 100% Complete
|
|
**Goal**: Automated UI testing for rapid iteration and AI agent verification.
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
The testing framework enables automated validation of UI behavior through:
|
|
- UI hierarchy inspection (JSON dump)
|
|
- Input simulation and recording
|
|
- Log verification for navigation events
|
|
- Visual regression testing via screenshot diff
|
|
- JSON test results compatible with CI/CD
|
|
|
|
---
|
|
|
|
## Current State
|
|
|
|
### Completed Components
|
|
|
|
| Component | File | Status |
|
|
|-----------|------|--------|
|
|
| Action Types | `designer/src/testing/action_types.h` | Complete |
|
|
| Action Recorder | `designer/src/testing/action_recorder.h/.cpp` | Complete |
|
|
| Action Player | `designer/src/testing/action_player.h/.cpp` | Complete |
|
|
| UI Inspector | `designer/src/testing/ui_inspector.cpp` | Complete |
|
|
| Visual Capture | `designer/src/testing/visual_capture.cpp` | Complete |
|
|
| Test Runner | `designer-test/src/test_runner.cpp` | Complete |
|
|
| CLI Integration | `designer/src/main.cpp` | Complete |
|
|
|
|
### Already Working
|
|
|
|
1. **Action Types** (`action_types.h`)
|
|
- TapAction, SwipeAction, LongPressAction
|
|
- ButtonAction, WaitAction, KeyAction
|
|
- ActionSequence container with metadata
|
|
|
|
2. **Action Recorder** (`action_recorder.h/.cpp`)
|
|
- Mouse down/up/move tracking
|
|
- Automatic gesture classification (tap vs swipe vs long press)
|
|
- JSON serialization/deserialization
|
|
- Configurable thresholds
|
|
|
|
3. **Action Player** (`action_player.h/.cpp`)
|
|
- Timestamp-based playback
|
|
- RmlUi event injection
|
|
- Pause/resume/stop controls
|
|
- Progress tracking
|
|
|
|
4. **UI Inspector** (`ui_inspector.cpp`)
|
|
- Full element tree traversal
|
|
- JSON export with bounds, IDs, classes
|
|
- Continuous hierarchy dumping mode
|
|
|
|
5. **Test Runner** (`designer-test/`)
|
|
- WindowController (Windows SendInput)
|
|
- HierarchyReader (element lookup)
|
|
- LogParser (navigation verification)
|
|
- All 5 navigation tests passing
|
|
|
|
---
|
|
|
|
## Recently Completed
|
|
|
|
### Screenshot Diff Implementation (Complete)
|
|
|
|
**File**: `designer/src/testing/visual_capture.cpp`
|
|
|
|
Implemented real PNG pixel-by-pixel comparison:
|
|
- `LoadPNG()` helper function loads PNG files using libpng
|
|
- `PixelsMatch()` compares pixels with configurable tolerance (default: 2)
|
|
- `CompareImages()` returns difference ratio (0.0 = identical, 1.0 = completely different)
|
|
|
|
### Recording/Playback CLI (Complete)
|
|
|
|
**File**: `designer/src/main.cpp`
|
|
|
|
Added CLI options and keyboard controls:
|
|
- `--record <file>` - Enable recording mode
|
|
- `--playback <file>` - Play back recorded actions
|
|
- F5 - Start/stop recording (saves to specified file)
|
|
- F6 - Pause/resume playback
|
|
|
|
**Usage**:
|
|
```bash
|
|
# Record interactions
|
|
mosis-designer.exe home.rml --record my-test.json
|
|
# Press F5 to start recording, interact, press F5 again to save
|
|
|
|
# Play back
|
|
mosis-designer.exe home.rml --playback my-test.json
|
|
```
|
|
|
|
---
|
|
|
|
## Completed: GLFW Input Hooks for Recording
|
|
|
|
### Task 2.3: GLFW Input Hooks
|
|
|
|
**Status**: Complete
|
|
|
|
**Solution Implemented**:
|
|
Forked the RmlUi backend files into `designer/src/backend/` with input recording hooks:
|
|
|
|
1. **RmlUi_Backend.h** - Added callback type definitions:
|
|
- `MouseButtonCallback` - Called on mouse button press/release
|
|
- `MouseMoveCallback` - Called on mouse movement
|
|
- `KeyCallback` - Called on key press/release
|
|
|
|
2. **RmlUi_Backend_GLFW_GL3.cpp** - Modified GLFW callbacks to:
|
|
- Track mouse position in framebuffer coordinates
|
|
- Call recording callbacks before forwarding to RmlUi
|
|
- Support all three callback types
|
|
|
|
3. **main.cpp** - Connected callbacks to ActionRecorder:
|
|
- Mouse button events trigger `RecordMouseDown`/`RecordMouseUp`
|
|
- Mouse move events trigger `RecordMouseMove`
|
|
- Key events trigger `RecordKey`
|
|
|
|
**Files Added**:
|
|
- `designer/src/backend/RmlUi_Backend.h`
|
|
- `designer/src/backend/RmlUi_Backend_GLFW_GL3.cpp`
|
|
- `designer/src/backend/RmlUi_Platform_GLFW.h`
|
|
- `designer/src/backend/RmlUi_Platform_GLFW.cpp`
|
|
|
|
---
|
|
|
|
## File Changes Summary
|
|
|
|
| File | Changes | Status |
|
|
|------|---------|--------|
|
|
| `designer/src/testing/visual_capture.cpp` | Real PNG pixel comparison | Done |
|
|
| `designer/src/main.cpp` | --record/--playback CLI, F5/F6 keys | Done |
|
|
| `designer/src/RmlUi_Backend.h` | Expose GLFW window for recording | Future |
|
|
|
|
---
|
|
|
|
## Test Recording Format
|
|
|
|
```json
|
|
{
|
|
"name": "Navigate to contacts and back",
|
|
"description": "Test navigation flow",
|
|
"screen_width": 540,
|
|
"screen_height": 960,
|
|
"initial_screen": "apps/home/home.rml",
|
|
"actions": [
|
|
{"type": "tap", "x": 413, "y": 1174, "timestamp": 0},
|
|
{"type": "wait", "duration": 1000, "timestamp": 100},
|
|
{"type": "tap", "x": 40, "y": 28, "timestamp": 1100},
|
|
{"type": "swipe", "x1": 100, "y1": 500, "x2": 100, "y2": 200, "duration": 300, "timestamp": 2000}
|
|
]
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [x] `CompareImages()` returns accurate pixel difference ratio
|
|
- [ ] `--record <file>` captures all mouse/key events to JSON (needs GLFW hooks)
|
|
- [x] `--playback <file>` replays recorded actions with correct timing
|
|
- [x] Recording stops gracefully on F5 or window close
|
|
- [x] Playback shows progress in console
|
|
- [x] Screenshot diff with 2-pixel tolerance per channel
|
|
|
|
---
|
|
|
|
## Future Enhancements (Not This Milestone)
|
|
|
|
- Visual diff output (highlight changed pixels)
|
|
- Parallel test execution
|
|
- Android action recording/playback
|
|
- Cross-platform test runner
|
|
- Coverage reporting
|