finish the testing framework

This commit is contained in:
2026-01-16 20:15:34 +01:00
parent 2e097e4e54
commit 8de36aa975
10 changed files with 1044 additions and 62 deletions

View File

@@ -10,6 +10,8 @@ The testing framework enables automated validation of UI behavior through:
2. **Input Simulation**: Mouse clicks via Windows SendInput API
3. **Log Verification**: Check navigation events via log file parsing
4. **Test Results**: JSON output compatible with CI/CD pipelines
5. **Action Recording/Playback**: Record and replay user interactions
6. **Visual Regression**: Screenshot comparison with pixel-level diff
## Components
@@ -21,8 +23,22 @@ The desktop designer serves as the test target. When launched with testing optio
mosis-designer.exe home.rml --log test.log --hierarchy hierarchy.json
```
- `--log`: Writes all RmlUi INFO logs to file (navigation events, errors)
- `--hierarchy`: Dumps UI element tree to JSON each frame
**Testing Options**:
| Option | Description |
|--------|-------------|
| `--log FILE` | Write all RmlUi INFO logs to file (navigation events, errors) |
| `--hierarchy FILE` | Dump UI element tree to JSON each frame |
| `--record FILE` | Enable action recording mode (F5 to start/stop) |
| `--playback FILE` | Play back recorded actions from JSON file |
**Keyboard Controls**:
| Key | Function |
|-----|----------|
| F5 | Start/stop recording (when `--record` is enabled) |
| F6 | Pause/resume playback (when `--playback` is enabled) |
| F12 | Take screenshot (saves to current directory) |
### Test Runner (designer-test.exe)
@@ -231,6 +247,124 @@ results.SaveToFile(resultsPath);
}
```
## Action Recording and Playback
The designer supports recording user interactions and playing them back for automated testing.
### Recording Actions
```bash
# Start designer with recording enabled
mosis-designer.exe home.rml --record my-test.json
# Press F5 to start recording
# Interact with the UI (clicks, swipes, etc.)
# Press F5 again to stop and save
```
Recording is automatically saved when you close the window.
### Playing Back Actions
```bash
# Play back a recorded test
mosis-designer.exe home.rml --playback my-test.json
```
Use F6 to pause/resume playback.
### Action Recording Format
```json
{
"name": "Navigate to contacts",
"description": "Test navigation flow",
"screen_width": 540,
"screen_height": 960,
"initial_screen": "apps/home/home.rml",
"actions": [
{"type": "tap", "x": 413, "y": 1174, "timestamp": 0},
{"type": "wait", "duration": 1000, "timestamp": 100},
{"type": "tap", "x": 40, "y": 28, "timestamp": 1100},
{"type": "swipe", "x1": 100, "y1": 500, "x2": 100, "y2": 200, "duration": 300, "timestamp": 2000}
]
}
```
### Supported Action Types
| Type | Fields | Description |
|------|--------|-------------|
| `tap` | x, y, timestamp | Single tap at coordinates |
| `swipe` | x1, y1, x2, y2, duration, timestamp | Swipe gesture |
| `long_press` | x, y, duration, timestamp | Long press gesture |
| `button` | button, timestamp | Hardware button ("back", "home") |
| `wait` | duration, timestamp | Pause between actions |
| `key` | key_code, pressed, timestamp | Keyboard input |
### Creating Test Files Manually
You can also create test files manually using the UI hierarchy to find element coordinates:
```bash
# Get element coordinates from hierarchy
mosis-designer.exe home.rml --hierarchy hierarchy.json
# Read hierarchy.json to find element bounds
# Write action JSON with those coordinates
```
## Screenshot Comparison
The testing framework includes pixel-level screenshot comparison for visual regression testing.
### Using Screenshot Comparison
```cpp
#include "testing/visual_capture.h"
// Capture a screenshot
mosis::testing::VisualCapture capture(540, 960);
capture.CaptureScreenshot("current.png");
// Compare two screenshots
float diff = mosis::testing::VisualCapture::CompareImages("baseline.png", "current.png");
// diff = 0.0 means identical
// diff = 1.0 means completely different
// Typical threshold: diff < 0.01 (less than 1% different)
```
### Comparison Details
- Compares RGBA pixels with a tolerance of 2 per channel
- Returns ratio of differing pixels (0.0 to 1.0)
- Different dimensions = 1.0 (completely different)
- Missing files = 1.0 (comparison failed)
### Visual Regression Test Example
```cpp
bool TestVisualRegression(TestContext& ctx) {
// Navigate to screen
GoHome(ctx);
ClickById(ctx, "dock-phone");
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
// Capture screenshot
mosis::testing::VisualCapture capture(ctx.width, ctx.height);
capture.CaptureScreenshot("dialer-current.png");
// Compare with baseline
float diff = mosis::testing::VisualCapture::CompareImages(
"baselines/dialer-expected.png",
"dialer-current.png"
);
// Allow up to 1% difference
return diff < 0.01f;
}
```
## Running Tests
### Command Line
@@ -389,9 +523,12 @@ class NavigationTest {
## Future Improvements
- [ ] Action recording (capture user interactions)
- [ ] Screenshot comparison (visual regression testing)
- [x] Action recording (capture user interactions) - *CLI and infrastructure complete*
- [x] Screenshot comparison (visual regression testing) - *Pixel-level diff implemented*
- [x] Action playback with timing - *Fully functional*
- [x] GLFW input hooks for automatic mouse recording - *Complete via forked backend*
- [ ] Android instrumentation test suite
- [ ] Parallel test execution
- [ ] Test coverage reporting
- [ ] Cross-platform test runner (desktop + Android)
- [ ] Visual diff output (highlight changed pixels)