# Automated Testing Framework The Mosis Designer supports two testing approaches: 1. **JSON Action Playback** - Built into the designer for scripted UI testing 2. **C++ Test Framework** - External test runner in `designer-test/` for programmatic testing --- ## JSON Action Playback The designer can record and playback UI interactions using JSON files. This is useful for: - Automated screenshot capture - Regression testing - Demo recordings - UI verification ### Command-Line Options ```bash # Run with action playback mosis-designer.exe --simulator --test-apps base-apps \ --playback test_navigation.json \ --screenshot-after result.png \ --hierarchy hierarchy.json ``` | Flag | Description | |------|-------------| | `--playback ` | JSON file containing actions to replay | | `--screenshot-after ` | Save screenshot after playback completes | | `--hierarchy ` | Dump UI hierarchy to JSON after playback | ### Recording Actions Press `R` in the designer to start/stop recording. Actions are saved to `recorded_actions.json`. ### Action Types #### Tap Single tap at coordinates. ```json { "type": "tap", "x": 270, "y": 480, "timestamp": 1000 } ``` #### Swipe Drag from one point to another. ```json { "type": "swipe", "x1": 270, "y1": 600, "x2": 270, "y2": 200, "duration": 300, "timestamp": 2000 } ``` #### Long Press Press and hold at coordinates. ```json { "type": "long_press", "x": 270, "y": 480, "duration": 800, "timestamp": 3000 } ``` #### Button System button press (back, home, recents). ```json { "type": "button", "button": "back", "timestamp": 4000 } ``` #### Wait Pause execution for a duration. ```json { "type": "wait", "duration": 1500, "timestamp": 5000 } ``` #### Key Raw keyboard input (for special keys). ```json { "type": "key", "key_code": 256, "pressed": true, "timestamp": 6000 } ``` Common key codes: - `256` - Escape (Back) - `301` - F12 (Debugger) - `294` - F5 (Reload) ### Complete Test File Example ```json { "name": "Navigate to Messages", "description": "Test navigation from home to Messages app", "screen_width": 540, "screen_height": 960, "initial_screen": "", "actions": [ { "type": "wait", "duration": 500, "timestamp": 0 }, { "type": "tap", "x": 207, "y": 220, "timestamp": 500 }, { "type": "wait", "duration": 1500, "timestamp": 600 } ] } ``` ### Finding Tap Coordinates Use the `--hierarchy` flag to dump UI element bounds: ```bash mosis-designer.exe --simulator --test-apps base-apps \ --playback wait_only.json \ --hierarchy hierarchy.json ``` The hierarchy JSON contains element bounds: ```json { "bounds": { "x": 171.5, "y": 183.2, "width": 72.0, "height": 72.0 }, "classes": ["app-icon-image"], "tag": "div" } ``` Calculate tap center: `x = bounds.x + width/2`, `y = bounds.y + height/2` ### Home Screen Grid Layout For the default 540x960 resolution with 4-column grid: | Row | Y Range | Center Y | |-----|---------|----------| | 1 | 64-167 | ~115 | | 2 | 183-286 | ~220 | | 3 | 302-405 | ~350 | | Column | X Range | Center X | |--------|---------|----------| | 1 | 20-145 | ~82 | | 2 | 145-270 | ~207 | | 3 | 270-395 | ~332 | | 4 | 395-520 | ~457 | Dock items are at Y ~910. --- ## Asset Path Requirements When testing apps in `base-apps/`, ensure shared assets are accessible. Apps use relative paths like `../../ui/html.rcss`. From `base-apps/com.mosis.app/`, the path `../../` resolves to `MosisService/`. Required directories at `MosisService/` root: ``` MosisService/ ├── ui/ # html.rcss, theme.rcss, components.rcss, layout.rcss ├── scripts/ # navigation.lua, layout.lua ├── icons/ # Shared icon assets └── base-apps/ # Test apps ``` Copy from `src/main/assets/`: ```bash cp -r src/main/assets/ui . cp -r src/main/assets/scripts . cp -r src/main/assets/icons . ``` --- ## C++ Test Framework The `designer-test/` project provides programmatic testing with element finding. ### Architecture | Component | Purpose | |-----------|---------| | WindowController | Sends mouse/keyboard via Windows SendInput API | | HierarchyReader | Parses UI hierarchy JSON, finds elements by ID/class | | LogParser | Monitors log file for navigation events | | TestRunner | Orchestrates execution, reports results | | UIInspector | Dumps hierarchy with atomic writes | ### Key Implementation Details - **Path Normalization**: RmlUi uses `|` instead of `:` in Windows paths - **Atomic File Writes**: Hierarchy written to `.tmp` then renamed - **Retry with Backoff**: HierarchyReader retries up to 10 times with exponential backoff - **Dynamic Back Button**: Finds back buttons by class instead of fixed coordinates ### Writing C++ Tests ```cpp bool ClickById(TestContext& ctx, const std::string& id) { ctx.hierarchy.Reload(); auto element = ctx.hierarchy.FindById(id); if (!element) return false; int x = element->bounds.centerX(); int y = element->bounds.centerY(); ScaleToPhysical(ctx, x, y); ctx.window.SendClick(x, y); return true; } bool TestNavigateToDialer(TestContext& ctx) { GoHome(ctx); ctx.log.Clear(); if (!ClickById(ctx, "dock-phone")) return false; std::this_thread::sleep_for(std::chrono::milliseconds(1000)); ctx.log.Reload(); return ctx.log.Contains("Loaded screen: apps/dialer/dialer.rml"); } ``` ### Test Output Results saved to `test_results.json`: ```json { "name": "Mosis Designer UI Tests", "summary": {"passed": 5, "failed": 0, "total": 5}, "tests": [ {"name": "Navigate to Dialer", "status": "passed", "duration_ms": 3500} ] } ``` --- ## Example Test Suite Test files for common scenarios: ### test_home_only.json ```json { "name": "Home Screenshot", "description": "Capture home screen", "screen_width": 540, "screen_height": 960, "actions": [ {"type": "wait", "duration": 500, "timestamp": 0} ] } ``` ### test_settings.json ```json { "name": "Navigate to Settings", "description": "Tap Settings app (Row 3, Col 1)", "screen_width": 540, "screen_height": 960, "actions": [ {"type": "wait", "duration": 500, "timestamp": 0}, {"type": "tap", "x": 82, "y": 350, "timestamp": 500}, {"type": "wait", "duration": 1500, "timestamp": 600} ] } ``` ### test_browser.json ```json { "name": "Navigate to Browser", "description": "Tap Browser in dock (Col 4)", "screen_width": 540, "screen_height": 960, "actions": [ {"type": "wait", "duration": 500, "timestamp": 0}, {"type": "tap", "x": 445, "y": 910, "timestamp": 500}, {"type": "wait", "duration": 1500, "timestamp": 600} ] } ``` ### Running Tests ```bash # Single test with screenshot mosis-designer.exe --simulator --test-apps base-apps \ --playback test_settings.json \ --screenshot-after screenshot_settings.png # Test with hierarchy dump for debugging mosis-designer.exe --simulator --test-apps base-apps \ --playback test_home_only.json \ --screenshot-after home.png \ --hierarchy hierarchy.json ``` --- ## Test Output Directory All test files and output should be placed in the `tests/` folder at the project root: ``` MosisService/ └── tests/ ├── test_navigation.json # Test action files ├── test_settings.json ├── screenshot_home.png # Screenshot output ├── screenshot_settings.png ├── hierarchy.json # Hierarchy dumps └── recorded_actions.json # Recorded actions ``` ### Usage ```bash # Run test with output to tests/ folder mosis-designer.exe --simulator --test-apps base-apps \ --playback tests/test_navigation.json \ --screenshot-after tests/screenshot_result.png \ --hierarchy tests/hierarchy.json ``` This keeps the project root clean and organizes all testing artifacts in one place. **Note**: The `tests/` folder is gitignored - test outputs are not committed to version control. --- ## Troubleshooting ### Tap not registering - Check coordinates against hierarchy bounds - Ensure tap is within the clickable element (e.g., `app-icon-image`, not `app-icon-label`) - Use hierarchy dump to verify element positions ### Stylesheet errors ``` [ERROR] Failed to load style sheet D|/Dev/.../ui/html.rcss ``` - Copy shared assets to MosisService root (see Asset Path Requirements) ### App not loading - Check console for `launchApp` call - Verify manifest.json exists in app directory - Check entry file path in manifest ### Screenshot shows wrong screen - Increase wait duration after tap - Verify navigation completed in console output