MosisService/TESTING.md

# Mosis UI Testing Framework

This document describes the automated UI testing framework for Mosis virtual smartphone.

## Overview

The testing framework enables automated validation of UI behavior through:

1. **UI Hierarchy Inspection**: JSON dump of all UI elements with bounds
2. **Input Simulation**: Mouse clicks via Windows SendInput API
3. **Log Verification**: Check navigation events via log file parsing
4. **Test Results**: JSON output compatible with CI/CD pipelines
5. **Action Recording/Playback**: Record and replay user interactions
6. **Visual Regression**: Screenshot comparison with pixel-level diff

## Components

### Designer (mosis-designer.exe)

The desktop designer serves as the test target. When launched with testing options:

```bash
mosis-designer.exe home.rml --log test.log --hierarchy hierarchy.json
```

**Testing Options**:

| Option | Description |
|--------|-------------|
| `--log FILE` | Write all RmlUi INFO logs to file (navigation events, errors) |
| `--hierarchy FILE` | Dump UI element tree to JSON each frame |
| `--record FILE` | Enable action recording mode (F5 to start/stop) |
| `--playback FILE` | Play back recorded actions from JSON file |

**Keyboard Controls**:

| Key | Function |
|-----|----------|
| F5 | Start/stop recording (when `--record` is enabled) |
| F6 | Pause/resume playback (when `--playback` is enabled) |
| F12 | Take screenshot (saves to current directory) |

### Test Runner (designer-test.exe)

Automated test executor that:

1. Launches designer with testing options
2. Waits for window to appear
3. Reads UI hierarchy to find elements
4. Sends input events to simulate user interaction
5. Verifies results via log file
6. Outputs JSON test results

## UI Hierarchy Format

The hierarchy JSON contains all visible UI elements:

```json
{
  "timestamp": 1705312200,
  "screen": "apps/home/home.rml",
  "resolution": {"width": 677, "height": 1202},
  "elements": {
    "tag": "body",
    "id": "",
    "classes": ["home-screen"],
    "bounds": {"x": 0, "y": 0, "width": 677, "height": 1202},
    "visible": true,
    "text": null,
    "children": [
      {
        "tag": "div",
        "id": "dock-phone",
        "classes": ["dock-item"],
        "bounds": {"x": 85, "y": 1138, "width": 56, "height": 56},
        "visible": true,
        "text": null,
        "children": [...]
      }
    ]
  }
}
```

### Key Fields

| Field | Description |
|-------|-------------|
| `tag` | HTML element tag (div, span, img, input) |
| `id` | Element ID attribute (empty if none) |
| `classes` | Array of CSS class names |
| `bounds` | Position and size in logical pixels |
| `visible` | Whether element is displayed |
| `text` | Text content (for leaf text nodes) |
| `children` | Nested child elements |

### Resolution Note

The `resolution` field contains RmlUi's logical dimensions, which may differ from physical window size due to DPI scaling. Tests must scale coordinates:

```cpp
// Scale from hierarchy (logical) to window (physical)
int physicalX = logicalX * windowWidth / hierarchyWidth;
int physicalY = logicalY * windowHeight / hierarchyHeight;
```

## Writing Tests

### Test Function Signature

```cpp
bool MyTest(TestContext& ctx);
```

TestContext provides:
- `ctx.window` - WindowController for input
- `ctx.log` - LogParser for log verification
- `ctx.hierarchy` - HierarchyReader for element lookup

### Finding Elements

```cpp
// By ID
auto element = ctx.hierarchy.FindById("dock-phone");

// By class (returns all matches)
auto buttons = ctx.hierarchy.FindByClass("btn-icon");

// By tag
auto divs = ctx.hierarchy.FindByTag("div");
```

### Clicking Elements

```cpp
bool ClickById(TestContext& ctx, const std::string& id) {
    ctx.hierarchy.Reload();  // Get fresh hierarchy

    auto element = ctx.hierarchy.FindById(id);
    if (!element || !element->visible) {
        return false;
    }

    // Get center point
    int x = element->bounds.centerX();
    int y = element->bounds.centerY();

    // Scale to physical coordinates
    ScaleToPhysical(ctx, x, y);

    // Send click
    ctx.window.SendClick(x, y);
    return true;
}
```

### Verifying Navigation

```cpp
bool TestNavigateToDialer(TestContext& ctx) {
    GoHome(ctx);       // Return to home screen
    ctx.log.Clear();   // Clear previous logs

    if (!ClickById(ctx, "dock-phone")) {
        return false;
    }

    // Wait for navigation animation
    std::this_thread::sleep_for(std::chrono::milliseconds(1000));

    // Check log for navigation event
    ctx.log.Reload();
    return ctx.log.Contains("Loaded screen: apps/dialer/dialer.rml");
}
```

### Navigation Helpers

```cpp
// Go back to home screen by clicking back buttons
void GoHome(TestContext& ctx) {
    for (int i = 0; i < 5; ++i) {
        ctx.hierarchy.Reload();
        auto elements = ctx.hierarchy.FindByClass("app-bar-nav");
        if (!elements.empty() && elements[0].visible) {
            int x = elements[0].bounds.centerX();
            int y = elements[0].bounds.centerY();
            ScaleToPhysical(ctx, x, y);
            ctx.window.SendClick(x, y);
        }
        std::this_thread::sleep_for(std::chrono::milliseconds(400));
    }
    std::this_thread::sleep_for(std::chrono::milliseconds(800));
}
```

## Registering Tests

In `main.cpp`:

```cpp
TestRunner runner;
runner.SetDesignerPath(designerPath);
runner.SetDocumentPath(documentPath);
runner.SetLogPath(logPath);
runner.SetHierarchyPath(hierarchyPath);

// Register tests
runner.AddTest("Navigate to Dialer", TestNavigateToDialer);
runner.AddTest("Navigate to Messages", TestNavigateToMessages);
runner.AddTest("Navigate to Contacts", TestNavigateToContacts);
runner.AddTest("Navigate to Browser", TestNavigateToBrowser);
runner.AddTest("Navigation Sequence", TestNavigationSequence);

// Run all tests
auto results = runner.RunAll();
results.SaveToFile(resultsPath);
```

## Test Results Format

```json
{
  "name": "Mosis Designer UI Tests",
  "summary": {
    "passed": 5,
    "failed": 0,
    "skipped": 0,
    "errors": 0,
    "total": 5,
    "duration_ms": 31162.3
  },
  "tests": [
    {
      "name": "Navigate to Dialer",
      "status": "passed",
      "message": "Test passed",
      "duration_ms": 4216.88
    },
    {
      "name": "Navigate to Browser",
      "status": "passed",
      "message": "Test passed",
      "duration_ms": 4067.34
    }
  ]
}
```

## Action Recording and Playback

The designer supports recording user interactions and playing them back for automated testing.

### Recording Actions

```bash
# Start designer with recording enabled
mosis-designer.exe home.rml --record my-test.json

# Press F5 to start recording
# Interact with the UI (clicks, swipes, etc.)
# Press F5 again to stop and save
```

Recording is automatically saved when you close the window.

### Playing Back Actions

```bash
# Play back a recorded test
mosis-designer.exe home.rml --playback my-test.json
```

Use F6 to pause/resume playback.

### Action Recording Format

```json
{
  "name": "Navigate to contacts",
  "description": "Test navigation flow",
  "screen_width": 540,
  "screen_height": 960,
  "initial_screen": "apps/home/home.rml",
  "actions": [
    {"type": "tap", "x": 413, "y": 1174, "timestamp": 0},
    {"type": "wait", "duration": 1000, "timestamp": 100},
    {"type": "tap", "x": 40, "y": 28, "timestamp": 1100},
    {"type": "swipe", "x1": 100, "y1": 500, "x2": 100, "y2": 200, "duration": 300, "timestamp": 2000}
  ]
}
```

### Supported Action Types

| Type | Fields | Description |
|------|--------|-------------|
| `tap` | x, y, timestamp | Single tap at coordinates |
| `swipe` | x1, y1, x2, y2, duration, timestamp | Swipe gesture |
| `long_press` | x, y, duration, timestamp | Long press gesture |
| `button` | button, timestamp | Hardware button ("back", "home") |
| `wait` | duration, timestamp | Pause between actions |
| `key` | key_code, pressed, timestamp | Keyboard input |

### Creating Test Files Manually

You can also create test files manually using the UI hierarchy to find element coordinates:

```bash
# Get element coordinates from hierarchy
mosis-designer.exe home.rml --hierarchy hierarchy.json
# Read hierarchy.json to find element bounds
# Write action JSON with those coordinates
```

## Screenshot Comparison

The testing framework includes pixel-level screenshot comparison for visual regression testing.

### Using Screenshot Comparison

```cpp
#include "testing/visual_capture.h"

// Capture a screenshot
mosis::testing::VisualCapture capture(540, 960);
capture.CaptureScreenshot("current.png");

// Compare two screenshots
float diff = mosis::testing::VisualCapture::CompareImages("baseline.png", "current.png");

// diff = 0.0 means identical
// diff = 1.0 means completely different
// Typical threshold: diff < 0.01 (less than 1% different)
```

### Comparison Details

- Compares RGBA pixels with a tolerance of 2 per channel
- Returns ratio of differing pixels (0.0 to 1.0)
- Different dimensions = 1.0 (completely different)
- Missing files = 1.0 (comparison failed)

### Visual Regression Test Example

```cpp
bool TestVisualRegression(TestContext& ctx) {
    // Navigate to screen
    GoHome(ctx);
    ClickById(ctx, "dock-phone");
    std::this_thread::sleep_for(std::chrono::milliseconds(1000));

    // Capture screenshot
    mosis::testing::VisualCapture capture(ctx.width, ctx.height);
    capture.CaptureScreenshot("dialer-current.png");

    // Compare with baseline
    float diff = mosis::testing::VisualCapture::CompareImages(
        "baselines/dialer-expected.png",
        "dialer-current.png"
    );

    // Allow up to 1% difference
    return diff < 0.01f;
}
```

## Running Tests

### Command Line

```bash
# Run with defaults (auto-detects paths)
designer-test.exe

# Override paths
designer-test.exe --designer /path/to/mosis-designer.exe \
                  --document /path/to/home.rml \
                  --log /path/to/output.log \
                  --hierarchy /path/to/hierarchy.json \
                  --results /path/to/results.json
```

### Exit Codes

- `0`: All tests passed
- `1`: One or more tests failed

## Best Practices

### Element Identification

1. **Use IDs** for clickable elements that tests need to find
2. **Use semantic classes** like `app-bar-nav` for navigation buttons
3. **Avoid relying on position** - use hierarchy lookup instead

### Timing

1. **Wait after navigation** (1000ms) for animations to complete
2. **Reload hierarchy** before each element lookup
3. **Use retry logic** for file reads (hierarchy may be mid-write)

### Test Independence

1. **Start from known state** - call GoHome() at start of each test
2. **Clear logs** before verification
3. **Don't depend on previous test state**

## Adding Test IDs to RML

When creating new screens, add IDs to interactive elements:

```html
<!-- Good: Has ID for testing -->
<div id="dock-phone" class="dock-item" onclick="navigateTo('dialer')">
    <img src="../../icons/phone.tga"/>
</div>

<!-- Good: Has class for back button detection -->
<div class="app-bar-nav btn-icon" onclick="goBack()">
    <img src="../../icons/back.tga"/>
</div>
```

## Troubleshooting

### Element Not Found

- Verify the screen has loaded (check hierarchy `screen` field)
- Check if element has the expected ID/class
- Ensure element is visible (`visible: true`)

### Click Not Registering

- Check coordinate scaling (hierarchy vs window size)
- Increase wait time for animations
- Verify window is in foreground

### Timing Issues

- Increase delays after GoHome() and navigation
- Add retries for hierarchy reload
- Check for race conditions in file I/O

## Android Testing

The Android app supports event injection for automated testing.

### Event Injection Methods

The `MainActivity` provides these methods for programmatic testing:

```kotlin
// Inject touch events with normalized coordinates (0.0-1.0)
MainActivity.instance?.injectTouchDown(x, y)
MainActivity.instance?.injectTouchMove(x, y)
MainActivity.instance?.injectTouchUp(x, y)
MainActivity.instance?.injectClick(x, y)  // Down + delay + Up
```

### Broadcast-Based Injection

For external test frameworks or ADB, send broadcasts:

```bash
# Inject a click at center (0.5, 0.5)
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" \
    --ef x 0.5 \
    --ef y 0.5

# Inject touch down
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "down" \
    --ef x 0.2 \
    --ef y 0.9

# Inject touch up
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "up" \
    --ef x 0.2 \
    --ef y 0.9
```

### UI Element Coordinates

Dock items are positioned at the bottom of the screen. Approximate normalized positions:

| Element | X | Y |
|---------|---|---|
| dock-phone | 0.16 | 0.97 |
| dock-messages | 0.39 | 0.97 |
| dock-contacts | 0.61 | 0.97 |
| dock-browser | 0.84 | 0.97 |

### Instrumentation Tests

For full integration tests, use Android's instrumentation framework:

```kotlin
@RunWith(AndroidJUnit4::class)
class NavigationTest {
    @get:Rule
    val activityRule = ActivityScenarioRule(MainActivity::class.java)

    @Test
    fun testNavigateToDialer() {
        // Wait for service to connect
        Thread.sleep(2000)

        // Click on phone dock icon (normalized coords)
        activityRule.scenario.onActivity { activity ->
            activity.injectClick(0.16f, 0.97f)
        }

        // Wait for navigation
        Thread.sleep(1000)

        // Verify navigation occurred (check logs or UI state)
    }
}
```

## Future Improvements

- [x] Action recording (capture user interactions) - *CLI and infrastructure complete*
- [x] Screenshot comparison (visual regression testing) - *Pixel-level diff implemented*
- [x] Action playback with timing - *Fully functional*
- [x] GLFW input hooks for automatic mouse recording - *Complete via forked backend*
- [ ] Android instrumentation test suite
- [ ] Parallel test execution
- [ ] Test coverage reporting
- [ ] Cross-platform test runner (desktop + Android)
- [ ] Visual diff output (highlight changed pixels)