MosisService/TESTING.md

Mosis UI Testing Framework

This document describes the automated UI testing framework for Mosis virtual smartphone.

Overview

The testing framework enables automated validation of UI behavior through:

1. UI Hierarchy Inspection: JSON dump of all UI elements with bounds
2. Input Simulation: Mouse clicks via Windows SendInput API
3. Log Verification: Check navigation events via log file parsing
4. Test Results: JSON output compatible with CI/CD pipelines

Components

Designer (mosis-designer.exe)

The desktop designer serves as the test target. When launched with testing options:

mosis-designer.exe home.rml --log test.log --hierarchy hierarchy.json

• --log: Writes all RmlUi INFO logs to file (navigation events, errors)
• --hierarchy: Dumps UI element tree to JSON each frame

Test Runner (designer-test.exe)

Automated test executor that:

1. Launches designer with testing options
2. Waits for window to appear
3. Reads UI hierarchy to find elements
4. Sends input events to simulate user interaction
5. Verifies results via log file
6. Outputs JSON test results

UI Hierarchy Format

The hierarchy JSON contains all visible UI elements:

{
  "timestamp": 1705312200,
  "screen": "apps/home/home.rml",
  "resolution": {"width": 677, "height": 1202},
  "elements": {
    "tag": "body",
    "id": "",
    "classes": ["home-screen"],
    "bounds": {"x": 0, "y": 0, "width": 677, "height": 1202},
    "visible": true,
    "text": null,
    "children": [
      {
        "tag": "div",
        "id": "dock-phone",
        "classes": ["dock-item"],
        "bounds": {"x": 85, "y": 1138, "width": 56, "height": 56},
        "visible": true,
        "text": null,
        "children": [...]
      }
    ]
  }
}

Key Fields

Field	Description
tag	HTML element tag (div, span, img, input)
id	Element ID attribute (empty if none)
classes	Array of CSS class names
bounds	Position and size in logical pixels
visible	Whether element is displayed
text	Text content (for leaf text nodes)
children	Nested child elements

Resolution Note

The resolution field contains RmlUi's logical dimensions, which may differ from physical window size due to DPI scaling. Tests must scale coordinates:

// Scale from hierarchy (logical) to window (physical)
int physicalX = logicalX * windowWidth / hierarchyWidth;
int physicalY = logicalY * windowHeight / hierarchyHeight;

Writing Tests

Test Function Signature

bool MyTest(TestContext& ctx);

TestContext provides:

• ctx.window - WindowController for input
• ctx.log - LogParser for log verification
• ctx.hierarchy - HierarchyReader for element lookup

Finding Elements

// By ID
auto element = ctx.hierarchy.FindById("dock-phone");

// By class (returns all matches)
auto buttons = ctx.hierarchy.FindByClass("btn-icon");

// By tag
auto divs = ctx.hierarchy.FindByTag("div");

Clicking Elements

bool ClickById(TestContext& ctx, const std::string& id) {
    ctx.hierarchy.Reload();  // Get fresh hierarchy

    auto element = ctx.hierarchy.FindById(id);
    if (!element || !element->visible) {
        return false;
    }

    // Get center point
    int x = element->bounds.centerX();
    int y = element->bounds.centerY();

    // Scale to physical coordinates
    ScaleToPhysical(ctx, x, y);

    // Send click
    ctx.window.SendClick(x, y);
    return true;
}

Verifying Navigation

bool TestNavigateToDialer(TestContext& ctx) {
    GoHome(ctx);       // Return to home screen
    ctx.log.Clear();   // Clear previous logs

    if (!ClickById(ctx, "dock-phone")) {
        return false;
    }

    // Wait for navigation animation
    std::this_thread::sleep_for(std::chrono::milliseconds(1000));

    // Check log for navigation event
    ctx.log.Reload();
    return ctx.log.Contains("Loaded screen: apps/dialer/dialer.rml");
}

Navigation Helpers

// Go back to home screen by clicking back buttons
void GoHome(TestContext& ctx) {
    for (int i = 0; i < 5; ++i) {
        ctx.hierarchy.Reload();
        auto elements = ctx.hierarchy.FindByClass("app-bar-nav");
        if (!elements.empty() && elements[0].visible) {
            int x = elements[0].bounds.centerX();
            int y = elements[0].bounds.centerY();
            ScaleToPhysical(ctx, x, y);
            ctx.window.SendClick(x, y);
        }
        std::this_thread::sleep_for(std::chrono::milliseconds(400));
    }
    std::this_thread::sleep_for(std::chrono::milliseconds(800));
}

Registering Tests

In main.cpp:

TestRunner runner;
runner.SetDesignerPath(designerPath);
runner.SetDocumentPath(documentPath);
runner.SetLogPath(logPath);
runner.SetHierarchyPath(hierarchyPath);

// Register tests
runner.AddTest("Navigate to Dialer", TestNavigateToDialer);
runner.AddTest("Navigate to Messages", TestNavigateToMessages);
runner.AddTest("Navigate to Contacts", TestNavigateToContacts);
runner.AddTest("Navigate to Browser", TestNavigateToBrowser);
runner.AddTest("Navigation Sequence", TestNavigationSequence);

// Run all tests
auto results = runner.RunAll();
results.SaveToFile(resultsPath);

Test Results Format

{
  "name": "Mosis Designer UI Tests",
  "summary": {
    "passed": 5,
    "failed": 0,
    "skipped": 0,
    "errors": 0,
    "total": 5,
    "duration_ms": 31162.3
  },
  "tests": [
    {
      "name": "Navigate to Dialer",
      "status": "passed",
      "message": "Test passed",
      "duration_ms": 4216.88
    },
    {
      "name": "Navigate to Browser",
      "status": "passed",
      "message": "Test passed",
      "duration_ms": 4067.34
    }
  ]
}

Running Tests

Command Line

# Run with defaults (auto-detects paths)
designer-test.exe

# Override paths
designer-test.exe --designer /path/to/mosis-designer.exe \
                  --document /path/to/home.rml \
                  --log /path/to/output.log \
                  --hierarchy /path/to/hierarchy.json \
                  --results /path/to/results.json

Exit Codes

• 0: All tests passed
• 1: One or more tests failed

Best Practices

Element Identification

1. Use IDs for clickable elements that tests need to find
2. Use semantic classes like app-bar-nav for navigation buttons
3. Avoid relying on position - use hierarchy lookup instead

Timing

1. Wait after navigation (1000ms) for animations to complete
2. Reload hierarchy before each element lookup
3. Use retry logic for file reads (hierarchy may be mid-write)

Test Independence

1. Start from known state - call GoHome() at start of each test
2. Clear logs before verification
3. Don't depend on previous test state

Adding Test IDs to RML

When creating new screens, add IDs to interactive elements:

<!-- Good: Has ID for testing -->
<div id="dock-phone" class="dock-item" onclick="navigateTo('dialer')">
    <img src="../../icons/phone.tga"/>
</div>

<!-- Good: Has class for back button detection -->
<div class="app-bar-nav btn-icon" onclick="goBack()">
    <img src="../../icons/back.tga"/>
</div>

Troubleshooting

Element Not Found

• Verify the screen has loaded (check hierarchy screen field)
• Check if element has the expected ID/class
• Ensure element is visible (visible: true)

Click Not Registering

• Check coordinate scaling (hierarchy vs window size)
• Increase wait time for animations
• Verify window is in foreground

Timing Issues

• Increase delays after GoHome() and navigation
• Add retries for hierarchy reload
• Check for race conditions in file I/O

Android Testing

The Android app supports event injection for automated testing.

Event Injection Methods

The MainActivity provides these methods for programmatic testing:

// Inject touch events with normalized coordinates (0.0-1.0)
MainActivity.instance?.injectTouchDown(x, y)
MainActivity.instance?.injectTouchMove(x, y)
MainActivity.instance?.injectTouchUp(x, y)
MainActivity.instance?.injectClick(x, y)  // Down + delay + Up

Broadcast-Based Injection

For external test frameworks or ADB, send broadcasts:

# Inject a click at center (0.5, 0.5)
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" \
    --ef x 0.5 \
    --ef y 0.5

# Inject touch down
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "down" \
    --ef x 0.2 \
    --ef y 0.9

# Inject touch up
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "up" \
    --ef x 0.2 \
    --ef y 0.9

UI Element Coordinates

Dock items are positioned at the bottom of the screen. Approximate normalized positions:

Element	X	Y
dock-phone	0.16	0.97
dock-messages	0.39	0.97
dock-contacts	0.61	0.97
dock-browser	0.84	0.97

Instrumentation Tests

For full integration tests, use Android's instrumentation framework:

@RunWith(AndroidJUnit4::class)
class NavigationTest {
    @get:Rule
    val activityRule = ActivityScenarioRule(MainActivity::class.java)

    @Test
    fun testNavigateToDialer() {
        // Wait for service to connect
        Thread.sleep(2000)

        // Click on phone dock icon (normalized coords)
        activityRule.scenario.onActivity { activity ->
            activity.injectClick(0.16f, 0.97f)
        }

        // Wait for navigation
        Thread.sleep(1000)

        // Verify navigation occurred (check logs or UI state)
    }
}

Future Improvements

• Action recording (capture user interactions)
• Screenshot comparison (visual regression testing)
• Android instrumentation test suite
• Parallel test execution
• Test coverage reporting
• Cross-platform test runner (desktop + Android)