MosisService/docs/TESTING-FRAMEWORK.md

Automated Testing Framework

The Mosis Designer supports two testing approaches:

1. JSON Action Playback - Built into the designer for scripted UI testing
2. C++ Test Framework - External test runner in designer-test/ for programmatic testing

---

JSON Action Playback

The designer can record and playback UI interactions using JSON files. This is useful for:

• Automated screenshot capture
• Regression testing
• Demo recordings
• UI verification

Command-Line Options

# Run with action playback
mosis-designer.exe --simulator --test-apps base-apps \
    --playback test_navigation.json \
    --screenshot-after result.png \
    --hierarchy hierarchy.json

Flag	Description
--playback <file>	JSON file containing actions to replay
--screenshot-after <file>	Save screenshot after playback completes
--hierarchy <file>	Dump UI hierarchy to JSON after playback

Recording Actions

Press R in the designer to start/stop recording. Actions are saved to recorded_actions.json.

Action Types

Tap

Single tap at coordinates.

{
  "type": "tap",
  "x": 270,
  "y": 480,
  "timestamp": 1000
}

Swipe

Drag from one point to another.

{
  "type": "swipe",
  "x1": 270,
  "y1": 600,
  "x2": 270,
  "y2": 200,
  "duration": 300,
  "timestamp": 2000
}

Long Press

Press and hold at coordinates.

{
  "type": "long_press",
  "x": 270,
  "y": 480,
  "duration": 800,
  "timestamp": 3000
}

Button

System button press (back, home, recents).

{
  "type": "button",
  "button": "back",
  "timestamp": 4000
}

Wait

Pause execution for a duration.

{
  "type": "wait",
  "duration": 1500,
  "timestamp": 5000
}

Key

Raw keyboard input (for special keys).

{
  "type": "key",
  "key_code": 256,
  "pressed": true,
  "timestamp": 6000
}

Common key codes:

• 256 - Escape (Back)
• 301 - F12 (Debugger)
• 294 - F5 (Reload)

Complete Test File Example

{
  "name": "Navigate to Messages",
  "description": "Test navigation from home to Messages app",
  "screen_width": 540,
  "screen_height": 960,
  "initial_screen": "",
  "actions": [
    {
      "type": "wait",
      "duration": 500,
      "timestamp": 0
    },
    {
      "type": "tap",
      "x": 207,
      "y": 220,
      "timestamp": 500
    },
    {
      "type": "wait",
      "duration": 1500,
      "timestamp": 600
    }
  ]
}

Finding Tap Coordinates

Use the --hierarchy flag to dump UI element bounds:

mosis-designer.exe --simulator --test-apps base-apps \
    --playback wait_only.json \
    --hierarchy hierarchy.json

The hierarchy JSON contains element bounds:

{
  "bounds": {
    "x": 171.5,
    "y": 183.2,
    "width": 72.0,
    "height": 72.0
  },
  "classes": ["app-icon-image"],
  "tag": "div"
}

Calculate tap center: x = bounds.x + width/2, y = bounds.y + height/2

Home Screen Grid Layout

For the default 540x960 resolution with 4-column grid:

Row	Y Range	Center Y
1	64-167	~115
2	183-286	~220
3	302-405	~350

Column	X Range	Center X
1	20-145	~82
2	145-270	~207
3	270-395	~332
4	395-520	~457

Dock items are at Y ~910.

---

Asset Path Requirements

When testing apps in base-apps/, ensure shared assets are accessible. Apps use relative paths like ../../ui/html.rcss.

From base-apps/com.mosis.app/, the path ../../ resolves to MosisService/.

Required directories at MosisService/ root:

MosisService/
├── ui/           # html.rcss, theme.rcss, components.rcss, layout.rcss
├── scripts/      # navigation.lua, layout.lua
├── icons/        # Shared icon assets
└── base-apps/    # Test apps

Copy from src/main/assets/:

cp -r src/main/assets/ui .
cp -r src/main/assets/scripts .
cp -r src/main/assets/icons .

---

C++ Test Framework

The designer-test/ project provides programmatic testing with element finding.

Architecture

Component	Purpose
WindowController	Sends mouse/keyboard via Windows SendInput API
HierarchyReader	Parses UI hierarchy JSON, finds elements by ID/class
LogParser	Monitors log file for navigation events
TestRunner	Orchestrates execution, reports results
UIInspector	Dumps hierarchy with atomic writes

Key Implementation Details

• Path Normalization: RmlUi uses | instead of : in Windows paths
• Atomic File Writes: Hierarchy written to .tmp then renamed
• Retry with Backoff: HierarchyReader retries up to 10 times with exponential backoff
• Dynamic Back Button: Finds back buttons by class instead of fixed coordinates

Writing C++ Tests

bool ClickById(TestContext& ctx, const std::string& id) {
    ctx.hierarchy.Reload();
    auto element = ctx.hierarchy.FindById(id);
    if (!element) return false;

    int x = element->bounds.centerX();
    int y = element->bounds.centerY();
    ScaleToPhysical(ctx, x, y);
    ctx.window.SendClick(x, y);
    return true;
}

bool TestNavigateToDialer(TestContext& ctx) {
    GoHome(ctx);
    ctx.log.Clear();

    if (!ClickById(ctx, "dock-phone")) return false;

    std::this_thread::sleep_for(std::chrono::milliseconds(1000));
    ctx.log.Reload();
    return ctx.log.Contains("Loaded screen: apps/dialer/dialer.rml");
}

Test Output

Results saved to test_results.json:

{
  "name": "Mosis Designer UI Tests",
  "summary": {"passed": 5, "failed": 0, "total": 5},
  "tests": [
    {"name": "Navigate to Dialer", "status": "passed", "duration_ms": 3500}
  ]
}

---

Example Test Suite

Test files for common scenarios:

test_home_only.json

{
  "name": "Home Screenshot",
  "description": "Capture home screen",
  "screen_width": 540,
  "screen_height": 960,
  "actions": [
    {"type": "wait", "duration": 500, "timestamp": 0}
  ]
}

test_settings.json

{
  "name": "Navigate to Settings",
  "description": "Tap Settings app (Row 3, Col 1)",
  "screen_width": 540,
  "screen_height": 960,
  "actions": [
    {"type": "wait", "duration": 500, "timestamp": 0},
    {"type": "tap", "x": 82, "y": 350, "timestamp": 500},
    {"type": "wait", "duration": 1500, "timestamp": 600}
  ]
}

test_browser.json

{
  "name": "Navigate to Browser",
  "description": "Tap Browser in dock (Col 4)",
  "screen_width": 540,
  "screen_height": 960,
  "actions": [
    {"type": "wait", "duration": 500, "timestamp": 0},
    {"type": "tap", "x": 445, "y": 910, "timestamp": 500},
    {"type": "wait", "duration": 1500, "timestamp": 600}
  ]
}

Running Tests

# Single test with screenshot
mosis-designer.exe --simulator --test-apps base-apps \
    --playback test_settings.json \
    --screenshot-after screenshot_settings.png

# Test with hierarchy dump for debugging
mosis-designer.exe --simulator --test-apps base-apps \
    --playback test_home_only.json \
    --screenshot-after home.png \
    --hierarchy hierarchy.json

---

Troubleshooting

Tap not registering

• Check coordinates against hierarchy bounds
• Ensure tap is within the clickable element (e.g., app-icon-image, not app-icon-label)
• Use hierarchy dump to verify element positions

Stylesheet errors

[ERROR] Failed to load style sheet D|/Dev/.../ui/html.rcss

• Copy shared assets to MosisService root (see Asset Path Requirements)

App not loading

• Check console for launchApp call
• Verify manifest.json exists in app directory
• Check entry file path in manifest

Screenshot shows wrong screen

• Increase wait duration after tap
• Verify navigation completed in console output