# Milestone 2: Testing Framework

**Status**: 95% Complete
**Goal**: Automated UI testing for rapid iteration and AI agent verification.

---

## Overview

The testing framework enables automated validation of UI behavior through:
- UI hierarchy inspection (JSON dump)
- Input simulation and recording
- Log verification for navigation events
- Visual regression testing via screenshot diff
- JSON test results compatible with CI/CD

---

## Current State

### Completed Components

| Component | File | Status |
|-----------|------|--------|
| Action Types | `designer/src/testing/action_types.h` | Complete |
| Action Recorder | `designer/src/testing/action_recorder.h/.cpp` | Complete |
| Action Player | `designer/src/testing/action_player.h/.cpp` | Complete |
| UI Inspector | `designer/src/testing/ui_inspector.cpp` | Complete |
| Visual Capture | `designer/src/testing/visual_capture.cpp` | Complete |
| Test Runner | `designer-test/src/test_runner.cpp` | Complete |
| CLI Integration | `designer/src/main.cpp` | Complete |

### Already Working

1. **Action Types** (`action_types.h`)
   - TapAction, SwipeAction, LongPressAction
   - ButtonAction, WaitAction, KeyAction
   - ActionSequence container with metadata

2. **Action Recorder** (`action_recorder.h/.cpp`)
   - Mouse down/up/move tracking
   - Automatic gesture classification (tap vs swipe vs long press)
   - JSON serialization/deserialization
   - Configurable thresholds

3. **Action Player** (`action_player.h/.cpp`)
   - Timestamp-based playback
   - RmlUi event injection
   - Pause/resume/stop controls
   - Progress tracking

4. **UI Inspector** (`ui_inspector.cpp`)
   - Full element tree traversal
   - JSON export with bounds, IDs, classes
   - Continuous hierarchy dumping mode

5. **Test Runner** (`designer-test/`)
   - WindowController (Windows SendInput)
   - HierarchyReader (element lookup)
   - LogParser (navigation verification)
   - All 5 navigation tests passing

---

## Recently Completed

### Screenshot Diff Implementation (Complete)

**File**: `designer/src/testing/visual_capture.cpp`

Implemented real PNG pixel-by-pixel comparison:
- `LoadPNG()` helper function loads PNG files using libpng
- `PixelsMatch()` compares pixels with configurable tolerance (default: 2)
- `CompareImages()` returns difference ratio (0.0 = identical, 1.0 = completely different)

### Recording/Playback CLI (Complete)

**File**: `designer/src/main.cpp`

Added CLI options and keyboard controls:
- `--record <file>` - Enable recording mode
- `--playback <file>` - Play back recorded actions
- F5 - Start/stop recording (saves to specified file)
- F6 - Pause/resume playback

**Usage**:
```bash
# Record interactions
mosis-designer.exe home.rml --record my-test.json
# Press F5 to start recording, interact, press F5 again to save

# Play back
mosis-designer.exe home.rml --playback my-test.json
```

---

## Remaining Task

### Task 2.3: GLFW Input Hooks for Recording

**Status**: Partially Complete
**Effort**: Medium
**Limitation**: Requires RmlUi Backend modification

**Current State**:
- Recording infrastructure is complete (ActionRecorder)
- Playback works fully (ActionPlayer calls RmlUi context directly)
- CLI and keyboard controls are wired up
- **Missing**: Direct GLFW callback access for mouse recording

**The Problem**:
The RmlUi Backend abstraction handles all GLFW callbacks internally and doesn't expose them for interception. To record actual mouse events, we would need to either:

1. **Modify RmlUi Backend** (third-party code)
   - Add callback hooks to `RmlUi_Backend_GLFW_GL3.cpp`
   - Expose GLFW window handle for custom callbacks

2. **Fork RmlUi Backends** (more maintainable)
   - Copy Backend files into designer project
   - Add recording hooks

3. **Alternative: Element-Based Recording**
   - Listen to RmlUi events after processing
   - Record element clicks by ID rather than coordinates
   - Less precise but avoids backend modification

**Workaround for Now**:
Tests can be created manually by:
1. Using the UI hierarchy to find element coordinates
2. Writing JSON test files directly
3. Using the external designer-test framework (Windows SendInput)

**Future Work**:
Consider option 2 (fork backends) when recording becomes a priority.

---

## File Changes Summary

| File | Changes | Status |
|------|---------|--------|
| `designer/src/testing/visual_capture.cpp` | Real PNG pixel comparison | Done |
| `designer/src/main.cpp` | --record/--playback CLI, F5/F6 keys | Done |
| `designer/src/RmlUi_Backend.h` | Expose GLFW window for recording | Future |

---

## Test Recording Format

```json
{
  "name": "Navigate to contacts and back",
  "description": "Test navigation flow",
  "screen_width": 540,
  "screen_height": 960,
  "initial_screen": "apps/home/home.rml",
  "actions": [
    {"type": "tap", "x": 413, "y": 1174, "timestamp": 0},
    {"type": "wait", "duration": 1000, "timestamp": 100},
    {"type": "tap", "x": 40, "y": 28, "timestamp": 1100},
    {"type": "swipe", "x1": 100, "y1": 500, "x2": 100, "y2": 200, "duration": 300, "timestamp": 2000}
  ]
}
```

---

## Acceptance Criteria

- [x] `CompareImages()` returns accurate pixel difference ratio
- [ ] `--record <file>` captures all mouse/key events to JSON (needs GLFW hooks)
- [x] `--playback <file>` replays recorded actions with correct timing
- [x] Recording stops gracefully on F5 or window close
- [x] Playback shows progress in console
- [x] Screenshot diff with 2-pixel tolerance per channel

---

## Future Enhancements (Not This Milestone)

- Visual diff output (highlight changed pixels)
- Parallel test execution
- Android action recording/playback
- Cross-platform test runner
- Coverage reporting