finish the testing framework

This commit is contained in:
2026-01-16 20:15:34 +01:00
parent 2e097e4e54
commit 8de36aa975
10 changed files with 1044 additions and 62 deletions

View File

@@ -1,6 +1,6 @@
# Milestone 2: Testing Framework
**Status**: 95% Complete
**Status**: 100% Complete
**Goal**: Automated UI testing for rapid iteration and AI agent verification.
---
@@ -95,44 +95,35 @@ mosis-designer.exe home.rml --playback my-test.json
---
## Remaining Task
## Completed: GLFW Input Hooks for Recording
### Task 2.3: GLFW Input Hooks for Recording
### Task 2.3: GLFW Input Hooks
**Status**: Partially Complete
**Effort**: Medium
**Limitation**: Requires RmlUi Backend modification
**Status**: Complete
**Current State**:
- Recording infrastructure is complete (ActionRecorder)
- Playback works fully (ActionPlayer calls RmlUi context directly)
- CLI and keyboard controls are wired up
- **Missing**: Direct GLFW callback access for mouse recording
**Solution Implemented**:
Forked the RmlUi backend files into `designer/src/backend/` with input recording hooks:
**The Problem**:
The RmlUi Backend abstraction handles all GLFW callbacks internally and doesn't expose them for interception. To record actual mouse events, we would need to either:
1. **RmlUi_Backend.h** - Added callback type definitions:
- `MouseButtonCallback` - Called on mouse button press/release
- `MouseMoveCallback` - Called on mouse movement
- `KeyCallback` - Called on key press/release
1. **Modify RmlUi Backend** (third-party code)
- Add callback hooks to `RmlUi_Backend_GLFW_GL3.cpp`
- Expose GLFW window handle for custom callbacks
2. **RmlUi_Backend_GLFW_GL3.cpp** - Modified GLFW callbacks to:
- Track mouse position in framebuffer coordinates
- Call recording callbacks before forwarding to RmlUi
- Support all three callback types
2. **Fork RmlUi Backends** (more maintainable)
- Copy Backend files into designer project
- Add recording hooks
3. **main.cpp** - Connected callbacks to ActionRecorder:
- Mouse button events trigger `RecordMouseDown`/`RecordMouseUp`
- Mouse move events trigger `RecordMouseMove`
- Key events trigger `RecordKey`
3. **Alternative: Element-Based Recording**
- Listen to RmlUi events after processing
- Record element clicks by ID rather than coordinates
- Less precise but avoids backend modification
**Workaround for Now**:
Tests can be created manually by:
1. Using the UI hierarchy to find element coordinates
2. Writing JSON test files directly
3. Using the external designer-test framework (Windows SendInput)
**Future Work**:
Consider option 2 (fork backends) when recording becomes a priority.
**Files Added**:
- `designer/src/backend/RmlUi_Backend.h`
- `designer/src/backend/RmlUi_Backend_GLFW_GL3.cpp`
- `designer/src/backend/RmlUi_Platform_GLFW.h`
- `designer/src/backend/RmlUi_Platform_GLFW.cpp`
---