finish the testing framework
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Milestone 2: Testing Framework
|
||||
|
||||
**Status**: 95% Complete
|
||||
**Status**: 100% Complete
|
||||
**Goal**: Automated UI testing for rapid iteration and AI agent verification.
|
||||
|
||||
---
|
||||
@@ -95,44 +95,35 @@ mosis-designer.exe home.rml --playback my-test.json
|
||||
|
||||
---
|
||||
|
||||
## Remaining Task
|
||||
## Completed: GLFW Input Hooks for Recording
|
||||
|
||||
### Task 2.3: GLFW Input Hooks for Recording
|
||||
### Task 2.3: GLFW Input Hooks
|
||||
|
||||
**Status**: Partially Complete
|
||||
**Effort**: Medium
|
||||
**Limitation**: Requires RmlUi Backend modification
|
||||
**Status**: Complete
|
||||
|
||||
**Current State**:
|
||||
- Recording infrastructure is complete (ActionRecorder)
|
||||
- Playback works fully (ActionPlayer calls RmlUi context directly)
|
||||
- CLI and keyboard controls are wired up
|
||||
- **Missing**: Direct GLFW callback access for mouse recording
|
||||
**Solution Implemented**:
|
||||
Forked the RmlUi backend files into `designer/src/backend/` with input recording hooks:
|
||||
|
||||
**The Problem**:
|
||||
The RmlUi Backend abstraction handles all GLFW callbacks internally and doesn't expose them for interception. To record actual mouse events, we would need to either:
|
||||
1. **RmlUi_Backend.h** - Added callback type definitions:
|
||||
- `MouseButtonCallback` - Called on mouse button press/release
|
||||
- `MouseMoveCallback` - Called on mouse movement
|
||||
- `KeyCallback` - Called on key press/release
|
||||
|
||||
1. **Modify RmlUi Backend** (third-party code)
|
||||
- Add callback hooks to `RmlUi_Backend_GLFW_GL3.cpp`
|
||||
- Expose GLFW window handle for custom callbacks
|
||||
2. **RmlUi_Backend_GLFW_GL3.cpp** - Modified GLFW callbacks to:
|
||||
- Track mouse position in framebuffer coordinates
|
||||
- Call recording callbacks before forwarding to RmlUi
|
||||
- Support all three callback types
|
||||
|
||||
2. **Fork RmlUi Backends** (more maintainable)
|
||||
- Copy Backend files into designer project
|
||||
- Add recording hooks
|
||||
3. **main.cpp** - Connected callbacks to ActionRecorder:
|
||||
- Mouse button events trigger `RecordMouseDown`/`RecordMouseUp`
|
||||
- Mouse move events trigger `RecordMouseMove`
|
||||
- Key events trigger `RecordKey`
|
||||
|
||||
3. **Alternative: Element-Based Recording**
|
||||
- Listen to RmlUi events after processing
|
||||
- Record element clicks by ID rather than coordinates
|
||||
- Less precise but avoids backend modification
|
||||
|
||||
**Workaround for Now**:
|
||||
Tests can be created manually by:
|
||||
1. Using the UI hierarchy to find element coordinates
|
||||
2. Writing JSON test files directly
|
||||
3. Using the external designer-test framework (Windows SendInput)
|
||||
|
||||
**Future Work**:
|
||||
Consider option 2 (fork backends) when recording becomes a priority.
|
||||
**Files Added**:
|
||||
- `designer/src/backend/RmlUi_Backend.h`
|
||||
- `designer/src/backend/RmlUi_Backend_GLFW_GL3.cpp`
|
||||
- `designer/src/backend/RmlUi_Platform_GLFW.h`
|
||||
- `designer/src/backend/RmlUi_Platform_GLFW.cpp`
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user