complete milestone 1

This commit is contained in:
2026-01-16 22:17:12 +01:00
parent 8de36aa975
commit baa77d4c95
8 changed files with 524 additions and 78 deletions

110
CLAUDE.md
View File

@@ -186,9 +186,17 @@ The designer-test (`designer-test/`) provides automated UI testing:
### Test Architecture ### Test Architecture
1. **WindowController**: Finds designer window, sends mouse/keyboard events via Windows SendInput API 1. **WindowController**: Finds designer window, sends mouse/keyboard events via Windows SendInput API
2. **HierarchyReader**: Parses UI hierarchy JSON to find elements by ID/class 2. **HierarchyReader**: Parses UI hierarchy JSON to find elements by ID/class (with retry logic and exponential backoff)
3. **LogParser**: Monitors log file for navigation events 3. **LogParser**: Monitors log file for navigation events
4. **TestRunner**: Orchestrates test execution, reports results 4. **TestRunner**: Orchestrates test execution, reports results
5. **UIInspector**: Dumps UI hierarchy with atomic writes (temp file + rename pattern)
### Key Implementation Details
- **Path Normalization**: RmlUi uses `|` instead of `:` in Windows paths (e.g., `D|\Dev\...`). The UIInspector normalizes paths for correct document matching.
- **Atomic File Writes**: Hierarchy files are written to `.tmp` then renamed to prevent partial reads.
- **Retry with Backoff**: HierarchyReader retries up to 10 times with exponential backoff (30ms base) and validates JSON completeness.
- **Dynamic Back Button**: `GoHome()` finds back buttons from hierarchy by class (`app-bar-nav` or `browser-nav-btn`) instead of fixed coordinates.
### Writing Tests ### Writing Tests
@@ -291,6 +299,15 @@ Back buttons use `app-bar-nav` class for automated GoHome:
</div> </div>
``` ```
Browser uses `browser-nav-btn` class for its toolbar back button:
```html
<div class="app-bar-nav browser-nav-btn" onclick="goBack()">
<img src="../../icons/back.tga"/>
</div>
```
The test framework's `FindBackButton()` searches for both classes to handle all screen layouts.
## Material Design Resources ## Material Design Resources
Material Design icons and components are available in the MosisDesigner repository: Material Design icons and components are available in the MosisDesigner repository:
@@ -373,6 +390,36 @@ CSS/JS component library implementing Material Design (reference implementation)
## Android Device Testing ## Android Device Testing
### Prerequisites
```bash
# Check connected device
adb devices -l
# Verify Mosis app is installed
adb shell pm list packages | grep mosis
```
### Build and Install
```bash
# Build debug APK
./gradlew assembleDebug
# Install on device
adb install -r build/outputs/apk/debug/MosisService-debug.apk
# Launch the app
adb shell am start -n com.omixlab.mosis/.MainActivity
```
### Run Gradle Connected Tests
```bash
# Run all connected Android tests
./gradlew connectedAndroidTest
```
### Event Injection via ADB ### Event Injection via ADB
Inject touch events for automated testing: Inject touch events for automated testing:
@@ -399,6 +446,48 @@ adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
| dock-messages | 0.39 | 0.97 | | dock-messages | 0.39 | 0.97 |
| dock-contacts | 0.61 | 0.97 | | dock-contacts | 0.61 | 0.97 |
| dock-browser | 0.84 | 0.97 | | dock-browser | 0.84 | 0.97 |
| back-button | 0.10 | 0.05 |
### Full Navigation Test Sequence
```bash
# Clear logs and run navigation test sequence
adb logcat -c
# Click Phone dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.16 --ef y 0.97
sleep 2
# Click back to return home
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.1 --ef y 0.05
sleep 2
# Click Messages dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.39 --ef y 0.97
sleep 2
# Click back to return home
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.1 --ef y 0.05
sleep 2
# Click Contacts dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.61 --ef y 0.97
sleep 2
# Click back to return home
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.1 --ef y 0.05
sleep 2
# Click Browser dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.84 --ef y 0.97
```
### Reading Logs ### Reading Logs
@@ -406,6 +495,25 @@ adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
# Filter for Mosis logs # Filter for Mosis logs
adb logcat -s MosisTest ServiceTester RMLUI adb logcat -s MosisTest ServiceTester RMLUI
# Filter for navigation events
adb logcat -d | grep -iE "navigat|loaded|goBack|rml"
# Save to file # Save to file
adb logcat -s MosisTest > mosis-log.txt adb logcat -s MosisTest > mosis-log.txt
# Clear logs
adb logcat -c
```
### Expected Log Output
Successful navigation shows these log patterns:
```
RMLUI: navigateTo called with: dialer
Loading screen: apps/dialer/dialer.rml
RMLUI: Navigated to: dialer (history depth: 1)
RMLUI: goBack called (history depth: 1)
Loading screen: apps/home/home.rml
RMLUI: Back to: home
``` ```

View File

@@ -17,25 +17,56 @@ bool HierarchyReader::Reload() {
return false; return false;
} }
// Retry a few times with delay to handle file write race conditions // Retry with increasing delays to handle file write race conditions
for (int attempt = 0; attempt < 5; ++attempt) { const int maxAttempts = 10;
const int baseDelayMs = 30;
for (int attempt = 0; attempt < maxAttempts; ++attempt) {
if (attempt > 0) { if (attempt > 0) {
std::this_thread::sleep_for(std::chrono::milliseconds(50)); // Exponential backoff: 30, 60, 120, ... ms
int delayMs = baseDelayMs * (1 << std::min(attempt - 1, 4));
std::this_thread::sleep_for(std::chrono::milliseconds(delayMs));
} }
std::ifstream file(m_path); // Read entire file into string first (to avoid partial reads)
if (!file.is_open()) { std::string content;
continue; // File might not exist yet, retry {
std::ifstream file(m_path, std::ios::binary);
if (!file.is_open()) {
continue; // File might not exist yet, retry
}
// Get file size
file.seekg(0, std::ios::end);
auto size = file.tellg();
if (size <= 0) {
continue; // Empty file, retry
}
file.seekg(0, std::ios::beg);
// Read content
content.resize(static_cast<size_t>(size));
file.read(content.data(), size);
if (!file.good()) {
continue; // Read error, retry
}
}
// Check for valid JSON ending (should end with "}")
size_t lastNonSpace = content.find_last_not_of(" \t\n\r");
if (lastNonSpace == std::string::npos || content[lastNonSpace] != '}') {
continue; // File is incomplete, retry
} }
try { try {
m_json = nlohmann::json::parse(file); m_json = nlohmann::json::parse(content);
m_loaded = true; m_loaded = true;
return true; return true;
} catch (const nlohmann::json::parse_error& e) { } catch (const nlohmann::json::parse_error& e) {
// File might be partially written, retry // File might be partially written, retry
if (attempt == 4) { if (attempt == maxAttempts - 1) {
std::cerr << "Failed to parse hierarchy JSON after 5 attempts: " << e.what() << std::endl; std::cerr << "Failed to parse hierarchy JSON after " << maxAttempts << " attempts: " << e.what() << std::endl;
} }
} }
} }

View File

@@ -6,6 +6,7 @@
#include <filesystem> #include <filesystem>
#include <thread> #include <thread>
#include <chrono> #include <chrono>
#include <optional>
using namespace mosis::test; using namespace mosis::test;
@@ -104,26 +105,88 @@ bool ClickByClassIndex(TestContext& ctx, const std::string& className, int index
return false; return false;
} }
// Helper: Wait for hierarchy to update to expected screen
bool WaitForScreen(TestContext& ctx, const std::string& expectedScreen, int timeoutMs = 3000) {
auto startTime = std::chrono::steady_clock::now();
while (true) {
std::this_thread::sleep_for(std::chrono::milliseconds(100));
ctx.hierarchy.Reload();
std::string currentScreen = ctx.hierarchy.GetScreenName();
if (currentScreen.find(expectedScreen) != std::string::npos) {
return true;
}
auto elapsed = std::chrono::steady_clock::now() - startTime;
if (std::chrono::duration_cast<std::chrono::milliseconds>(elapsed).count() >= timeoutMs) {
return false;
}
}
}
// Helper: Find a back button in the hierarchy
std::optional<ElementInfo> FindBackButton(TestContext& ctx) {
// Try to find back button by class (app-bar-nav is used in most screens)
auto elements = ctx.hierarchy.FindByClass("app-bar-nav");
for (const auto& elem : elements) {
if (elem.visible && elem.bounds.width > 0 && elem.bounds.height > 0) {
return elem;
}
}
// Try browser-specific back button
elements = ctx.hierarchy.FindByClass("browser-nav-btn");
for (const auto& elem : elements) {
if (elem.visible && elem.bounds.width > 0 && elem.bounds.height > 0) {
return elem;
}
}
return std::nullopt;
}
// Helper: Go back to home screen by clicking back button multiple times // Helper: Go back to home screen by clicking back button multiple times
void GoHome(TestContext& ctx) { void GoHome(TestContext& ctx) {
// Click back button (app-bar-nav) up to 5 times to ensure we're at home // Wait longer for hierarchy file to be updated from any previous navigation
// The designer writes hierarchy every frame, but there can be a race condition
std::this_thread::sleep_for(std::chrono::milliseconds(1500));
for (int i = 0; i < 5; ++i) { for (int i = 0; i < 5; ++i) {
// Wait then reload to get fresh hierarchy data
std::this_thread::sleep_for(std::chrono::milliseconds(300));
ctx.hierarchy.Reload(); ctx.hierarchy.Reload();
// Look for the back button by class
auto elements = ctx.hierarchy.FindByClass("app-bar-nav"); // Check hierarchy to see if dock-phone exists (indicating home screen)
if (!elements.empty()) { auto dockPhone = ctx.hierarchy.FindById("dock-phone");
auto& btn = elements[0]; if (dockPhone && dockPhone->visible) {
if (btn.visible && btn.bounds.width > 0) { std::cout << " GoHome: At home screen (dock-phone found)" << std::endl;
int x = btn.bounds.centerX(); break;
int y = btn.bounds.centerY();
ScaleToPhysical(ctx, x, y);
ctx.window.SendClick(x, y);
}
} }
std::this_thread::sleep_for(std::chrono::milliseconds(400));
// Find back button from hierarchy
auto backBtn = FindBackButton(ctx);
int x, y;
if (backBtn) {
x = backBtn->bounds.centerX();
y = backBtn->bounds.centerY();
std::cout << " GoHome: Found back button at (" << x << "," << y << ")" << std::endl;
} else {
// Fallback to default position if no back button found
x = 48;
y = 36;
std::cout << " GoHome: No back button found, using default position" << std::endl;
}
ScaleToPhysical(ctx, x, y);
std::cout << " GoHome: Clicking back at (" << x << "," << y << ")" << std::endl;
ctx.window.SendClick(x, y);
// Wait for navigation animation to complete
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
} }
// Extra wait for animations to fully complete and hierarchy to update
std::this_thread::sleep_for(std::chrono::milliseconds(800)); // Final verification - wait and reload hierarchy
std::this_thread::sleep_for(std::chrono::milliseconds(500));
ctx.hierarchy.Reload();
std::cout << " GoHome: Final screen = " << ctx.hierarchy.GetScreenName() << std::endl;
} }
// Test: Navigate to dialer by clicking Phone dock icon // Test: Navigate to dialer by clicking Phone dock icon

View File

@@ -6,8 +6,35 @@
namespace mosis::test { namespace mosis::test {
// Callback for EnumWindows to find window by partial title match
struct FindWindowData {
std::string searchTitle;
HWND foundHwnd = nullptr;
};
static BOOL CALLBACK EnumWindowsCallback(HWND hwnd, LPARAM lParam) {
auto* data = reinterpret_cast<FindWindowData*>(lParam);
char title[256] = {0};
GetWindowTextA(hwnd, title, sizeof(title));
// Check if the window title contains our search string
if (std::string(title).find(data->searchTitle) != std::string::npos) {
data->foundHwnd = hwnd;
return FALSE; // Stop enumeration
}
return TRUE; // Continue enumeration
}
bool WindowController::FindWindow(const std::string& title) { bool WindowController::FindWindow(const std::string& title) {
m_hwnd = ::FindWindowA(nullptr, title.c_str()); // Use EnumWindows to find window by partial title match
// This allows finding "Mosis Designer - home.rml" when searching for "Mosis Designer"
FindWindowData data;
data.searchTitle = title;
EnumWindows(EnumWindowsCallback, reinterpret_cast<LPARAM>(&data));
m_hwnd = data.foundHwnd;
if (!m_hwnd) { if (!m_hwnd) {
return false; return false;
} }
@@ -80,24 +107,33 @@ LPARAM WindowController::PhoneToClientLParam(int phoneX, int phoneY) {
bool WindowController::SendMouseDown(int phoneX, int phoneY) { bool WindowController::SendMouseDown(int phoneX, int phoneY) {
if (!m_hwnd) return false; if (!m_hwnd) return false;
// Convert to screen coordinates for SendInput // Convert phone coordinates to client coordinates
int clientX = static_cast<int>(phoneX * m_info.scaleX); int clientX = static_cast<int>(phoneX * m_info.scaleX);
int clientY = static_cast<int>(phoneY * m_info.scaleY); int clientY = static_cast<int>(phoneY * m_info.scaleY);
// Get DPI info for debugging
UINT dpi = GetDpiForWindow(m_hwnd);
// Calculate screen coordinates from client position
// On DPI-aware systems, Windows APIs return consistent coordinate spaces
int screenX = m_info.clientX + clientX; int screenX = m_info.clientX + clientX;
int screenY = m_info.clientY + clientY; int screenY = m_info.clientY + clientY;
// Use SendInput for proper GLFW compatibility // Ensure window is foreground before clicking
// First move the cursor to the position SetForegroundWindow(m_hwnd);
SetCursorPos(screenX, screenY); Sleep(10); // Small delay
// Use SendInput for GLFW compatibility
SetCursorPos(screenX, screenY);
Sleep(10); // Small delay for cursor move
// Send mouse down via SendInput
INPUT input = {}; INPUT input = {};
input.type = INPUT_MOUSE; input.type = INPUT_MOUSE;
input.mi.dwFlags = MOUSEEVENTF_LEFTDOWN; input.mi.dwFlags = MOUSEEVENTF_LEFTDOWN;
SendInput(1, &input, sizeof(INPUT)); SendInput(1, &input, sizeof(INPUT));
std::cout << "MouseDown at phone(" << phoneX << "," << phoneY << ") -> screen(" std::cout << "MouseDown at phone(" << phoneX << "," << phoneY << ") -> screen("
<< screenX << "," << screenY << ")" << std::endl; << screenX << "," << screenY << ") dpi=" << dpi << std::endl;
return true; return true;
} }
@@ -105,12 +141,15 @@ bool WindowController::SendMouseDown(int phoneX, int phoneY) {
bool WindowController::SendMouseUp(int phoneX, int phoneY) { bool WindowController::SendMouseUp(int phoneX, int phoneY) {
if (!m_hwnd) return false; if (!m_hwnd) return false;
// Send mouse up via SendInput Sleep(10); // Small delay before release
INPUT input = {}; INPUT input = {};
input.type = INPUT_MOUSE; input.type = INPUT_MOUSE;
input.mi.dwFlags = MOUSEEVENTF_LEFTUP; input.mi.dwFlags = MOUSEEVENTF_LEFTUP;
SendInput(1, &input, sizeof(INPUT)); SendInput(1, &input, sizeof(INPUT));
Sleep(10); // Small delay after release
std::cout << "MouseUp at phone(" << phoneX << "," << phoneY << ")" << std::endl; std::cout << "MouseUp at phone(" << phoneX << "," << phoneY << ")" << std::endl;
return true; return true;

View File

@@ -17,6 +17,7 @@
#include "testing/visual_capture.h" #include "testing/visual_capture.h"
#include <RmlUi/Lua/Interpreter.h> #include <RmlUi/Lua/Interpreter.h>
#include <iostream> #include <iostream>
#include <fstream>
#include <filesystem> #include <filesystem>
#include <memory> #include <memory>
#include <chrono> #include <chrono>
@@ -31,6 +32,7 @@ static RenderInterface_GL3* g_render_interface = nullptr;
static mosis::DesktopPlatform* g_platform = nullptr; static mosis::DesktopPlatform* g_platform = nullptr;
static mosis::HotReload* g_hot_reload = nullptr; static mosis::HotReload* g_hot_reload = nullptr;
static std::string g_current_document_path; static std::string g_current_document_path;
static std::string g_current_screen_url; // For hierarchy dump - tracks current screen
static bool g_needs_reload = false; static bool g_needs_reload = false;
// Resolution presets // Resolution presets
@@ -52,8 +54,13 @@ static std::string g_test_output_path; // Output file for record/screenshot/hie
static mosis::testing::ActionRecorder* g_action_recorder = nullptr; static mosis::testing::ActionRecorder* g_action_recorder = nullptr;
static mosis::testing::ActionPlayer* g_action_player = nullptr; static mosis::testing::ActionPlayer* g_action_player = nullptr;
// Logging and hierarchy dump for testing
static std::string g_log_file_path; // Log file path (--log)
static std::string g_hierarchy_file_path; // Continuous hierarchy dump path (--hierarchy)
static std::ofstream g_log_file;
// Forward declarations // Forward declarations
bool InitializeRmlUi(const std::string& assets_path); bool InitializeRmlUi(const std::string& assets_path, int fb_width, int fb_height);
void ShutdownRmlUi(); void ShutdownRmlUi();
bool LoadDocument(const std::string& path); bool LoadDocument(const std::string& path);
void ReloadDocument(); void ReloadDocument();
@@ -119,25 +126,36 @@ static void MouseButtonCallback(GLFWwindow* window, int button, int action, int
double xpos, ypos; double xpos, ypos;
glfwGetCursorPos(window, &xpos, &ypos); glfwGetCursorPos(window, &xpos, &ypos);
// Convert from physical (GLFW) to logical (RmlUi) coordinates
// GLFW reports physical pixels, but RmlUi context uses logical pixels
float scaleX, scaleY;
glfwGetWindowContentScale(window, &scaleX, &scaleY);
int logicalX = static_cast<int>(xpos / scaleX);
int logicalY = static_cast<int>(ypos / scaleY);
// Debug log for click detection
Rml::Log::Message(Rml::Log::LT_INFO, "Mouse %s at physical(%.0f, %.0f) logical(%d, %d) button=%d",
action == GLFW_PRESS ? "down" : "up", xpos, ypos, logicalX, logicalY, button);
int key_modifier = 0; int key_modifier = 0;
if (mods & GLFW_MOD_CONTROL) key_modifier |= Rml::Input::KM_CTRL; if (mods & GLFW_MOD_CONTROL) key_modifier |= Rml::Input::KM_CTRL;
if (mods & GLFW_MOD_SHIFT) key_modifier |= Rml::Input::KM_SHIFT; if (mods & GLFW_MOD_SHIFT) key_modifier |= Rml::Input::KM_SHIFT;
if (mods & GLFW_MOD_ALT) key_modifier |= Rml::Input::KM_ALT; if (mods & GLFW_MOD_ALT) key_modifier |= Rml::Input::KM_ALT;
// Update mouse position before processing button event // Update mouse position before processing button event (using logical coords)
g_context->ProcessMouseMove(static_cast<int>(xpos), static_cast<int>(ypos), key_modifier); g_context->ProcessMouseMove(logicalX, logicalY, key_modifier);
if (button == GLFW_MOUSE_BUTTON_LEFT) { if (button == GLFW_MOUSE_BUTTON_LEFT) {
if (action == GLFW_PRESS) { if (action == GLFW_PRESS) {
// Record mouse down in record mode // Record mouse down in record mode (use logical coords)
if (g_action_recorder && g_action_recorder->IsRecording()) { if (g_action_recorder && g_action_recorder->IsRecording()) {
g_action_recorder->RecordMouseDown(static_cast<int>(xpos), static_cast<int>(ypos)); g_action_recorder->RecordMouseDown(logicalX, logicalY);
} }
g_context->ProcessMouseButtonDown(0, key_modifier); g_context->ProcessMouseButtonDown(0, key_modifier);
} else if (action == GLFW_RELEASE) { } else if (action == GLFW_RELEASE) {
// Record mouse up in record mode // Record mouse up in record mode
if (g_action_recorder && g_action_recorder->IsRecording()) { if (g_action_recorder && g_action_recorder->IsRecording()) {
g_action_recorder->RecordMouseUp(static_cast<int>(xpos), static_cast<int>(ypos)); g_action_recorder->RecordMouseUp(logicalX, logicalY);
} }
g_context->ProcessMouseButtonUp(0, key_modifier); g_context->ProcessMouseButtonUp(0, key_modifier);
} }
@@ -146,7 +164,14 @@ static void MouseButtonCallback(GLFWwindow* window, int button, int action, int
static void CursorPosCallback(GLFWwindow* window, double xpos, double ypos) { static void CursorPosCallback(GLFWwindow* window, double xpos, double ypos) {
if (!g_context) return; if (!g_context) return;
g_context->ProcessMouseMove(static_cast<int>(xpos), static_cast<int>(ypos), 0);
// Convert from physical to logical coordinates
float scaleX, scaleY;
glfwGetWindowContentScale(window, &scaleX, &scaleY);
int logicalX = static_cast<int>(xpos / scaleX);
int logicalY = static_cast<int>(ypos / scaleY);
g_context->ProcessMouseMove(logicalX, logicalY, 0);
} }
static void ScrollCallback(GLFWwindow* window, double xoffset, double yoffset) { static void ScrollCallback(GLFWwindow* window, double xoffset, double yoffset) {
@@ -154,13 +179,21 @@ static void ScrollCallback(GLFWwindow* window, double xoffset, double yoffset) {
g_context->ProcessMouseWheel(static_cast<float>(-yoffset), 0); g_context->ProcessMouseWheel(static_cast<float>(-yoffset), 0);
} }
static void FramebufferSizeCallback(GLFWwindow* window, int width, int height) {
// Update render interface viewport when framebuffer size changes (DPI scaling)
if (g_render_interface) {
g_render_interface->SetViewport(width, height);
std::cout << "Framebuffer resized to: " << width << "x" << height << std::endl;
}
}
// System interface for RmlUi // System interface for RmlUi
class DesktopSystemInterface : public Rml::SystemInterface { class DesktopSystemInterface : public Rml::SystemInterface {
public: public:
double GetElapsedTime() override { double GetElapsedTime() override {
return glfwGetTime(); return glfwGetTime();
} }
bool LogMessage(Rml::Log::Type type, const Rml::String& message) override { bool LogMessage(Rml::Log::Type type, const Rml::String& message) override {
const char* type_str = ""; const char* type_str = "";
switch (type) { switch (type) {
@@ -170,6 +203,12 @@ public:
default: type_str = "[DEBUG]"; break; default: type_str = "[DEBUG]"; break;
} }
std::cout << type_str << " " << message << std::endl; std::cout << type_str << " " << message << std::endl;
// Also log to file if configured
if (g_log_file.is_open()) {
g_log_file << type_str << " " << message << std::endl;
g_log_file.flush();
}
return true; return true;
} }
}; };
@@ -181,7 +220,9 @@ static void PrintUsage() {
std::cout << "Usage: mosis-designer [options] [document.rml]\n" std::cout << "Usage: mosis-designer [options] [document.rml]\n"
<< "\nOptions:\n" << "\nOptions:\n"
<< " --resolution WxH Set window resolution (default: 540x960)\n" << " --resolution WxH Set window resolution (default: 540x960)\n"
<< " --assets PATH Set assets directory (default: assets)\n" << " --assets PATH Set assets directory (default: derived from document)\n"
<< " --log FILE Write all log messages to file\n"
<< " --hierarchy FILE Continuously dump UI hierarchy to JSON\n"
<< "\nTest modes:\n" << "\nTest modes:\n"
<< " --record FILE Record user actions to JSON file\n" << " --record FILE Record user actions to JSON file\n"
<< " --playback FILE Playback actions from JSON file\n" << " --playback FILE Playback actions from JSON file\n"
@@ -200,7 +241,8 @@ int main(int argc, char* argv[]) {
// Parse arguments // Parse arguments
std::string document_path; std::string document_path;
std::string assets_path = "assets"; // Default relative to executable std::string assets_path; // Will be derived from document path if not specified
bool assets_path_specified = false;
for (int i = 1; i < argc; ++i) { for (int i = 1; i < argc; ++i) {
std::string arg = argv[i]; std::string arg = argv[i];
@@ -216,6 +258,11 @@ int main(int argc, char* argv[]) {
} }
} else if (arg == "--assets" && i + 1 < argc) { } else if (arg == "--assets" && i + 1 < argc) {
assets_path = argv[++i]; assets_path = argv[++i];
assets_path_specified = true;
} else if (arg == "--log" && i + 1 < argc) {
g_log_file_path = argv[++i];
} else if (arg == "--hierarchy" && i + 1 < argc) {
g_hierarchy_file_path = argv[++i];
} else if (arg == "--record" && i + 1 < argc) { } else if (arg == "--record" && i + 1 < argc) {
g_test_mode = TestMode::Record; g_test_mode = TestMode::Record;
g_test_output_path = argv[++i]; g_test_output_path = argv[++i];
@@ -232,12 +279,41 @@ int main(int argc, char* argv[]) {
document_path = arg; document_path = arg;
} }
} }
// Default document // Default document
if (document_path.empty()) { if (document_path.empty()) {
document_path = "apps/home/home.rml"; document_path = "apps/home/home.rml";
} }
// Derive assets path from document path if not specified
// The document path should be something like .../assets/apps/home/home.rml
// We want to find the "assets" directory in the path
if (!assets_path_specified) {
fs::path doc_path = fs::absolute(document_path);
fs::path current = doc_path.parent_path();
// Walk up the directory tree looking for a folder that ends with "assets"
// or contains typical asset folders like "apps", "ui", "fonts"
while (!current.empty() && current.has_parent_path()) {
std::string folder_name = current.filename().string();
if (folder_name == "assets") {
assets_path = current.string();
break;
}
// Check if this folder contains typical asset subfolders
if (fs::exists(current / "apps") && fs::exists(current / "ui")) {
assets_path = current.string();
break;
}
current = current.parent_path();
}
// Fall back to "assets" relative to executable if not found
if (assets_path.empty()) {
assets_path = "assets";
}
}
// Make assets_path absolute // Make assets_path absolute
assets_path = fs::absolute(assets_path).string(); assets_path = fs::absolute(assets_path).string();
std::cout << "Assets path: " << assets_path << std::endl; std::cout << "Assets path: " << assets_path << std::endl;
@@ -255,7 +331,22 @@ int main(int argc, char* argv[]) {
} else if (g_test_mode == TestMode::DumpHierarchy) { } else if (g_test_mode == TestMode::DumpHierarchy) {
std::cout << "Hierarchy dump mode: will save to " << g_test_output_path << std::endl; std::cout << "Hierarchy dump mode: will save to " << g_test_output_path << std::endl;
} }
// Open log file if specified
if (!g_log_file_path.empty()) {
g_log_file.open(g_log_file_path, std::ios::out | std::ios::trunc);
if (g_log_file.is_open()) {
std::cout << "Logging to: " << g_log_file_path << std::endl;
} else {
std::cerr << "Warning: Failed to open log file: " << g_log_file_path << std::endl;
}
}
// Log hierarchy file path if specified
if (!g_hierarchy_file_path.empty()) {
std::cout << "Hierarchy dump to: " << g_hierarchy_file_path << std::endl;
}
// Initialize GLFW // Initialize GLFW
glfwSetErrorCallback(ErrorCallback); glfwSetErrorCallback(ErrorCallback);
if (!glfwInit()) { if (!glfwInit()) {
@@ -269,7 +360,9 @@ int main(int argc, char* argv[]) {
glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE); glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE);
glfwWindowHint(GLFW_RESIZABLE, GLFW_FALSE); glfwWindowHint(GLFW_RESIZABLE, GLFW_FALSE);
g_window = glfwCreateWindow(g_width, g_height, "Mosis Designer", nullptr, nullptr); // Create window with document name in title
std::string window_title = "Mosis Designer - " + fs::path(document_path).filename().string();
g_window = glfwCreateWindow(g_width, g_height, window_title.c_str(), nullptr, nullptr);
if (!g_window) { if (!g_window) {
std::cerr << "Failed to create GLFW window" << std::endl; std::cerr << "Failed to create GLFW window" << std::endl;
glfwTerminate(); glfwTerminate();
@@ -284,6 +377,7 @@ int main(int argc, char* argv[]) {
glfwSetMouseButtonCallback(g_window, MouseButtonCallback); glfwSetMouseButtonCallback(g_window, MouseButtonCallback);
glfwSetCursorPosCallback(g_window, CursorPosCallback); glfwSetCursorPosCallback(g_window, CursorPosCallback);
glfwSetScrollCallback(g_window, ScrollCallback); glfwSetScrollCallback(g_window, ScrollCallback);
glfwSetFramebufferSizeCallback(g_window, FramebufferSizeCallback);
// Load OpenGL functions with GLAD // Load OpenGL functions with GLAD
if (!gladLoadGLLoader((GLADloadproc)glfwGetProcAddress)) { if (!gladLoadGLLoader((GLADloadproc)glfwGetProcAddress)) {
@@ -295,14 +389,19 @@ int main(int argc, char* argv[]) {
std::cout << "OpenGL " << glGetString(GL_VERSION) << std::endl; std::cout << "OpenGL " << glGetString(GL_VERSION) << std::endl;
// Get actual framebuffer size (may differ from window size on HiDPI displays)
int fb_width, fb_height;
glfwGetFramebufferSize(g_window, &fb_width, &fb_height);
std::cout << "Framebuffer size: " << fb_width << "x" << fb_height << std::endl;
// Create platform abstraction and set as global singleton // Create platform abstraction and set as global singleton
auto platform = std::make_unique<mosis::DesktopPlatform>(g_window, g_width, g_height); auto platform = std::make_unique<mosis::DesktopPlatform>(g_window, g_width, g_height);
platform->SetAssetsPath(assets_path); platform->SetAssetsPath(assets_path);
g_platform = platform.get(); g_platform = platform.get();
mosis::SetPlatform(std::move(platform)); mosis::SetPlatform(std::move(platform));
// Initialize RmlUi // Initialize RmlUi (use framebuffer size for rendering, logical size for context)
if (!InitializeRmlUi(assets_path)) { if (!InitializeRmlUi(assets_path, fb_width, fb_height)) {
std::cerr << "Failed to initialize RmlUi" << std::endl; std::cerr << "Failed to initialize RmlUi" << std::endl;
glfwDestroyWindow(g_window); glfwDestroyWindow(g_window);
glfwTerminate(); glfwTerminate();
@@ -356,7 +455,7 @@ int main(int argc, char* argv[]) {
glfwSwapBuffers(g_window); glfwSwapBuffers(g_window);
if (g_test_mode == TestMode::Screenshot) { if (g_test_mode == TestMode::Screenshot) {
mosis::testing::VisualCapture capture(g_width, g_height); mosis::testing::VisualCapture capture(fb_width, fb_height);
if (capture.CaptureScreenshot(g_test_output_path)) { if (capture.CaptureScreenshot(g_test_output_path)) {
std::cout << "Screenshot saved to: " << g_test_output_path << std::endl; std::cout << "Screenshot saved to: " << g_test_output_path << std::endl;
} else { } else {
@@ -412,6 +511,13 @@ int main(int argc, char* argv[]) {
g_render_interface->EndFrame(0); g_render_interface->EndFrame(0);
} }
// Dump UI hierarchy each frame if configured
if (!g_hierarchy_file_path.empty() && g_context) {
mosis::testing::UIInspector inspector(g_context);
inspector.SetCurrentScreen(g_current_screen_url);
inspector.SaveHierarchy(g_hierarchy_file_path);
}
glfwSwapBuffers(g_window); glfwSwapBuffers(g_window);
} }
@@ -432,6 +538,9 @@ int main(int argc, char* argv[]) {
g_hot_reload->Stop(); g_hot_reload->Stop();
delete g_hot_reload; delete g_hot_reload;
} }
if (g_log_file.is_open()) {
g_log_file.close();
}
ShutdownRmlUi(); ShutdownRmlUi();
glfwDestroyWindow(g_window); glfwDestroyWindow(g_window);
glfwTerminate(); glfwTerminate();
@@ -440,14 +549,15 @@ int main(int argc, char* argv[]) {
} }
bool InitializeRmlUi(const std::string& assets_path) { bool InitializeRmlUi(const std::string& assets_path, int fb_width, int fb_height) {
// Create render interface // Create render interface
g_render_interface = new RenderInterface_GL3(); g_render_interface = new RenderInterface_GL3();
if (!*g_render_interface) { if (!*g_render_interface) {
std::cerr << "Failed to create GL3 render interface" << std::endl; std::cerr << "Failed to create GL3 render interface" << std::endl;
return false; return false;
} }
g_render_interface->SetViewport(g_width, g_height); // Use framebuffer size (physical pixels) for the render interface viewport
g_render_interface->SetViewport(fb_width, fb_height);
// Initialize RmlUi // Initialize RmlUi
Rml::SetSystemInterface(&g_system_interface); Rml::SetSystemInterface(&g_system_interface);
@@ -466,7 +576,6 @@ bool InitializeRmlUi(const std::string& assets_path) {
lua_State* L = Rml::Lua::Interpreter::GetLuaState(); lua_State* L = Rml::Lua::Interpreter::GetLuaState();
lua_pushcfunction(L, [](lua_State* L) -> int { lua_pushcfunction(L, [](lua_State* L) -> int {
const char* path = luaL_checkstring(L, 1); const char* path = luaL_checkstring(L, 1);
std::cout << "loadScreen called: " << path << std::endl;
if (!g_context) { if (!g_context) {
lua_pushboolean(L, false); lua_pushboolean(L, false);
@@ -488,10 +597,12 @@ bool InitializeRmlUi(const std::string& assets_path) {
if (document) { if (document) {
document->Show(); document->Show();
g_current_document_path = path; g_current_document_path = path;
std::cout << "Loaded: " << path << std::endl; g_current_screen_url = path; // Track current screen for hierarchy dump
// Log using RmlUi logging so it appears in log file
Rml::Log::Message(Rml::Log::LT_INFO, "Loaded screen: %s", path);
lua_pushboolean(L, true); lua_pushboolean(L, true);
} else { } else {
std::cerr << "Failed to load: " << path << std::endl; Rml::Log::Message(Rml::Log::LT_ERROR, "Failed to load screen: %s", path);
lua_pushboolean(L, false); lua_pushboolean(L, false);
} }
return 1; return 1;
@@ -557,8 +668,9 @@ bool LoadDocument(const std::string& path) {
document->Show(); document->Show();
g_current_document_path = path; g_current_document_path = path;
g_current_screen_url = path; // Track current screen for hierarchy dump
std::cout << "Loaded: " << path << std::endl; std::cout << "Loaded: " << path << std::endl;
return true; return true;
} }

View File

@@ -1,6 +1,7 @@
// D:\Dev\Mosis\MosisService\designer\src\desktop_file_interface.cpp // D:\Dev\Mosis\MosisService\designer\src\desktop_file_interface.cpp
#include "desktop_file_interface.h" #include "desktop_file_interface.h"
#include <cstdio> #include <cstdio>
#include <cctype>
#include <filesystem> #include <filesystem>
namespace fs = std::filesystem; namespace fs = std::filesystem;
@@ -16,12 +17,20 @@ void DesktopFileInterface::SetAssetsPath(const std::string& path) {
} }
std::string DesktopFileInterface::ResolvePath(const std::string& path) const { std::string DesktopFileInterface::ResolvePath(const std::string& path) const {
std::string resolved = path;
// Handle URL-encoded Windows drive letters (D| -> D:)
// RmlUi sometimes encodes the colon in Windows paths
if (resolved.size() >= 2 && std::isalpha(resolved[0]) && resolved[1] == '|') {
resolved[1] = ':';
}
// If path is absolute, use it directly // If path is absolute, use it directly
if (fs::path(path).is_absolute()) { if (fs::path(resolved).is_absolute()) {
return path; return resolved;
} }
// Otherwise, prepend assets path // Otherwise, prepend assets path
return m_assets_path + path; return m_assets_path + resolved;
} }
Rml::FileHandle DesktopFileInterface::Open(const Rml::String& path) { Rml::FileHandle DesktopFileInterface::Open(const Rml::String& path) {

View File

@@ -2,11 +2,27 @@
#include "ui_inspector.h" #include "ui_inspector.h"
#include <fstream> #include <fstream>
#include <sstream> #include <sstream>
#include <filesystem>
namespace mosis::testing { namespace mosis::testing {
using json = nlohmann::json; using json = nlohmann::json;
// Helper to normalize RmlUi URLs for comparison
// RmlUi uses '|' instead of ':' for Windows drive letters
static std::string NormalizeUrl(const std::string& url) {
std::string normalized = url;
// Replace | with : (for Windows drive letters)
for (char& c : normalized) {
if (c == '|') c = ':';
}
// Normalize backslashes to forward slashes
for (char& c : normalized) {
if (c == '\\') c = '/';
}
return normalized;
}
UIInspector::UIInspector(Rml::Context* context) UIInspector::UIInspector(Rml::Context* context)
: m_context(context) : m_context(context)
{ {
@@ -24,10 +40,16 @@ json UIInspector::ElementToJson(Rml::Element* element) const {
j["id"] = id; j["id"] = id;
} }
// Classes // Classes - split into array
std::string class_str = element->GetAttribute<Rml::String>("class", ""); std::string class_str = element->GetAttribute<Rml::String>("class", "");
if (!class_str.empty()) { if (!class_str.empty()) {
j["classes"] = class_str; json classes_arr = json::array();
std::istringstream iss(class_str);
std::string cls;
while (iss >> cls) {
classes_arr.push_back(cls);
}
j["classes"] = classes_arr;
} }
// Bounds // Bounds
@@ -75,32 +97,63 @@ json UIInspector::ElementToJson(Rml::Element* element) const {
json UIInspector::DumpHierarchy() const { json UIInspector::DumpHierarchy() const {
json result; json result;
// Get current timestamp // Get current timestamp
auto now = std::chrono::system_clock::now(); auto now = std::chrono::system_clock::now();
auto time_t = std::chrono::system_clock::to_time_t(now); auto time_t = std::chrono::system_clock::to_time_t(now);
std::stringstream ss; std::stringstream ss;
ss << std::put_time(std::localtime(&time_t), "%Y-%m-%dT%H:%M:%S"); ss << std::put_time(std::localtime(&time_t), "%Y-%m-%dT%H:%M:%S");
result["timestamp"] = ss.str(); result["timestamp"] = ss.str();
// Resolution // Resolution
auto dimensions = m_context->GetDimensions(); auto dimensions = m_context->GetDimensions();
result["resolution"] = { result["resolution"] = {
{"width", dimensions.x}, {"width", dimensions.x},
{"height", dimensions.y} {"height", dimensions.y}
}; };
// Current screen (document URL) // Current screen - use override if set, otherwise detect from documents
std::string screen = ""; std::string screen = m_current_screen;
if (m_context->GetNumDocuments() > 0) { Rml::ElementDocument* main_doc = nullptr;
auto* doc = m_context->GetDocument(0);
// Normalize the current screen path for comparison
std::string normalized_current = NormalizeUrl(m_current_screen);
// Find the document matching the current screen, or first non-debugger document
for (int i = 0; i < m_context->GetNumDocuments(); i++) {
auto* doc = m_context->GetDocument(i);
if (doc) { if (doc) {
screen = doc->GetSourceURL(); std::string url = doc->GetSourceURL();
// Skip debugger documents
if (url.find("__rmlui") == std::string::npos) {
// Normalize the document URL for comparison
std::string normalized_url = NormalizeUrl(url);
// If we have a current screen override, match it
if (!m_current_screen.empty()) {
if (normalized_url.find(normalized_current) != std::string::npos ||
normalized_current.find(normalized_url) != std::string::npos) {
main_doc = doc;
break;
}
} else if (!main_doc) {
// No override, use first non-debugger document
screen = url;
main_doc = doc;
}
}
} }
} }
result["screen"] = screen; result["screen"] = screen;
// Elements - dump all documents // Elements - dump the main document body (expected format for HierarchyReader)
if (main_doc) {
result["elements"] = ElementToJson(main_doc);
} else {
result["elements"] = json::object();
}
// Also include all documents for more detailed inspection
result["documents"] = json::array(); result["documents"] = json::array();
for (int i = 0; i < m_context->GetNumDocuments(); i++) { for (int i = 0; i < m_context->GetNumDocuments(); i++) {
auto* doc = m_context->GetDocument(i); auto* doc = m_context->GetDocument(i);
@@ -108,24 +161,51 @@ json UIInspector::DumpHierarchy() const {
json doc_json; json doc_json;
doc_json["url"] = doc->GetSourceURL(); doc_json["url"] = doc->GetSourceURL();
doc_json["title"] = doc->GetTitle(); doc_json["title"] = doc->GetTitle();
// Dump document root (ElementDocument inherits from Element) // Dump document root (ElementDocument inherits from Element)
doc_json["body"] = ElementToJson(doc); doc_json["body"] = ElementToJson(doc);
result["documents"].push_back(doc_json); result["documents"].push_back(doc_json);
} }
} }
return result; return result;
} }
bool UIInspector::SaveHierarchy(const std::string& path) const { bool UIInspector::SaveHierarchy(const std::string& path) const {
json hierarchy = DumpHierarchy(); json hierarchy = DumpHierarchy();
std::string content = hierarchy.dump(2);
std::ofstream file(path);
if (!file) return false; // Use temporary file + rename for atomic writes
std::string temp_path = path + ".tmp";
file << hierarchy.dump(2);
{
std::ofstream file(temp_path, std::ios::binary | std::ios::trunc);
if (!file) {
return false;
}
file.write(content.c_str(), content.size());
file.flush();
if (!file.good()) {
file.close();
return false;
}
file.close();
}
// Delete the old file first to avoid rename conflicts on Windows
std::error_code ec;
std::filesystem::remove(path, ec); // Ignore errors if file doesn't exist
// Rename temp file to final path
std::filesystem::rename(temp_path, path, ec);
if (ec) {
// Fallback: copy and delete if rename fails
std::filesystem::copy_file(temp_path, path,
std::filesystem::copy_options::overwrite_existing, ec);
std::filesystem::remove(temp_path, ec);
}
return true; return true;
} }

View File

@@ -12,9 +12,12 @@ public:
UIInspector(Rml::Context* context); UIInspector(Rml::Context* context);
~UIInspector() = default; ~UIInspector() = default;
// Set current screen URL override (to handle stale document detection)
void SetCurrentScreen(const std::string& screenUrl) { m_current_screen = screenUrl; }
// Dump entire UI hierarchy to JSON // Dump entire UI hierarchy to JSON
nlohmann::json DumpHierarchy() const; nlohmann::json DumpHierarchy() const;
// Save hierarchy to file // Save hierarchy to file
bool SaveHierarchy(const std::string& path) const; bool SaveHierarchy(const std::string& path) const;
@@ -42,6 +45,7 @@ private:
Rml::Element* FindElementRecursive(Rml::Element* root, const std::string& selector) const; Rml::Element* FindElementRecursive(Rml::Element* root, const std::string& selector) const;
Rml::Context* m_context; Rml::Context* m_context;
std::string m_current_screen; // Override for screen URL detection
}; };
} // namespace mosis::testing } // namespace mosis::testing