omar/MosisService
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add shell architecture with persistent status bar, nav bar, and content fragments

Browse Source
This commit is contained in:
omigamedev
2026-01-20 10:09:47 +01:00
parent 1f91d7508e
commit 2db7eea9f1
15 changed files with 1456 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/CLAUDE.md
View File

@@ -23,7 +23,7 @@ Mosis is a **virtual smartphone OS** for VR games and applications. It provides
| MosisService | ✅ Working | RmlUi rendering, touch input, navigation |
| App Management | ✅ Working | Install/uninstall apps, sandbox integration |
| Lua Sandbox | ✅ Working | 149 security tests passing |
| Desktop Designer | ✅ Working | Hot-reload, hierarchy dump, recording |
| Desktop Designer | ✅ Working | Hot-reload, shell mode, hierarchy dump, recording |
| Designer Tests | ✅ 5/5 Passing | Navigation tests automated |
| MosisVR (Unity) | ✅ Building | OpenGL backend working, Vulkan in progress |
| MosisUnreal | ✅ Working | Vulkan texture import via UE5 RHI, phone actor with mesh |
@@ -35,6 +35,7 @@ Mosis is a **virtual smartphone OS** for VR games and applications. It provides
| Android Service | `src/main/` | Native service running RmlUi renderer |
| App Management | `src/main/cpp/apps/` | App install/uninstall/launch with sandbox |
| Lua Sandbox | `src/main/cpp/sandbox/` | Per-app Lua isolation (22 modules) |
| System Shell | `src/main/assets/apps/shell/` | Persistent status bar, nav bar, overlays |
| Desktop Designer | `designer/` | UI development with hot-reload |
| Designer Tests | `designer-test/` | Automated UI testing framework |
| Sandbox Tests | `sandbox-test/` | Lua sandbox security tests (149 tests) |
@@ -48,7 +49,7 @@ All detailed documentation is in `docs/`:
|----------|-------------|
| [BUILD-COMMANDS.md](BUILD-COMMANDS.md) | Android, Desktop Designer, and test build commands |
| [ARCHITECTURE.md](ARCHITECTURE.md) | Native libraries, IPC flow, code structure |
| [DESKTOP-DESIGNER.md](DESKTOP-DESIGNER.md) | Hot-reload, recording, key files |
| [DESKTOP-DESIGNER.md](DESKTOP-DESIGNER.md) | Shell architecture, hot-reload, recording |
| [TESTING-FRAMEWORK.md](TESTING-FRAMEWORK.md) | Automated UI testing, writing tests |
| [UI-ASSETS.md](UI-ASSETS.md) | Asset structure, navigation system, element IDs |
| [MATERIAL-DESIGN.md](MATERIAL-DESIGN.md) | Icons, MDL components, usage guide |

202
docs/DESKTOP-DESIGNER.md
View File

@@ -3,6 +3,7 @@
The desktop designer (`designer/`) provides rapid UI development with:
- **Hot-reload**: Automatically reloads when RML/RCSS/Lua files change
- **Shell Mode**: Persistent system UI (status bar, nav bar) with apps loading into container
- **UI Hierarchy Dumping**: Exports element tree to JSON for inspection
- **Screenshot Capture**: PNG export via F12 key
- **Logging**: Detailed output for debugging navigation and events
@@ -13,44 +14,57 @@ The desktop designer (`designer/`) provides rapid UI development with:
| File | Purpose |
|------|---------|
| `designer/src/main.cpp` | Main entry point, GLFW window, event loop |
| `designer/src/desktop_kernel.cpp` | RmlUi context management, rendering |
| `designer/main.cpp` | Main entry point, GLFW window, event loop |
| `designer/src/desktop_sandbox.cpp` | Per-app sandbox isolation |
| `designer/src/app_discovery.cpp` | Scans directories for app manifests |
| `designer/src/testing/ui_inspector.cpp` | UI hierarchy JSON export |
| `designer/src/testing/visual_capture.cpp` | PNG screenshot capture and comparison |
| `designer/src/testing/action_recorder.cpp` | Record user interactions to JSON |
| `designer/src/testing/visual_capture.cpp` | PNG screenshot capture |
| `designer/src/testing/action_recorder.cpp` | Record user interactions |
| `designer/src/testing/action_player.cpp` | Playback recorded actions |
| `designer/src/backend/RmlUi_Backend_GLFW_GL3.cpp` | GLFW backend with input hooks |
## Command Line Options
```
--simulator Enable simulator mode (shows home screen with third-party apps)
--test-apps <path> Path to test-apps directory (default: auto-detect)
--assets <path> Set assets directory (default: derived from document)
--log <path> Write logs to file
--hierarchy <path> Dump UI hierarchy JSON each frame
--dump Single-shot dump mode (screenshot + hierarchy)
--record <path> Enable recording mode (F5 to start/stop)
--playback <path> Play back recorded actions from JSON
--resolution WxH Set window resolution (default: 540x960)
General:
--resolution WxH Set window resolution (default: 540x960)
--assets PATH Set assets directory (default: derived from document)
--log FILE Write all log messages to file
--hierarchy FILE Continuously dump UI hierarchy to JSON
Simulator mode:
--simulator Run in simulator mode (uses shell by default)
--no-shell Disable shell (use direct document loading)
--test-apps PATH Path to test-apps directory (default: ./base-apps)
Test modes:
--record FILE Record user actions to JSON file
--playback FILE Playback actions from JSON file
--screenshot FILE Take screenshot and exit
--dump-hierarchy FILE Dump UI hierarchy to JSON and exit
```
## Running the Designer
### Simulator Mode (Recommended for Testing Apps)
To run the designer with third-party test apps visible:
### Simulator Mode with Shell (Recommended)
```bash
cd MosisService
./designer/build/Release/mosis-designer.exe --simulator --test-apps test-apps
./designer/build/Release/mosis-designer.exe --simulator --test-apps base-apps
```
This will:
- Load the home screen from `src/main/assets/apps/home/home.rml`
- Scan `test-apps/` for apps with valid `manifest.json`
- Display discovered apps in the home screen grid
- Enable the `mosis.apps` Lua API for app launching
- Load the **system shell** with persistent status bar and navigation bar
- Load home screen content into the shell's app container
- Scan `base-apps/` for apps with valid `manifest.json`
- Enable app navigation with proper back button support
### Simulator Mode without Shell
To test the old direct-document approach:
```bash
./designer/build/Release/mosis-designer.exe --simulator --test-apps base-apps --no-shell
```
### Direct Document Mode
@@ -64,6 +78,144 @@ To load a specific RML document directly:
| Key | Function |
|-----|----------|
| F5 | Start/stop recording (when --record is enabled) |
| F6 | Pause/resume playback (when --playback is enabled) |
| F12 | Take screenshot |
| F5 | Reload document / Start-stop recording |
| F12 | Toggle RmlUi debugger |
| ESC | Back navigation |
| R | Start/Stop recording (in interactive mode) |
---
## Shell Architecture
The shell (`apps/shell/`) provides a persistent system UI layer that prevents apps from taking over the full screen.
### Shell Components
```
┌──────────────────────────────┐
│ Status Bar (36px) │ ← Time, WiFi, Signal, Battery
├──────────────────────────────┤
│ │
│ App Container │ ← App content loads here
│ (flex: 1) │
│ │
├──────────────────────────────┤
│ Navigation Bar (56px) │ ← Back, Home, Recents
└──────────────────────────────┘
Overlay Layers (z-index order):
- Dialog overlay (z: 600) ← Modal dialogs
- Toast container (z: 500) ← Toast notifications
- Notification panel (z: 400) ← Pull-down notifications
- Loading overlay (z: 300) ← App loading spinner
```
### Shell Files
| File | Purpose |
|------|---------|
| `apps/shell/shell.rml` | Shell document with status bar, nav bar, overlays |
| `apps/shell/shell.lua` | Navigation, toasts, dialogs, notifications |
### App Content Fragments
Apps in shell mode use **content fragments** (`*_content.rml`) instead of full documents:
```rml
<!-- camera_content.rml - No <rml>, <head>, or document structure -->
<style>
.camera-content { ... }
</style>
<div class="camera-content">
<!-- App UI here -->
</div>
```
### Shell Lua API
Functions available to apps via shell:
```lua
-- Navigation
shellNavigateTo("settings") -- Navigate to system app
shellLaunchApp(id, path, entry) -- Launch external app
shellGoBack() -- Go to previous app
shellGoHome() -- Go to home (clear history)
-- UI Feedback
showToast("Message") -- Show toast notification
showToast("Error!", "error") -- Toast types: default, success, error, warning
showDialog("Title", "Message", onConfirm, onCancel)
-- Notifications
addNotification("Title", "Text", icon, app_id)
toggleNotifications() -- Show/hide notification panel
clearNotifications()
```
### Creating Content Fragments
1. Create `appname_content.rml` (not a full document)
2. Include styles via `<style>` tag
3. Use shell functions for navigation (`shellNavigateTo`, `showToast`, etc.)
4. Don't include status bar or nav bar (shell provides these)
Example:
```rml
<!-- settings_content.rml -->
<style>
.settings-list { padding: 16px; }
.settings-item { padding: 16px; cursor: pointer; }
</style>
<div class="settings-list">
<div class="settings-item" onclick="showToast('WiFi settings')">
WiFi
</div>
<div class="settings-item" onclick="showToast('Bluetooth settings')">
Bluetooth
</div>
</div>
```
### Content Fragment Locations
| App | Content Fragment Path |
|-----|----------------------|
| Home | `apps/home/home_content.rml` |
| Camera | `apps/camera/camera_content.rml` |
| Dialer | `apps/dialer/dialer_content.rml` |
| Messages | `apps/messages/messages_content.rml` |
| Browser | `apps/browser/browser_content.rml` |
| Settings | `apps/settings/settings_content.rml` |
| Music | `apps/music/music_content.rml` |
| Store | `apps/store/store_content.rml` |
| Contacts | `apps/contacts/contacts_content.rml` |
---
## Hot Reload
The designer watches for file changes in the assets directory:
- **RML files**: Reloads document
- **RCSS files**: Reloads stylesheets
- **Lua files**: Reloads scripts
- **TGA/PNG files**: Reloads textures
Press F5 to force reload.
## UI Hierarchy Dump
Use `--hierarchy` to continuously dump the UI element tree:
```bash
./designer/build/Release/mosis-designer.exe --simulator --hierarchy ui_tree.json
```
The JSON contains element bounds, classes, IDs, and visibility - useful for automated testing.
## Action Recording and Playback
See [TESTING-FRAMEWORK.md](TESTING-FRAMEWORK.md) for details on recording and playing back UI interactions.