omar/MosisService

Fork 0

Files

omigamedev d76627ebc3 add Developer Portal architecture summary to docs

2026-01-18 20:49:03 +01:00

32 KiB

Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Company

OmixLab LTD - Package namespace: com.omixlab

Git Commit Guidelines

IMPORTANT: When creating git commits:

DO NOT add yourself as a co-author (no Co-Authored-By lines)
Commit messages must be a single line - keep it concise and descriptive

Project Overview

Mosis is a virtual smartphone OS for VR games and applications. It provides a phone-like device that users can interact with inside VR environments, with real smartphone functionality.

Current Status

Component	Status	Notes
MosisService	✅ Working	RmlUi rendering, touch input, navigation
Desktop Designer	✅ Working	Hot-reload, hierarchy dump, recording
Designer Tests	✅ 5/5 Passing	Navigation tests automated
MosisVR (Unity)	✅ Building	OpenGL backend working, Vulkan in progress
MosisUnreal	✅ Working	Vulkan texture import via UE5 RHI, phone actor with mesh

Project Components

Component	Location	Purpose
Android Service	`src/main/`	Native service running RmlUi renderer
Desktop Designer	`designer/`	UI development with hot-reload
Designer Tests	`designer-test/`	Automated UI testing framework
Sandbox Tests	`sandbox-test/`	Lua sandbox security tests
Lua Sandbox	`src/main/cpp/sandbox/`	Per-app Lua isolation
UI Assets	`src/main/assets/`	Shared RML/RCSS/Lua assets

Build Commands

Android (Gradle)

# Build entire project
./gradlew build

# Build debug APK
./gradlew assembleDebug

# Build release APK
./gradlew assembleRelease

# Clean build outputs
./gradlew clean

# Build with verbose output
./gradlew build --info --stacktrace

# Run lint checks
./gradlew lint

# Run unit tests
./gradlew test

# Run connected device tests
./gradlew connectedAndroidTest

Desktop Designer (CMake)

# Configure (from designer/ folder)
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake

# Build
cmake --build build --config Debug

# Run with hot-reload
./build/Debug/mosis-designer.exe ../src/main/assets/apps/home/home.rml

# Run with logging and hierarchy dump
./build/Debug/mosis-designer.exe ../src/main/assets/apps/home/home.rml --log output.log --hierarchy hierarchy.json

Designer Tests (CMake)

# Configure (from designer-test/ folder)
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake

# Build
cmake --build build --config Debug

# Run tests
./build/Debug/designer-test.exe

Sandbox Security Tests (CMake)

# Configure (from sandbox-test/ folder)
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake

# Build
cmake --build build --config Debug

# Run all tests (uber command)
./run_tests.bat

# Or run directly
./build/Debug/sandbox-test.exe

# Run specific test
./build/Debug/sandbox-test.exe --test DangerousGlobals
./build/Debug/sandbox-test.exe --test Memory
./build/Debug/sandbox-test.exe --test CPU

Environment Requirements

Required environment variables:

ANDROID_HOME - Android SDK path
ANDROID_NDK_HOME - Android NDK path (version 29.0.14206865)
VCPKG_ROOT - vcpkg package manager root

Architecture Overview

MosisService is an Android application combining Kotlin UI with native C++ libraries for UI rendering via Android's Binder IPC system.

Two Native Libraries

mosis-service (libmosis-service.so):

Main Android Binder service implementation
Implements IMosisService.aidl interface for touch events and initialization
Contains the Kernel rendering engine with RmlUi integration
Links against RmlUi for HTML/CSS-like UI rendering

mosis-test (libmosis-test.so):

Test/rendering client implementation
Implements IMosisListener.aidl for receiving callbacks
OpenGL ES 2.0 rendering pipeline using GLAD

IPC Flow

Kotlin NativeService → JNI → mosis-service (IMosisService)
                              ↓
                         IMosisListener callbacks
                              ↓
                         mosis-test (rendering client)

Key Interfaces (AIDL)

IMosisService: initOS(), onTouchDown(), onTouchMove(), onTouchUp()

IMosisListener (oneway/async): onServiceInitialized(), onBufferAvailable(), onFrameAvailable()

Native Code Structure (src/main/cpp/)

kernel.cpp - Core rendering engine, RmlUi integration, event processing
mosis-service.cpp - Binder service implementation, JNI entry points
mosis-test.cpp - Test client implementation
egl_context.cpp - OpenGL ES context management
render_target.cpp - Framebuffer and buffer management
RmlUi_Renderer_GL3.cpp - RmlUi OpenGL renderer backend
assets_manager.cpp - Android AssetManager integration

Code Style

C++23 standard with modern features (std::span, std::format)
PascalCase for classes/functions, camelCase for variables
RAII principles with smart pointers
Kotlin code follows Android conventions

Dependencies

vcpkg manages native dependencies (RmlUi, GLFW, Freetype, Lua, libpng, nlohmann-json)
CMake build system with vcpkg toolchain integration
Android target architecture: arm64-v8a only
Desktop target: Windows x64 (MSVC)

Desktop Designer

The desktop designer (designer/) provides rapid UI development with:

Hot-reload: Automatically reloads when RML/RCSS/Lua files change
UI Hierarchy Dumping: Exports element tree to JSON for inspection
Screenshot Capture: PNG export via F12 key
Logging: Detailed output for debugging navigation and events
Action Recording: Record mouse/keyboard interactions to JSON
Action Playback: Replay recorded interactions with timing

Key Files

File	Purpose
`designer/src/main.cpp`	Main entry point, GLFW window, event loop
`designer/src/desktop_kernel.cpp`	RmlUi context management, rendering
`designer/src/testing/ui_inspector.cpp`	UI hierarchy JSON export
`designer/src/testing/visual_capture.cpp`	PNG screenshot capture and comparison
`designer/src/testing/action_recorder.cpp`	Record user interactions to JSON
`designer/src/testing/action_player.cpp`	Playback recorded actions
`designer/src/backend/RmlUi_Backend_GLFW_GL3.cpp`	GLFW backend with input hooks

Command Line Options

--log <path>        Write logs to file
--hierarchy <path>  Dump UI hierarchy JSON each frame
--dump              Single-shot dump mode (screenshot + hierarchy)
--record <path>     Enable recording mode (F5 to start/stop)
--playback <path>   Play back recorded actions from JSON

Keyboard Controls

Key	Function
F5	Start/stop recording (when --record is enabled)
F6	Pause/resume playback (when --playback is enabled)
F12	Take screenshot

Automated Testing Framework

The designer-test (designer-test/) provides automated UI testing:

Test Architecture

WindowController: Finds designer window, sends mouse/keyboard events via Windows SendInput API
HierarchyReader: Parses UI hierarchy JSON to find elements by ID/class (with retry logic and exponential backoff)
LogParser: Monitors log file for navigation events
TestRunner: Orchestrates test execution, reports results
UIInspector: Dumps UI hierarchy with atomic writes (temp file + rename pattern)

Key Implementation Details

Path Normalization: RmlUi uses | instead of : in Windows paths (e.g., D|\Dev\...). The UIInspector normalizes paths for correct document matching.
Atomic File Writes: Hierarchy files are written to .tmp then renamed to prevent partial reads.
Retry with Backoff: HierarchyReader retries up to 10 times with exponential backoff (30ms base) and validates JSON completeness.
Dynamic Back Button: GoHome() finds back buttons from hierarchy by class (app-bar-nav or browser-nav-btn) instead of fixed coordinates.

Writing Tests

// Find element by ID and click it
bool ClickById(TestContext& ctx, const std::string& id) {
    ctx.hierarchy.Reload();
    auto element = ctx.hierarchy.FindById(id);
    if (!element) return false;

    int x = element->bounds.centerX();
    int y = element->bounds.centerY();
    ScaleToPhysical(ctx, x, y);  // Convert logical to physical coords
    ctx.window.SendClick(x, y);
    return true;
}

// Test example
bool TestNavigateToDialer(TestContext& ctx) {
    GoHome(ctx);
    ctx.log.Clear();

    if (!ClickById(ctx, "dock-phone")) return false;

    std::this_thread::sleep_for(std::chrono::milliseconds(1000));
    ctx.log.Reload();
    return ctx.log.Contains("Loaded screen: apps/dialer/dialer.rml");
}

Test Output

Tests produce JSON results at test_results.json:

{
  "name": "Mosis Designer UI Tests",
  "summary": {"passed": 5, "failed": 0, "total": 5},
  "tests": [
    {"name": "Navigate to Dialer", "status": "passed", "duration_ms": 3500}
  ]
}

UI Assets Structure

All UI assets are in src/main/assets/:

assets/
├── apps/                    # System apps (RML documents)
│   ├── home/home.rml        # Home screen launcher
│   ├── dialer/dialer.rml    # Phone dialer
│   ├── messages/            # Messages app
│   ├── contacts/            # Contacts app
│   ├── settings/            # Settings app
│   └── browser/             # Web browser
├── ui/                      # Shared stylesheets
│   ├── html.rcss            # Base HTML element styles
│   ├── theme.rcss           # Design tokens (colors, typography)
│   └── components.rcss      # Reusable UI components
├── scripts/                 # Lua scripts
│   └── navigation.lua       # Navigation system
├── icons/                   # TGA icon files (24x24, 32x32)
└── fonts/                   # TTF fonts (LatoLatin, Roboto)

Navigation is handled by scripts/navigation.lua:

-- Navigate to a screen
navigateTo('dialer')     -- Push to history, animate forward

-- Go back
goBack()                  -- Pop from history, animate back

-- Go home
goHome()                  -- Clear history, return to home

Element IDs for Testing

Key elements have IDs for automated testing:

ID	Location	Purpose
`dock-phone`	home.rml	Phone dock icon
`dock-messages`	home.rml	Messages dock icon
`dock-contacts`	home.rml	Contacts dock icon
`dock-browser`	home.rml	Browser dock icon
`app-settings`	home.rml	Settings app icon

Back buttons use app-bar-nav class for automated GoHome:

<div class="app-bar-nav btn-icon" onclick="goBack()">
    <img src="../../icons/back.tga"/>
</div>

Browser uses browser-nav-btn class for its toolbar back button:

<div class="app-bar-nav browser-nav-btn" onclick="goBack()">
    <img src="../../icons/back.tga"/>
</div>

The test framework's FindBackButton() searches for both classes to handle all screen layouts.

Material Design Resources

Material Design icons and components are available in the MosisDesigner repository:

Material Design Icons

Location: D:\Dev\Mosis\MosisDesigner\material-design-icons

A comprehensive icon library from Google with 2000+ icons across 20 categories:

Category	Examples
action	home, search, settings, delete, info
alert	error, warning, notification
av	play, pause, volume, mic
communication	phone, message, email, contacts
content	add, remove, copy, paste
device	battery, wifi, bluetooth, gps
editor	format, text, color, brush
file	folder, attachment, download, upload
hardware	keyboard, mouse, phone, tablet
home	lightbulb, thermostat, security
image	camera, photo, filter, tune
maps	location, directions, navigation
navigation	arrow, chevron, menu, close
notification	sync, update, event
places	hotel, restaurant, airport
search	search variants
social	share, person, group, notifications
toggle	star, checkbox, radio

Available Formats:

src/ - SVG source files organized by category
png/ - PNG files at multiple DPIs (24dp, 36dp, 48dp)
font/ - Icon fonts (WOFF, TTF)
symbols/ - Material Symbols variable font (newer)
variablefont/ - Variable font files

Icon Styles:

Outlined (default)
Filled
Rounded
Sharp
Two-tone (Material Icons only)

Material Design Lite

Location: D:\Dev\Mosis\MosisDesigner\material-design-lite

CSS/JS component library implementing Material Design (reference implementation):

Directory	Contents
`src/`	SASS source for components
`docs/`	Component documentation
`templates/`	Page templates

Key Components (for design reference):

Buttons (raised, flat, FAB)
Cards
Dialogs
Lists
Menus
Navigation drawers
Progress indicators
Sliders
Snackbars
Tables
Tabs
Text fields
Tooltips

Using Icons in Mosis

Find icon at https://fonts.google.com/icons
Export SVG from material-design-icons/src/<category>/<name>/
Convert to TGA using image tool (24x24 or 32x32, RGBA)
Place in src/main/assets/icons/
Reference in RML: <img src="../../icons/<name>.tga"/>

Android Device Testing

Prerequisites

# Check connected device
adb devices -l

# Verify Mosis app is installed
adb shell pm list packages | grep mosis

Build and Install

# Build debug APK
./gradlew assembleDebug

# Install on device
adb install -r build/outputs/apk/debug/MosisService-debug.apk

# Launch the app
adb shell am start -n com.omixlab.mosis/.MainActivity

Run Gradle Connected Tests

# Run all connected Android tests
./gradlew connectedAndroidTest

Event Injection via ADB

Inject touch events for automated testing:

# Click at normalized coordinates (0.0-1.0)
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" --ef x 0.5 --ef y 0.5

# Touch down
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "down" --ef x 0.2 --ef y 0.9

# Touch up
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "up" --ef x 0.2 --ef y 0.9

Dock Element Coordinates (Normalized)

Element	X	Y
dock-phone	0.16	0.97
dock-messages	0.39	0.97
dock-contacts	0.61	0.97
dock-browser	0.84	0.97
back-button	0.10	0.05

# Clear logs and run navigation test sequence
adb logcat -c

# Click Phone dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" --ef x 0.16 --ef y 0.97
sleep 2

# Click back to return home
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" --ef x 0.1 --ef y 0.05
sleep 2

# Click Messages dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" --ef x 0.39 --ef y 0.97
sleep 2

# Click back to return home
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" --ef x 0.1 --ef y 0.05
sleep 2

# Click Contacts dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" --ef x 0.61 --ef y 0.97
sleep 2

# Click back to return home
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" --ef x 0.1 --ef y 0.05
sleep 2

# Click Browser dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
    --es touch_type "click" --ef x 0.84 --ef y 0.97

Reading Logs

# Filter for Mosis logs
adb logcat -s MosisTest ServiceTester RMLUI

# Filter for navigation events
adb logcat -d | grep -iE "navigat|loaded|goBack|rml"

# Save to file
adb logcat -s MosisTest > mosis-log.txt

# Clear logs
adb logcat -c

Expected Log Output

Successful navigation shows these log patterns:

RMLUI: navigateTo called with: dialer
Loading screen: apps/dialer/dialer.rml
RMLUI: Navigated to: dialer (history depth: 1)

RMLUI: goBack called (history depth: 1)
Loading screen: apps/home/home.rml
RMLUI: Back to: home

Documentation Guidelines

IMPORTANT: Always document progress and new commands to avoid rediscovery.

Where to Put Documentation

Content Type	Location
General concepts, architecture	`MosisService/CLAUDE.md` (this file)
Unreal plugin docs	`MosisUnreal/Plugins/MosisSDK/README.md`
Unity package docs	`MosisVR/Packages/com.omixlab.mosis_sdk/README.md`
Project-specific build commands	Each project's own docs

DO NOT put documentation in the root D:\Dev\Mosis\ directory - it is not versioned.

What to Document

Build commands - Every new build command discovered or created
Environment setup - Required environment variables, SDK versions
Architecture decisions - Why something was done a certain way
Issues and solutions - Problems encountered and how they were fixed
File locations - Where important files are and what they do

When to Document

After completing a milestone or feature
When discovering new build commands
When fixing a non-obvious issue
When adding new dependencies or requirements

Game Engine Integrations

MosisService provides a virtual phone that game engines can display and interact with.

Integration Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        MosisService                              │
│              (OpenGL ES rendering → AHardwareBuffer)             │
└───────────────────────────┬─────────────────────────────────────┘
                            │ Binder IPC + Shared Memory
          ┌─────────────────┴─────────────────┐
          ▼                                   ▼
┌─────────────────────┐             ┌─────────────────────┐
│    MosisUnreal      │             │      MosisVR        │
│   (UE5.5 Plugin)    │             │  (Unity Package)    │
│   Vulkan Import     │             │  Vulkan/OpenGL      │
└─────────────────────┘             └─────────────────────┘

MosisUnreal (Unreal Engine 5.5)

Location: D:\Dev\Mosis\MosisUnreal\Plugins\MosisSDK\

Build Commands

:: Windows Editor Build
"D:\Epic\UE_5.5\Engine\Build\BatchFiles\Build.bat" ^
    MosisUnrealEditor Win64 Development ^
    -Project="D:\Dev\Mosis\MosisUnreal\MosisUnreal.uproject"

:: Android APK Build
"D:\Epic\UE_5.5\Engine\Build\BatchFiles\RunUAT.bat" ^
    BuildCookRun ^
    -project="D:\Dev\Mosis\MosisUnreal\MosisUnreal.uproject" ^
    -platform=Android -clientconfig=Development ^
    -build -cook -stage -pak -package -noP4

:: Clean Build (delete these folders first)
rmdir /s /q "Intermediate\Build"
rmdir /s /q "Binaries"

Output Files

Build	Output
Windows Editor	`Plugins/MosisSDK/Binaries/Win64/UnrealEditor-MosisSDK.dll`
Android APK	`Binaries/Android/MosisUnreal-arm64.apk`
Android OBB	`Binaries/Android/main.1.com.omixlab.MosisUnreal.obb`

Requirements

Android SDK Platform 36 (for AIDL binder headers)
Android Build Tools 36.1.0 (for AIDL compiler)
ANDROID_HOME environment variable set

Quest Deployment

IMPORTANT: Git Bash on Windows converts Unix paths to Windows paths. Use MSYS_NO_PATHCONV=1 prefix for all adb push commands.

# Set Quest device ID (check with: adb devices -l)
QUEST=2G0YC5ZF7W01T0

# Install APK
adb -s $QUEST install -r D:/Dev/Mosis/MosisUnreal/Binaries/Android/MosisUnreal-arm64.apk

# Create OBB temp directory
adb -s $QUEST shell "mkdir -p /data/local/tmp/obb/com.omixlab.MosisUnreal"

# Push OBB to temp location (MUST use MSYS_NO_PATHCONV=1)
MSYS_NO_PATHCONV=1 adb -s $QUEST push D:/Dev/Mosis/MosisUnreal/Binaries/Android/main.1.com.omixlab.MosisUnreal.obb /data/local/tmp/obb/com.omixlab.MosisUnreal/

# Move OBB to final location
adb -s $QUEST shell "rm -rf /sdcard/Android/obb/com.omixlab.MosisUnreal"
adb -s $QUEST shell "mv /data/local/tmp/obb/com.omixlab.MosisUnreal /sdcard/Android/obb/"

# Start MosisService first, then MosisUnreal
adb -s $QUEST shell am start -n com.omixlab.mosis/.MainActivity
sleep 3
adb -s $QUEST shell am start -n com.omixlab.MosisUnreal/com.epicgames.unreal.GameActivity

# Monitor logs
adb -s $QUEST logcat -s MosisSDK MosisOS MosisTest

Full rebuild and deploy sequence:

# 1. Build Android APK+OBB
"D:\Epic\UE_5.5\Engine\Build\BatchFiles\RunUAT.bat" BuildCookRun \
    -project="D:\Dev\Mosis\MosisUnreal\MosisUnreal.uproject" \
    -platform=Android -clientconfig=Development \
    -build -cook -stage -pak -package -noP4

# 2. Stop app, push APK and OBB, restart
QUEST=2G0YC5ZF7W01T0
adb -s $QUEST shell am force-stop com.omixlab.MosisUnreal
adb -s $QUEST install -r D:/Dev/Mosis/MosisUnreal/Binaries/Android/MosisUnreal-arm64.apk
adb -s $QUEST shell "mkdir -p /data/local/tmp/obb/com.omixlab.MosisUnreal"
MSYS_NO_PATHCONV=1 adb -s $QUEST push D:/Dev/Mosis/MosisUnreal/Binaries/Android/main.1.com.omixlab.MosisUnreal.obb /data/local/tmp/obb/com.omixlab.MosisUnreal/
adb -s $QUEST shell "rm -rf /sdcard/Android/obb/com.omixlab.MosisUnreal && mv /data/local/tmp/obb/com.omixlab.MosisUnreal /sdcard/Android/obb/"
adb -s $QUEST shell am start -n com.omixlab.MosisUnreal/com.epicgames.unreal.GameActivity

Common issues:

App stuck on loading screen: OBB not pushed or wrong version
adb push path errors: Missing MSYS_NO_PATHCONV=1 prefix
Service not connecting: Start com.omixlab.mosis before the game

VR Pointer Interaction

The MosisSDK includes UMosisPointerComponent for VR ray-based touch interaction.

Key Files:

Public/MosisPointerComponent.h - Component header
Private/MosisPointerComponent.cpp - Implementation

Features:

Raycast from component transform (attach as child of motion controller)
Automatic detection of AMosisPhoneActor hits
Touch state machine: Down/Move/Up events
Optional Enhanced Input binding via TriggerAction property
Manual control via SetTriggerPressed(bool)
Debug ray visualization

Blueprint Setup:

Add MosisPointerComponent as child of MotionControllerComponent
Set TriggerAction to your trigger input action (optional)
Touch events are sent automatically when pointing at phone and triggering

C++ Setup:

// In VR Pawn header
UPROPERTY(VisibleAnywhere)
UMosisPointerComponent* RightPointer;

// In constructor
RightPointer = CreateDefaultSubobject<UMosisPointerComponent>(TEXT("RightPointer"));
RightPointer->SetupAttachment(RightMotionController);

// In BeginPlay (if using Enhanced Input)
RightPointer->TriggerAction = TriggerInputAction;

Manual Control (without Enhanced Input):

// In your input handling code
void AMyVRPawn::OnTriggerPressed()
{
    RightPointer->SetTriggerPressed(true);
}

void AMyVRPawn::OnTriggerReleased()
{
    RightPointer->SetTriggerPressed(false);
}

Component Properties:

Property	Type	Default	Description
`RayLength`	float	500.0	Maximum ray length in cm
`bShowDebugRay`	bool	false	Draw debug visualization
`DebugRayColor`	FLinearColor	Red	Ray color when not hitting
`DebugRayHitColor`	FLinearColor	Green	Ray color when hitting phone
`TraceChannel`	ECollisionChannel	Visibility	Collision channel for raycast
`TriggerAction`	UInputAction*	nullptr	Enhanced Input action for trigger

Blueprint Functions:

Function	Returns	Description
`IsPointingAtPhone()`	bool	True if ray hits a phone actor
`GetTargetPhone()`	AMosisPhoneActor*	Currently targeted phone (or nullptr)
`GetHitLocation()`	FVector	World location of ray hit
`GetRayOrigin()`	FVector	Ray start position
`GetRayDirection()`	FVector	Ray direction vector
`IsTriggerPressed()`	bool	Current trigger state
`SetTriggerPressed(bool)`	void	Manual trigger control

Touch State Machine:

Idle ──[raycast hit]──► Hover ──[trigger]──► Pressed
  ▲                       │                      │
  │                       │ raycast miss         │ trigger release
  │                       ▼                      ▼
  └──────────────────── Idle ◄────────────────────

Events:
- Idle → Pressed: SendTouch(Down)
- Pressed + moving: SendTouch(Move)
- Pressed → Idle: SendTouch(Up)

MosisVR (Unity 6000.3.2f1)

Location: D:\Dev\Mosis\MosisVR\Packages\com.omixlab.mosis_sdk\

Direct APK Build (Recommended)

"C:\Program Files\Unity\Hub\Editor\6000.3.2f1\Editor\Unity.exe" ^
    -batchmode -quit -nographics ^
    -projectPath "D:\Dev\Mosis\MosisVR" ^
    -executeMethod BuildScript.BuildAndroidDirectCI ^
    -outputPath "D:\Dev\Mosis\Builds\Unity\Android\MosisVR.apk"

Export + Gradle Build

For more control, export a Gradle project then build separately:

:: Step 1: Export from Unity
"C:\Program Files\Unity\Hub\Editor\6000.3.2f1\Editor\Unity.exe" ^
    -batchmode -quit -nographics ^
    -projectPath "D:\Dev\Mosis\MosisVR" ^
    -executeMethod BuildScript.BuildAndroidCI ^
    -export true ^
    -outputPath "D:\Dev\Mosis\Builds\Unity\Android\MosisVR"

:: Step 2: Build with Gradle
cd D:\Dev\Mosis\Builds\Unity\Android\MosisVR
gradle assembleRelease
:: APK at: launcher\build\outputs\apk\release\launcher-release.apk

Unity Editor Manual Build

File > Build Settings > Android
Player Settings: IL2CPP, ARM64, Vulkan + OpenGLES3
For direct APK: Uncheck "Export Project", click Build
For export: Check "Export Project", click Export

Native Plugin Build (Manual)

The native plugin builds automatically via CMake during Unity's build. To rebuild manually:

cd Packages/com.omixlab.mosis_sdk/Plugins/Android/cpp
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%ANDROID_NDK_HOME%/build/cmake/android.toolchain.cmake ^
    -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-29
cmake --build build

Device Testing (Both Engines)

# Install MosisService first
adb install -r MosisService-debug.apk

# Install game client
adb install -r MosisUnreal-arm64.apk  # or MosisVR.apk

# Launch service
adb shell am start -n com.omixlab.mosis/.MainActivity

# Launch client
adb shell am start -n com.omixlab.MosisUnreal/com.epicgames.unreal.GameActivity
# or for Unity:
adb shell am start -n com.omixlab.mosisvr/com.unity3d.player.UnityPlayerActivity

# Monitor all Mosis logs
adb logcat -s MosisSDK MosisTest RMLUI Vulkan

Vulkan HardwareBuffer Import

Both game engines use Vulkan to import AHardwareBuffer from MosisService.

Required Vulkan Extensions

VK_ANDROID_external_memory_android_hardware_buffer
VK_KHR_external_memory
VK_KHR_dedicated_allocation

Import Pattern

// 1. Query buffer properties
VkAndroidHardwareBufferPropertiesANDROID props = {
    .sType = VK_STRUCTURE_TYPE_ANDROID_HARDWARE_BUFFER_PROPERTIES_ANDROID
};
VkAndroidHardwareBufferFormatPropertiesANDROID formatProps = {
    .sType = VK_STRUCTURE_TYPE_ANDROID_HARDWARE_BUFFER_FORMAT_PROPERTIES_ANDROID
};
props.pNext = &formatProps;
vkGetAndroidHardwareBufferPropertiesANDROID(device, buffer, &props);

// 2. Create image with external memory
VkExternalMemoryImageCreateInfo extInfo = {
    .sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO,
    .handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_ANDROID_HARDWARE_BUFFER_BIT_ANDROID
};

AHardwareBuffer_Desc desc;
AHardwareBuffer_describe(buffer, &desc);

VkImageCreateInfo imageInfo = {
    .sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO,
    .pNext = &extInfo,
    .imageType = VK_IMAGE_TYPE_2D,
    .format = formatProps.format,
    .extent = {desc.width, desc.height, 1},
    .mipLevels = 1,
    .arrayLayers = 1,
    .samples = VK_SAMPLE_COUNT_1_BIT,
    .tiling = VK_IMAGE_TILING_OPTIMAL,
    .usage = VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT,
    .sharingMode = VK_SHARING_MODE_EXCLUSIVE
};
vkCreateImage(device, &imageInfo, nullptr, &image);

// 3. Import memory from HardwareBuffer
VkImportAndroidHardwareBufferInfoANDROID importInfo = {
    .sType = VK_STRUCTURE_TYPE_IMPORT_ANDROID_HARDWARE_BUFFER_INFO_ANDROID,
    .buffer = buffer
};

VkMemoryDedicatedAllocateInfo dedicatedInfo = {
    .sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO,
    .pNext = &importInfo,
    .image = image
};

VkMemoryAllocateInfo allocInfo = {
    .sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO,
    .pNext = &dedicatedInfo,
    .allocationSize = props.allocationSize,
    .memoryTypeIndex = FindMemoryType(props.memoryTypeBits, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)
};
vkAllocateMemory(device, &allocInfo, nullptr, &memory);
vkBindImageMemory(device, image, memory, 0);

Synchronization

Currently using CPU synchronization (vkQueueWaitIdle / glFinish). Future improvement: use Vulkan semaphores for GPU-GPU sync.

Double Buffering

The imported image is copied to a local texture each frame to prevent data races with MosisService rendering.

Developer Portal

The Developer Portal is a web application for app developers to publish and manage their Mosis apps. Planning documents are in DEV_PORTAL_M01-M12.md files.

Architecture Decisions

Component	Technology	Rationale
Backend	Go 1.22+	Simple, fast, single binary deployment
Router	Chi	Lightweight, idiomatic Go HTTP routing
Database	SQLite + Litestream	Zero-ops, continuous backup to NAS storage
Frontend	htmx + Go templates	Server-rendered, minimal JS, fast
Storage	Synology NAS filesystem	Self-hosted, local volume mounts
Auth	OAuth2 (GitHub/Google) + JWT	golang-jwt for tokens, Ed25519 for signing
CLI	Go + Cobra	Cross-platform single binary
Docs	Hugo + Docsy	Static site served from /docs/

Deployment Target

Self-hosted on Synology NAS via Docker:

/volume1/mosis/
├── data/
│   ├── portal.db       # Main SQLite database
│   └── telemetry.db    # Separate telemetry database
├── packages/           # Uploaded app packages
│   └── {dev_id}/{app_id}/{version}/
├── backups/            # Litestream replicas
└── docs/               # Hugo static site output

Milestone Documents

File	Topic	Status
DEV_PORTAL_M01_OVERVIEW.md	Project overview	Decided
DEV_PORTAL_M02_WEB_STACK.md	Go + Chi + htmx	Decided
DEV_PORTAL_M03_DATABASE.md	SQLite + Litestream	Decided
DEV_PORTAL_M04_AUTH.md	OAuth2 + JWT + Ed25519	Decided
DEV_PORTAL_M05_FRONTEND.md	htmx + Go templates	Decided
DEV_PORTAL_M06_API.md	REST API design	Decided
DEV_PORTAL_M07_STORAGE.md	NAS filesystem	Decided
DEV_PORTAL_M08_TELEMETRY.md	SQLite + background workers	Decided
DEV_PORTAL_M09_REVIEW.md	Go validation workers	Decided
DEV_PORTAL_M10_DEVICE.md	C++ AppManager	Decided
DEV_PORTAL_M11_CLI.md	Go + Cobra CLI	Decided
DEV_PORTAL_M12_DOCS.md	Hugo + Docsy	Decided

Key Design Principles

Single container - Portal runs as one Docker container on NAS
No external services - SQLite, local filesystem, Pagefind search
Pure Go - modernc.org/sqlite (no CGO required)
Server-rendered - htmx for interactivity, no SPA framework
Background workers - Go goroutines for aggregation/cleanup jobs

32 KiB Raw Blame History

CLAUDE.md

Company

Git Commit Guidelines

Project Overview

Current Status

Project Components

Build Commands

Android (Gradle)

Desktop Designer (CMake)

Designer Tests (CMake)

Sandbox Security Tests (CMake)

Environment Requirements

Architecture Overview

Two Native Libraries

IPC Flow

Key Interfaces (AIDL)

Native Code Structure (src/main/cpp/)

Code Style

Dependencies

Desktop Designer

Key Files

Command Line Options

Keyboard Controls

Automated Testing Framework

Test Architecture

Key Implementation Details

Writing Tests

Test Output

UI Assets Structure

Navigation System

Element IDs for Testing

CSS Classes for Navigation

Material Design Resources

Material Design Icons

Material Design Lite

Using Icons in Mosis

Android Device Testing

Prerequisites

Build and Install

Run Gradle Connected Tests

Event Injection via ADB

Dock Element Coordinates (Normalized)

Full Navigation Test Sequence

Reading Logs

Expected Log Output

Documentation Guidelines

Where to Put Documentation

What to Document

When to Document

Game Engine Integrations

Integration Architecture

MosisUnreal (Unreal Engine 5.5)

Build Commands

Output Files

Requirements

Quest Deployment

VR Pointer Interaction

MosisVR (Unity 6000.3.2f1)

Direct APK Build (Recommended)

Export + Gradle Build

Unity Editor Manual Build

Native Plugin Build (Manual)

Device Testing (Both Engines)

Vulkan HardwareBuffer Import

Required Vulkan Extensions

Import Pattern

Synchronization

Double Buffering

Developer Portal

Architecture Decisions

Deployment Target

Milestone Documents

Key Design Principles

32 KiB

Raw Blame History