omar/MosisService
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

move docs to docs/ folder, merge architecture files, update references

Browse Source
This commit is contained in:
omigamedev
2026-01-19 09:02:11 +01:00
parent 1b34b0e974
commit 010e11cf6b
68 changed files with 1741 additions and 1350 deletions
Download Patch File Download Diff File Expand all files Collapse all files

70
docs/AGENTS.md Normal file
View File

@@ -0,0 +1,70 @@
# MosisService Agent Guidelines
## Build, Lint, and Test Commands
### Build Commands
- `./gradlew build` - Build the entire project
- `./gradlew assembleDebug` - Build debug version
- `./gradlew assembleRelease` - Build release version
- `./gradlew clean` - Clean build outputs
- `./gradlew :app:build` - Build specific module (if needed)
### Lint Commands
- `./gradlew lint` - Run lint checks
- `./gradlew lintDebug` - Run lint for debug build
- `./gradlew lintRelease` - Run lint for release build
### Test Commands
- `./gradlew test` - Run unit tests
- `./gradlew connectedAndroidTest` - Run connected device tests
- `./gradlew testDebugUnitTest` - Run debug unit tests
- `./gradlew testReleaseUnitTest` - Run release unit tests
- `./gradlew :app:test` - Run tests for specific module
- `./gradlew :app:testDebugUnitTest` - Test specific module debug unit tests
**Important Note:** The project contains C++ native code that uses CMake with both `mosis-service` and `mosis-test` libraries defined in `CMakeLists.txt`. Tests are built as shared libraries with the suffix `.so`.
## Code Style Guidelines
### Naming Conventions
- `PascalCase` for class and function names (e.g., `MosisService`, `RenderTexture`)
- `camelCase` for variables and parameters (e.g., `renderTarget`, `inputData`)
- `UPPER_CASE` for constants (e.g., `MAX_BUFFER_SIZE`, `DEFAULT_TIMEOUT`)
- Class names should be descriptive and follow Android naming conventions (e.g., `MosisService`)
- Files should use meaningful names that match their content
### Imports & Formatting
- Import statements are grouped logically: Android imports, third-party imports, first-party imports
- Imports are sorted alphabetically within each group
- Use of fully qualified names is allowed for clarity when needed
### Types & Variables
- Use primitive types when possible
- Use `final` keyword for constants
- Prefer `val` over `var` in Kotlin
- Use `final` for native C++ variables that don't change
- Use appropriate data types (int, long, float, double) based on precision requirements
### Error Handling
- C++ code uses standard C++ exception handling
- Android code leverages standard Android error handling patterns
- Use try-catch blocks appropriately in Kotlin code
- Log errors via Android's logging framework in Kotlin
- C++ logging via `logger.cpp` component
### Coding Standards
- Follow Android development best practices
- C++ code should follow modern C++23 conventions (including use of std::span, std::format, etc.)
- Use RAII (Resource Acquisition Is Initialization) principles in C++
- Ensure memory management is handled properly in native code
- Avoid magic numbers in C++ code - use constants instead
- Use smart pointers like `std::unique_ptr` and `std::shared_ptr` where appropriate
- Proper resource cleanup in destructors
- Avoid memory leaks in C++ components
### Structure
- Kotlin code following Android project structures
- C++ code organized in separate files within `src/main/cpp/`
- Clear separation between platform-specific code and shared logic
- Use Android build system capabilities for multi-architecture support
- Follow Gradle conventions for dependency management

129
docs/ANDROID-TESTING.md Normal file
View File

@@ -0,0 +1,129 @@
# Android Device Testing
## Prerequisites
```bash
# Check connected device
adb devices -l
# Verify Mosis app is installed
adb shell pm list packages | grep mosis
```
## Build and Install
```bash
# Build debug APK
./gradlew assembleDebug
# Install on device
adb install -r build/outputs/apk/debug/MosisService-debug.apk
# Launch the app
adb shell am start -n com.omixlab.mosis/.MainActivity
```
## Run Gradle Connected Tests
```bash
# Run all connected Android tests
./gradlew connectedAndroidTest
```
## Event Injection via ADB
Inject touch events for automated testing:
```bash
# Click at normalized coordinates (0.0-1.0)
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.5 --ef y 0.5
# Touch down
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "down" --ef x 0.2 --ef y 0.9
# Touch up
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "up" --ef x 0.2 --ef y 0.9
```
## Dock Element Coordinates (Normalized)
| Element | X | Y |
|---------|---|---|
| dock-phone | 0.16 | 0.97 |
| dock-messages | 0.39 | 0.97 |
| dock-contacts | 0.61 | 0.97 |
| dock-browser | 0.84 | 0.97 |
| back-button | 0.10 | 0.05 |
## Full Navigation Test Sequence
```bash
# Clear logs and run navigation test sequence
adb logcat -c
# Click Phone dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.16 --ef y 0.97
sleep 2
# Click back to return home
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.1 --ef y 0.05
sleep 2
# Click Messages dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.39 --ef y 0.97
sleep 2
# Click back to return home
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.1 --ef y 0.05
sleep 2
# Click Contacts dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.61 --ef y 0.97
sleep 2
# Click back to return home
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.1 --ef y 0.05
sleep 2
# Click Browser dock icon
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" --ef x 0.84 --ef y 0.97
```
## Reading Logs
```bash
# Filter for Mosis logs
adb logcat -s MosisTest ServiceTester RMLUI
# Filter for navigation events
adb logcat -d | grep -iE "navigat|loaded|goBack|rml"
# Save to file
adb logcat -s MosisTest > mosis-log.txt
# Clear logs
adb logcat -c
```
## Expected Log Output
Successful navigation shows these log patterns:
```
RMLUI: navigateTo called with: dialer
Loading screen: apps/dialer/dialer.rml
RMLUI: Navigated to: dialer (history depth: 1)
RMLUI: goBack called (history depth: 1)
Loading screen: apps/home/home.rml
RMLUI: Back to: home
```

99
docs/APP-MANAGEMENT.md Normal file
View File

@@ -0,0 +1,99 @@
# App Management System
The device-side app management system handles installation, updates, and launching of third-party apps.
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ AppManager │
│ - Install/Uninstall apps from .mosis packages │
│ - Track installed apps in JSON registry │
│ - Manage app data/cache directories │
└─────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ LuaSandboxManager │
│ - StartApp/StopApp lifecycle │
│ - Per-app isolated Lua environments │
│ - Resource limits (memory, CPU, timers) │
└─────────────────────────────────────────────────────────────┘
```
## App Package Format (.mosis)
Apps are distributed as ZIP files with `.mosis` extension:
```
myapp.mosis
├── manifest.json # Required: app metadata
├── main.rml # Entry point (RmlUi document)
├── styles.rcss # Stylesheets
├── scripts/ # Lua scripts
│ └── app.lua
└── assets/ # Icons, images, etc.
```
## Manifest Format
```json
{
"id": "com.example.myapp",
"name": "My App",
"version": "1.0.0",
"version_code": 1,
"entry": "main.rml",
"icon": "icon.tga",
"description": "App description",
"developer": {
"name": "Developer Name",
"email": "dev@example.com"
},
"permissions": [
"network",
"storage",
"camera"
],
"min_api_version": 1
}
```
## App Lifecycle
```cpp
// Install from local file
app_manager->InstallFromFile("/path/to/app.mosis", [](auto progress) {
LOG_INFO("Install progress: %s %.0f%%",
InstallProgress::StageName(progress.stage),
progress.progress * 100);
});
// Launch app (starts sandbox)
app_manager->LaunchApp("com.example.myapp");
// Check if running
bool running = app_manager->IsAppRunning("com.example.myapp");
// Stop app (cleanup sandbox)
app_manager->StopApp("com.example.myapp");
// Uninstall (stops if running, removes files)
app_manager->Uninstall("com.example.myapp", false); // keep_data=false
```
## Directory Structure
```
/data/data/com.omixlab.mosis/files/
├── apps/
│ └── com.example.myapp/
│ ├── package/ # Extracted app files
│ ├── data/ # App persistent data (VirtualFS)
│ ├── cache/ # App cache (clearable)
│ └── db/ # SQLite databases
├── downloads/ # Temporary download location
├── backups/ # App data backups
└── config/
└── apps.json # Installed apps registry
```

667
docs/APP_SPECS.md Normal file
View File

@@ -0,0 +1,667 @@
# Mosis App Specifications
This document defines the complete specification for Mosis apps, including package format, manifest schema, permissions, and available APIs.
---
## Package Format
Mosis apps are distributed as `.mosis` files, which are signed ZIP archives.
### File Structure
```
com.developer.appname-1.0.0.mosis
├── manifest.json # App metadata (required)
├── META-INF/
│ ├── MANIFEST.MF # SHA-256 hashes of all files
│ ├── CERT.PEM # Developer's public key
│ └── CERT.SIG # Ed25519 signature of MANIFEST.MF
├── assets/
│ ├── main.rml # Entry point (required)
│ ├── screens/ # Additional RML screens
│ ├── styles/ # RCSS stylesheets
│ └── scripts/ # Lua scripts
├── icons/
│ ├── icon-32.png # 32x32 icon
│ ├── icon-64.png # 64x64 icon
│ └── icon-128.png # 128x128 icon
└── locales/ # Optional i18n files
├── en.json
└── es.json
```
### Size Limits
| Limit | Value |
|-------|-------|
| Max package size | 50 MB |
| Max individual file | 10 MB |
| Max files count | 1000 |
| Max path length | 256 chars |
| Max manifest size | 64 KB |
### Allowed File Types
| Category | Extensions |
|----------|------------|
| UI | `.rml` |
| Styles | `.rcss` |
| Scripts | `.lua` |
| Images | `.png`, `.jpg`, `.jpeg`, `.tga`, `.webp` |
| Fonts | `.ttf`, `.otf` |
| Data | `.json` |
| Audio | `.ogg`, `.wav`, `.mp3` |
**Forbidden**: `.exe`, `.dll`, `.so`, `.dylib`, `.sh`, `.bat`, `.ps1`, `.py`, `.js`, nested archives
---
## Manifest Schema
### Required Fields
```json
{
"$schema": "https://mosis.dev/schemas/manifest-v1.json",
"id": "com.developer.appname",
"name": "My App",
"version": "1.0.0",
"version_code": 1,
"entry": "assets/main.rml",
"min_mosis_version": "1.0.0"
}
```
### Full Schema
```json
{
"$schema": "https://mosis.dev/schemas/manifest-v1.json",
"id": "com.developer.appname",
"name": "My App",
"version": "1.0.0",
"version_code": 1,
"description": "A short description of the app",
"entry": "assets/main.rml",
"author": {
"name": "Developer Name",
"email": "dev@example.com",
"url": "https://developer.com"
},
"permissions": [
"storage",
"network.internet",
"camera"
],
"icons": {
"32": "icons/icon-32.png",
"64": "icons/icon-64.png",
"128": "icons/icon-128.png"
},
"min_mosis_version": "1.0.0",
"target_mosis_version": "1.2.0",
"category": "utilities",
"tags": ["productivity", "tools"],
"orientation": "portrait",
"background_color": "#FFFFFF",
"locales": ["en", "es", "fr"],
"default_locale": "en",
"network": {
"allowed_domains": [
"api.example.com",
"*.cdn.example.com"
],
"allow_http": false,
"max_connections": 10
}
}
```
### Field Reference
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `id` | string | Yes | Unique package ID (reverse domain: `^[a-z][a-z0-9]*(\.[a-z][a-z0-9]*)+$`) |
| `name` | string | Yes | Display name (max 30 chars) |
| `version` | string | Yes | Semantic version (X.Y.Z) |
| `version_code` | integer | Yes | Incremental build number (must increase with each release) |
| `entry` | string | Yes | Path to entry RML file |
| `min_mosis_version` | string | Yes | Minimum Mosis version required |
| `description` | string | No | Short description (max 80 chars) |
| `author` | object | No | Author information |
| `permissions` | array | No | Required permissions |
| `icons` | object | No | Icon paths by size |
| `category` | string | No | App store category |
| `tags` | array | No | Searchable tags |
| `orientation` | string | No | `portrait`, `landscape`, `any` |
| `background_color` | string | No | Hex color for loading screen |
| `locales` | array | No | Supported locale codes |
---
## Signing
### Algorithm
- **Key type**: Ed25519
- **Signature size**: 64 bytes
- **Hash**: SHA-256 for file manifests
### MANIFEST.MF Format
```
Manifest-Version: 1.0
Created-By: mosis-cli 1.0.0
Name: manifest.json
SHA-256-Digest: base64encodedHash==
Name: assets/main.rml
SHA-256-Digest: base64encodedHash==
Name: assets/scripts/main.lua
SHA-256-Digest: base64encodedHash==
```
### Signing Flow
1. Generate `MANIFEST.MF` with SHA-256 hash of each file
2. Sign `MANIFEST.MF` with developer's Ed25519 private key
3. Store signature in `META-INF/CERT.SIG`
4. Include developer's public key in `META-INF/CERT.PEM`
### Verification Flow
1. Extract `META-INF/MANIFEST.MF`
2. Verify signature using `CERT.PEM`
3. Verify `CERT.PEM` is registered with developer account
4. Verify each file hash matches `MANIFEST.MF`
---
## Permissions
Apps must declare required permissions in the manifest. Some permissions are auto-granted, others require user approval.
### Permission Categories
| Permission | Description | Risk Level |
|------------|-------------|------------|
| `storage` | App-private file access | Normal (auto-granted) |
| `network.internet` | HTTP requests | Dangerous |
| `network.websocket` | WebSocket connections | Dangerous |
| `camera` | Camera access | Dangerous |
| `microphone` | Audio recording | Dangerous |
| `location.coarse` | Approximate location | Dangerous |
| `location.fine` | Precise GPS location | Dangerous |
| `contacts.read` | Read contacts | Dangerous |
| `contacts.write` | Modify contacts | Dangerous |
| `bluetooth` | Bluetooth access | Dangerous |
| `sensors.body` | Body sensors (heart rate) | Dangerous |
| `clipboard.read` | Read clipboard | Dangerous |
| `clipboard.write` | Write to clipboard | Dangerous |
| `system.notifications` | Show notifications | Normal |
### Requesting Permissions
```lua
-- Request a dangerous permission (shows system dialog)
local granted = permissions.request("camera")
if granted then
-- Permission granted, can now use camera API
camera.start()
else
-- Permission denied, show fallback UI
showPermissionDeniedMessage()
end
```
### User Gesture Requirements
Dangerous hardware permissions (`camera`, `microphone`, `location`) require a recent user gesture (within 5 seconds) before the permission request is shown.
---
## Runtime Limits
Apps run in a sandboxed Lua environment with resource limits.
### Memory Limits
| Resource | Limit |
|----------|-------|
| Total memory | 16 MB |
| Max string size | 1 MB |
| Max table entries | 100,000 |
### CPU Limits
| Context | Instruction Limit | Approx Time |
|---------|-------------------|-------------|
| Event handler | 1,000,000 | ~10ms |
| Init/load | 10,000,000 | ~100ms |
| Timer callback | 500,000 | ~5ms |
### Storage Limits
| Resource | Limit |
|----------|-------|
| App storage | 100 MB |
| Cache storage | 50 MB |
| Single file | 50 MB |
| Files per directory | 10,000 |
| SQLite database | 50 MB |
---
## Lua APIs
### Safe Built-in Globals
```lua
-- Retained from standard Lua
print -- Redirected to system logger
type, tonumber, tostring
pairs, ipairs, next
pcall, xpcall
assert, error
select, unpack
string.* -- Except string.dump
table.*
math.*
utf8.*
```
### Removed Globals (Security)
```lua
-- These do NOT exist in the sandbox
dofile, loadfile, load, loadstring
rawget, rawset, rawequal, rawlen
getmetatable, setmetatable
collectgarbage
os, io, debug, package, require
ffi, jit, newproxy
```
---
## Storage API
```lua
-- App-private storage (auto-granted)
storage.write("data/config.json", json.encode(config))
local data = storage.read("data/config.json")
local exists = storage.exists("data/config.json")
storage.delete("data/config.json")
storage.mkdir("data/subfolder")
local files = storage.list("data/")
-- Cache storage (can be cleared by system)
storage.write("cache/temp.dat", data)
-- Temp storage (cleared on app stop)
storage.write("temp/session.dat", data)
```
---
## Database API (SQLite)
```lua
-- Open database (creates if not exists)
local db = database.open("mydata.db")
-- Execute SQL
db:exec([[
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
email TEXT UNIQUE
)
]])
-- Prepared statements with parameters
local stmt = db:prepare("INSERT INTO users (name, email) VALUES (?, ?)")
stmt:bind("John Doe", "john@example.com")
stmt:exec()
-- Query with results
local rows = db:query("SELECT * FROM users WHERE name LIKE ?", "%John%")
for _, row in ipairs(rows) do
print(row.id, row.name, row.email)
end
-- Close when done
db:close()
```
---
## Network API
### HTTP Requests
```lua
-- Requires "network.internet" permission
local response = network.request({
url = "https://api.example.com/data",
method = "POST", -- GET, POST, PUT, DELETE, PATCH
headers = {
["Content-Type"] = "application/json",
["Authorization"] = "Bearer " .. token
},
body = json.encode({key = "value"}),
timeout = 30000 -- milliseconds
})
-- Response structure
if response.error then
print("Error:", response.error)
else
print("Status:", response.status) -- 200, 404, etc.
print("Body:", response.body) -- Response body string
-- response.headers: table of headers
end
```
### WebSocket
```lua
-- Requires "network.websocket" permission
local ws = network.websocket("wss://api.example.com/ws")
ws:on("open", function()
ws:send(json.encode({type = "hello"}))
end)
ws:on("message", function(data)
local msg = json.decode(data)
print("Received:", msg.type)
end)
ws:on("close", function(code, reason)
print("Closed:", code, reason)
end)
ws:on("error", function(err)
print("Error:", err)
end)
-- Send data
ws:send("Hello, server!")
-- Close connection
ws:close()
```
---
## Camera API
```lua
-- Requires "camera" permission + user gesture
local granted = permissions.request("camera")
if not granted then return end
-- Start camera preview
camera.start({
facing = "back", -- "back" or "front"
resolution = "720p", -- "480p", "720p", "1080p"
target = "preview" -- Element ID to render preview
})
-- Capture photo
camera.capture(function(image)
-- image.data: base64 encoded JPEG
-- image.width, image.height: dimensions
storage.write("photos/capture.jpg", base64.decode(image.data))
end)
-- Stop camera
camera.stop()
```
---
## Microphone API
```lua
-- Requires "microphone" permission + user gesture
local granted = permissions.request("microphone")
if not granted then return end
-- Start recording
microphone.start({
format = "wav", -- "wav" or "ogg"
sample_rate = 44100,
channels = 1
})
-- Stop and get recording
microphone.stop(function(audio)
-- audio.data: base64 encoded audio
-- audio.duration: length in seconds
storage.write("recordings/voice.wav", base64.decode(audio.data))
end)
```
---
## Location API
```lua
-- Requires "location.fine" or "location.coarse" permission
local granted = permissions.request("location.fine")
if not granted then return end
-- Get current location
location.get(function(pos)
if pos.error then
print("Error:", pos.error)
else
print("Lat:", pos.latitude)
print("Lon:", pos.longitude)
print("Accuracy:", pos.accuracy) -- meters
end
end)
-- Watch location changes
local watch_id = location.watch(function(pos)
print("Location updated:", pos.latitude, pos.longitude)
end, {
interval = 5000, -- minimum time between updates (ms)
distance = 10 -- minimum distance change (meters)
})
-- Stop watching
location.unwatch(watch_id)
```
---
## Sensors API
```lua
-- Motion sensors (no permission required)
sensors.accelerometer.start(function(data)
print("Accel:", data.x, data.y, data.z)
end, { interval = 100 }) -- ms
sensors.gyroscope.start(function(data)
print("Gyro:", data.x, data.y, data.z)
end)
sensors.magnetometer.start(function(data)
print("Mag:", data.x, data.y, data.z)
end)
-- Stop sensor
sensors.accelerometer.stop()
-- Body sensors (requires "sensors.body" permission)
local granted = permissions.request("sensors.body")
if granted then
sensors.heartrate.start(function(data)
print("Heart rate:", data.bpm)
end)
end
```
---
## Timer API
```lua
-- One-shot timer
local timer_id = timer.after(1000, function()
print("1 second elapsed")
end)
-- Repeating timer
local interval_id = timer.every(500, function()
print("Every 500ms")
end)
-- Cancel timers
timer.cancel(timer_id)
timer.cancel(interval_id)
```
---
## JSON API
```lua
-- Encode Lua table to JSON string
local json_str = json.encode({
name = "John",
age = 30,
active = true,
tags = {"lua", "mosis"}
})
-- Decode JSON string to Lua table
local data = json.decode(json_str)
print(data.name) -- "John"
```
---
## Crypto API
```lua
-- Hashing
local hash = crypto.sha256("hello world")
local hash = crypto.sha512("hello world")
local hash = crypto.md5("hello world") -- Not for security
-- HMAC
local mac = crypto.hmac("sha256", "secret key", "message")
-- Random bytes
local random = crypto.random(32) -- 32 random bytes (base64)
-- UUID generation
local uuid = crypto.uuid() -- "550e8400-e29b-41d4-a716-446655440000"
```
---
## Navigation
```lua
-- Navigate to another screen
navigateTo("screens/settings.rml")
-- Go back
goBack()
-- Go to home screen
goHome()
-- Get current screen
local screen = getCurrentScreen()
```
---
## App Categories
Valid categories for the `category` manifest field:
| Category | Description |
|----------|-------------|
| `utilities` | Tools and utilities |
| `productivity` | Work and productivity |
| `communication` | Messaging and social |
| `entertainment` | Games and media |
| `lifestyle` | Health, fitness, lifestyle |
| `finance` | Banking and finance |
| `education` | Learning and reference |
| `news` | News and magazines |
| `travel` | Travel and navigation |
| `shopping` | Shopping and commerce |
---
## Error Handling
```lua
-- Safe API calls with pcall
local ok, result = pcall(function()
return network.request({url = "https://api.example.com"})
end)
if not ok then
print("Error:", result)
end
-- API errors return error field
local response = network.request({url = "https://invalid"})
if response.error then
print("Network error:", response.error)
end
```
---
## Best Practices
### Do
- Request only the permissions you need
- Handle permission denials gracefully
- Cache network responses when appropriate
- Clean up timers and resources when done
- Use try/catch patterns for error handling
- Store sensitive data encrypted
### Don't
- Request permissions before they're needed
- Store sensitive data in plain text
- Make unnecessary network requests
- Hold camera/microphone open unnecessarily
- Create infinite loops or excessive recursion
- Assume network is always available
---
## References
- [RmlUi Documentation](https://mikke89.github.io/RmlUiDoc/)
- [Lua 5.4 Reference Manual](https://www.lua.org/manual/5.4/)
- [Ed25519 Specification](https://ed25519.cr.yp.to/)

105
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,105 @@
# Architecture Overview
MosisService is an Android application combining Kotlin UI with native C++ libraries for UI rendering via Android's Binder IPC system.
## Two Native Libraries
**mosis-service** (`libmosis-service.so`):
- Main Android Binder service implementation
- Implements `IMosisService.aidl` interface for touch events and initialization
- Contains the Kernel rendering engine with RmlUi integration
- Links against RmlUi for HTML/CSS-like UI rendering
**mosis-test** (`libmosis-test.so`):
- Test/rendering client implementation
- Implements `IMosisListener.aidl` for receiving callbacks
- OpenGL ES 2.0 rendering pipeline using GLAD
## IPC Flow
```
Kotlin NativeService → JNI → mosis-service (IMosisService)
↓
IMosisListener callbacks
↓
mosis-test (rendering client)
```
## Key Interfaces (AIDL)
`IMosisService`: `initOS()`, `onTouchDown()`, `onTouchMove()`, `onTouchUp()`
`IMosisListener` (oneway/async): `onServiceInitialized()`, `onBufferAvailable()`, `onFrameAvailable()`
## Multi-threading Model
- Service layer runs in main thread for event handling
- Rendering loop runs in dedicated thread managed by Kernel
- Async task processing for UI updates
- Thread-safe communication between components using std::mutex
## Rendering Pipeline
1. **Initialization**: Service connects to test layer, creates EGL context
2. **Buffer Management**: Hardware buffers allocated and shared between layers
3. **Event Processing**: Touch events processed and forwarded to Kernel
4. **Rendering Loop**: Continuous rendering with frame synchronization
5. **UI Updates**: RmlUi engine updates UI based on events and data
### Data Flow
```
User Touch Events → IMosisService.onTouch* → Kernel.process → RmlUi UI Updates
Service Initialized → IMosisListener.onServiceInitialized → Rendering Setup
Buffer Available → IMosisListener.onBufferAvailable → Texture Creation
Frame Available → IMosisListener.onFrameAvailable → Frame Rendering
```
## Native Code Structure (src/main/cpp/)
**Core:**
- `kernel.cpp` - Core rendering engine, RmlUi integration, event processing
- `mosis-service.cpp` - Binder service implementation, JNI entry points
- `mosis-test.cpp` - Test client implementation
- `egl_context.cpp` - OpenGL ES context management
- `render_target.cpp` - Framebuffer and buffer management
- `RmlUi_Renderer_GL3.cpp` - RmlUi OpenGL renderer backend
- `assets_manager.cpp` - Android AssetManager integration
**App Management (`apps/`):**
- `app_manager.cpp` - App install/uninstall/launch lifecycle
- `app_api.cpp` - Lua API bindings for app management
- `update_service.cpp` - Background update checking
**Lua Sandbox (`sandbox/`):**
- `sandbox_manager.cpp` - Multi-app sandbox orchestrator
- `lua_sandbox.cpp` - Core Lua sandbox with resource limits
- `permission_gate.cpp` - Permission system (normal/dangerous/signature)
- `virtual_fs.cpp` - Per-app virtual filesystem with quotas
- `database_manager.cpp` - SQLite database per app
- `network_manager.cpp` - HTTP request validation
- `websocket_manager.cpp` - WebSocket connections
- `timer_manager.cpp` - setTimeout/setInterval implementation
- `json_api.cpp` - Safe JSON encode/decode
- `crypto_api.cpp` - Cryptographic functions (SHA256, HMAC)
- `camera_interface.cpp` - Camera access with indicators
- `microphone_interface.cpp` - Microphone access with indicators
- `audio_output.cpp` - Audio playback
- `location_interface.cpp` - GPS with precision reduction
- `sensor_interface.cpp` - Accelerometer, gyroscope, etc.
- `bluetooth_interface.cpp` - Bluetooth device access
- `contacts_interface.cpp` - Contacts read/write
- `message_bus.cpp` - Inter-app communication
## Code Style
- C++23 standard with modern features (std::span, std::format)
- PascalCase for classes/functions, camelCase for variables
- RAII principles with smart pointers
- Kotlin code follows Android conventions
## Dependencies
- vcpkg manages native dependencies (RmlUi, GLFW, Freetype, Lua, libpng, nlohmann-json, minizip, sqlite3)
- CMake build system with vcpkg toolchain integration
- Android target architecture: arm64-v8a only
- Desktop target: Windows x64 (MSVC)

86
docs/BUILD-COMMANDS.md Normal file
View File

@@ -0,0 +1,86 @@
# Build Commands
## Android (Gradle)
```bash
# Build entire project
./gradlew build
# Build debug APK
./gradlew assembleDebug
# Build release APK
./gradlew assembleRelease
# Clean build outputs
./gradlew clean
# Build with verbose output
./gradlew build --info --stacktrace
# Run lint checks
./gradlew lint
# Run unit tests
./gradlew test
# Run connected device tests
./gradlew connectedAndroidTest
```
## Desktop Designer (CMake)
```bash
# Configure (from designer/ folder)
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake
# Build
cmake --build build --config Debug
# Run with hot-reload
./build/Debug/mosis-designer.exe ../src/main/assets/apps/home/home.rml
# Run with logging and hierarchy dump
./build/Debug/mosis-designer.exe ../src/main/assets/apps/home/home.rml --log output.log --hierarchy hierarchy.json
```
## Designer Tests (CMake)
```bash
# Configure (from designer-test/ folder)
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake
# Build
cmake --build build --config Debug
# Run tests
./build/Debug/designer-test.exe
```
## Sandbox Security Tests (CMake)
```bash
# Configure (from sandbox-test/ folder)
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake
# Build
cmake --build build --config Debug
# Run all tests (uber command)
./run_tests.bat
# Or run directly
./build/Debug/sandbox-test.exe
# Run specific test
./build/Debug/sandbox-test.exe --test DangerousGlobals
./build/Debug/sandbox-test.exe --test Memory
./build/Debug/sandbox-test.exe --test CPU
```
## Environment Requirements
Required environment variables:
- `ANDROID_HOME` - Android SDK path
- `ANDROID_NDK_HOME` - Android NDK path (version 29.0.14206865)
- `VCPKG_ROOT` - vcpkg package manager root

210
docs/BUILD.md Normal file
View File

@@ -0,0 +1,210 @@
# MosisService Build Configuration
## Overview
This document outlines the build configuration and toolchain information for the MosisService project, optimized for maximum compiler feedback and debugging capabilities on Windows.
## Gradle Build Configuration
### Build Flags and Options
The project uses Gradle with Android build system that includes the following optimization flags:
### Android Build Configuration
In `build.gradle.kts`:
- Compile SDK: API level 36 (Android 14)
- Target SDK: API level 36
- Min SDK: API level 34 (Android 14)
- ndkVersion: 29.0.14206865
- ABI: arm64-v8a only
### Optimization and Debug Flags
- JVM target: JDK 11
- C++ standard: C++23
- Debug builds include full symbol information
- Release builds include optimization flags (-O2)
## CMake Build Configuration
### CMake Flags for Maximum Feedback
In `CMakeLists.txt`:
```
set(CMAKE_CXX_STANDARD 23)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
```
### Compiler Flags for Windows
```
# Debug flags for maximum feedback
set(CMAKE_CXX_FLAGS_DEBUG "-g -O0 -Wall -Wextra -Wpedantic -Werror")
set(CMAKE_CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
# Enable all warnings
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-unused-parameter -Wno-unused-variable")
```
### C++23 Features and Debugging
- Enabled C++23 standard with std::span, std::format
- Debug build includes full debug symbols
- Address Sanitizer enabled in debug builds
- Undefined Behavior Sanitizer enabled in debug builds
## Windows Build Toolchain
### Required Tools
1. **Android Studio with Android NDK**
- Android SDK
- Android NDK (version 29.0.14206865)
- CMake (version 3.22.1 or higher)
2. **Visual Studio 2022**
- C++ development tools
- Windows SDK
- CMake tools for Visual Studio
3. **vcpkg**
- Package manager for dependencies
- RmlUi with Lua support (if available)
### Environment Variables
```
ANDROID_HOME=C:\Users\%USERNAME%\AppData\Local\Android\Sdk
ANDROID_NDK_HOME=C:\Users\%USERNAME%\AppData\Local\Android\Sdk\ndk\29.0.14206865
VCPKG_ROOT=C:\Tools\vcpkg
```
### Build Commands
For maximum feedback:
#### Debug Build with All Warnings
```
./gradlew assembleDebug -Pandroid.jetifier.enable=true -Pandroid.build.legacyBundleTool=false
```
#### Release Build with Optimization
```
./gradlew assembleRelease -Pandroid.jetifier.enable=true -Pandroid.build.legacyBundleTool=false
```
## Build Process Steps
### 1. Prerequisites Setup
- Install Android Studio with NDK
- Install Visual Studio 2022 with C++ tools
- Set up vcpkg and install required dependencies
- Configure environment variables
### 2. Build Configuration
```
# Clean previous builds
./gradlew clean
# Build with verbose output for maximum feedback
./gradlew build --info --stacktrace --configure-on-demand
# Build with all warnings enabled
./gradlew compileDebugKotlin --warning-mode all
```
### 3. CMake Configuration
To enable full compiler feedback in CMake:
```
# Debug configuration with all warnings
cmake -DCMAKE_BUILD_TYPE=Debug
-DCMAKE_CXX_FLAGS_DEBUG="-g -O0 -Wall -Wextra -Wpedantic -Werror -fsanitize=address,undefined"
-DCMAKE_CXX_STANDARD=23
-DCMAKE_CXX_STANDARD_REQUIRED=ON
.
# Release configuration
cmake -DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_FLAGS_RELEASE="-O3 -DNDEBUG -flto=full"
.
```
## Output Locations
### Android APKs
```
build\outputs\apk\debug\app-debug.apk
build\outputs\apk\release\app-release-unsigned.apk
```
### Native Libraries
```
build\intermediates\cmake\debug\obj\arm64-v8a\libmosis-service.so
build\intermediates\cmake\debug\obj\arm64-v8a\libmosis-test.so
```
### Build Artifacts
```
build\intermediates\cxx\Debug\1r2562ic\arm64-v8a\
build\intermediates\cmake\debug\output\lib\arm64-v8a\
```
## Troubleshooting Build Issues
### Common Debug Flags
```
# Enable verbose C++ compilation
./gradlew compileDebugCpp --info
# Enable all warnings and stop on first error
./gradlew build -PcompileDebugCpp.warningMode=all
# Show compiler output
./gradlew build --console=plain
```
### Debugging Symbols
To include debugging symbols:
```
# In CMakeLists.txt
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -g3 -ggdb")
```
### Compiler-Specific Flags
```
# For MSVC (if using Visual Studio)
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /W4 /WX /Od /Zi")
# For Clang (if building with clang)
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -fstandalone-debug -fexceptions -frtti")
```
## Toolchain Locations
### Android SDK
```
C:\Users\%USERNAME%\AppData\Local\Android\Sdk
```
### Android NDK
```
C:\Users\%USERNAME%\AppData\Local\Android\Sdk\ndk\29.0.14206865
```
### Visual Studio Tools
```
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\bin\Hostx64\x64\
```
### CMake Location
```
C:\Program Files\CMake\bin\cmake.exe
```
## Recommended Build Commands
### For Development with Maximum Feedback
```
# Clean and build with maximum warnings
./gradlew clean && ./gradlew build --warning-mode all --info --stacktrace
# Debug build with sanitizer
./gradlew assembleDebug -Pandroid.jetifier.enable=true
```
### For Release Builds
```
# Release build with optimization
./gradlew assembleRelease -Pandroid.jetifier.enable=true
```

106
docs/CLAUDE.md Normal file
View File

@@ -0,0 +1,106 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Company
**OmixLab LTD** - Package namespace: `com.omixlab`
## Git Commit Guidelines
**IMPORTANT**: When creating git commits:
- **DO NOT** add yourself as a co-author (no `Co-Authored-By` lines)
- **Commit messages must be a single line** - keep it concise and descriptive
## Project Overview
Mosis is a **virtual smartphone OS** for VR games and applications. It provides a phone-like device that users can interact with inside VR environments, with real smartphone functionality.
### Current Status
| Component | Status | Notes |
|-----------|--------|-------|
| MosisService | ✅ Working | RmlUi rendering, touch input, navigation |
| App Management | ✅ Working | Install/uninstall apps, sandbox integration |
| Lua Sandbox | ✅ Working | 149 security tests passing |
| Desktop Designer | ✅ Working | Hot-reload, hierarchy dump, recording |
| Designer Tests | ✅ 5/5 Passing | Navigation tests automated |
| MosisVR (Unity) | ✅ Building | OpenGL backend working, Vulkan in progress |
| MosisUnreal | ✅ Working | Vulkan texture import via UE5 RHI, phone actor with mesh |
### Project Components
| Component | Location | Purpose |
|-----------|----------|---------|
| Android Service | `src/main/` | Native service running RmlUi renderer |
| App Management | `src/main/cpp/apps/` | App install/uninstall/launch with sandbox |
| Lua Sandbox | `src/main/cpp/sandbox/` | Per-app Lua isolation (22 modules) |
| Desktop Designer | `designer/` | UI development with hot-reload |
| Designer Tests | `designer-test/` | Automated UI testing framework |
| Sandbox Tests | `sandbox-test/` | Lua sandbox security tests (149 tests) |
| UI Assets | `src/main/assets/` | Shared RML/RCSS/Lua assets |
## Detailed Documentation
All detailed documentation is in `docs/`:
| Document | Description |
|----------|-------------|
| [BUILD-COMMANDS.md](BUILD-COMMANDS.md) | Android, Desktop Designer, and test build commands |
| [ARCHITECTURE.md](ARCHITECTURE.md) | Native libraries, IPC flow, code structure |
| [DESKTOP-DESIGNER.md](DESKTOP-DESIGNER.md) | Hot-reload, recording, key files |
| [TESTING-FRAMEWORK.md](TESTING-FRAMEWORK.md) | Automated UI testing, writing tests |
| [UI-ASSETS.md](UI-ASSETS.md) | Asset structure, navigation system, element IDs |
| [MATERIAL-DESIGN.md](MATERIAL-DESIGN.md) | Icons, MDL components, usage guide |
| [ANDROID-TESTING.md](ANDROID-TESTING.md) | ADB commands, event injection, logs |
| [GAME-ENGINES.md](GAME-ENGINES.md) | Unreal & Unity integration, Vulkan import |
| [DEVELOPER-PORTAL.md](DEVELOPER-PORTAL.md) | Portal architecture, milestones |
| [APP-MANAGEMENT.md](APP-MANAGEMENT.md) | Package format, manifest, lifecycle |
| [LUA-SANDBOX.md](LUA-SANDBOX.md) | Security features, permissions, APIs |
## Quick Reference
### Environment Requirements
- `ANDROID_HOME` - Android SDK path
- `ANDROID_NDK_HOME` - Android NDK path (version 29.0.14206865)
- `VCPKG_ROOT` - vcpkg package manager root
### Common Build Commands
```bash
# Android APK
./gradlew assembleDebug
# Desktop Designer (from designer/)
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake
cmake --build build --config Debug
# Sandbox Tests (from sandbox-test/)
./run_tests.bat
```
### Code Style
- C++23 standard with modern features (std::span, std::format)
- PascalCase for classes/functions, camelCase for variables
- RAII principles with smart pointers
- Kotlin code follows Android conventions
### Dependencies
- vcpkg: RmlUi, GLFW, Freetype, Lua, libpng, nlohmann-json, minizip, sqlite3
- Android: arm64-v8a only
- Desktop: Windows x64 (MSVC)
## Documentation Guidelines
**IMPORTANT**: Always document progress and new commands to avoid rediscovery.
| Content Type | Location |
|--------------|----------|
| General concepts, architecture | `MosisService/docs/CLAUDE.md` or `docs/` |
| Unreal plugin docs | `MosisUnreal/Plugins/MosisSDK/README.md` |
| Unity package docs | `MosisVR/Packages/com.omixlab.mosis_sdk/README.md` |
**DO NOT** put documentation in the root `D:\Dev\Mosis\` directory - it is not versioned.

40
docs/DESKTOP-DESIGNER.md Normal file
View File

@@ -0,0 +1,40 @@
# Desktop Designer
The desktop designer (`designer/`) provides rapid UI development with:
- **Hot-reload**: Automatically reloads when RML/RCSS/Lua files change
- **UI Hierarchy Dumping**: Exports element tree to JSON for inspection
- **Screenshot Capture**: PNG export via F12 key
- **Logging**: Detailed output for debugging navigation and events
- **Action Recording**: Record mouse/keyboard interactions to JSON
- **Action Playback**: Replay recorded interactions with timing
## Key Files
| File | Purpose |
|------|---------|
| `designer/src/main.cpp` | Main entry point, GLFW window, event loop |
| `designer/src/desktop_kernel.cpp` | RmlUi context management, rendering |
| `designer/src/testing/ui_inspector.cpp` | UI hierarchy JSON export |
| `designer/src/testing/visual_capture.cpp` | PNG screenshot capture and comparison |
| `designer/src/testing/action_recorder.cpp` | Record user interactions to JSON |
| `designer/src/testing/action_player.cpp` | Playback recorded actions |
| `designer/src/backend/RmlUi_Backend_GLFW_GL3.cpp` | GLFW backend with input hooks |
## Command Line Options
```
--log <path> Write logs to file
--hierarchy <path> Dump UI hierarchy JSON each frame
--dump Single-shot dump mode (screenshot + hierarchy)
--record <path> Enable recording mode (F5 to start/stop)
--playback <path> Play back recorded actions from JSON
```
## Keyboard Controls
| Key | Function |
|-----|----------|
| F5 | Start/stop recording (when --record is enabled) |
| F6 | Pause/resume playback (when --playback is enabled) |
| F12 | Take screenshot |

56
docs/DEVELOPER-PORTAL.md Normal file
View File

@@ -0,0 +1,56 @@
# Developer Portal
The Developer Portal is a web application for app developers to publish and manage their Mosis apps. Planning documents are in `DEV_PORTAL_M01-M12.md` files.
## Architecture Decisions
| Component | Technology | Rationale |
|-----------|------------|-----------|
| Backend | Go 1.22+ | Simple, fast, single binary deployment |
| Router | Chi | Lightweight, idiomatic Go HTTP routing |
| Database | SQLite + Litestream | Zero-ops, continuous backup to NAS storage |
| Frontend | htmx + Go templates | Server-rendered, minimal JS, fast |
| Storage | Synology NAS filesystem | Self-hosted, local volume mounts |
| Auth | OAuth2 (GitHub/Google) + JWT | golang-jwt for tokens, Ed25519 for signing |
| CLI | Go + Cobra | Cross-platform single binary |
| Docs | Hugo + Docsy | Static site served from /docs/ |
## Deployment Target
Self-hosted on Synology NAS via Docker:
```
/volume1/mosis/
├── data/
│ ├── portal.db # Main SQLite database
│ └── telemetry.db # Separate telemetry database
├── packages/ # Uploaded app packages
│ └── {dev_id}/{app_id}/{version}/
├── backups/ # Litestream replicas
└── docs/ # Hugo static site output
```
## Milestone Documents
| File | Topic | Status |
|------|-------|--------|
| DEV_PORTAL_M01_OVERVIEW.md | Project overview | Decided |
| DEV_PORTAL_M02_WEB_STACK.md | Go + Chi + htmx | Decided |
| DEV_PORTAL_M03_DATABASE.md | SQLite + Litestream | Decided |
| DEV_PORTAL_M04_AUTH.md | OAuth2 + JWT + Ed25519 | Decided |
| DEV_PORTAL_M05_FRONTEND.md | htmx + Go templates | Decided |
| DEV_PORTAL_M06_API.md | REST API design | Decided |
| DEV_PORTAL_M07_STORAGE.md | NAS filesystem | Decided |
| DEV_PORTAL_M08_TELEMETRY.md | SQLite + background workers | Decided |
| DEV_PORTAL_M09_REVIEW.md | Go validation workers | Decided |
| DEV_PORTAL_M10_DEVICE.md | C++ AppManager | Decided |
| DEV_PORTAL_M11_CLI.md | Go + Cobra CLI | Decided |
| DEV_PORTAL_M12_DOCS.md | Hugo + Docsy | Decided |
## Key Design Principles
1. **Single container** - Portal runs as one Docker container on NAS
2. **No external services** - SQLite, local filesystem, Pagefind search
3. **Pure Go** - `modernc.org/sqlite` (no CGO required)
4. **Server-rendered** - htmx for interactivity, no SPA framework
5. **Background workers** - Go goroutines for aggregation/cleanup jobs

412
docs/DEV_PORTAL_M01_APP_PACKAGE.md Normal file
View File

@@ -0,0 +1,412 @@
# Milestone 1: App Package Format
**Status**: Decided
**Goal**: Define how apps are bundled, signed, and validated.
## Decision
**Signed ZIP (Option C)** with JAR/APK-style signing using Ed25519:
```
Format: ZIP archive with .mosis extension
Signing: Ed25519 (crypto/ed25519 stdlib)
Manifest: META-INF/MANIFEST.MF with SHA-256 hashes
Validation: Go package (mosis-portal/pkg/package)
```
### Rationale
1. **Standard tooling** - ZIP format works with all archive tools
2. **Proven approach** - JAR/APK signing is battle-tested
3. **Ed25519** - Fast, secure, small signatures (64 bytes)
4. **Go stdlib** - crypto/ed25519 and archive/zip in standard library
5. **Easy inspection** - Developers can unzip and view contents
### Package Structure
```
com.developer.appname-1.0.0.mosis (ZIP archive)
├── manifest.json # App metadata (JSON)
├── META-INF/
│ ├── MANIFEST.MF # SHA-256 hashes of all files
│ └── CERT.SIG # Ed25519 signature of MANIFEST.MF
├── icons/
│ ├── icon-32.png
│ ├── icon-64.png
│ └── icon-128.png
└── assets/
├── main.rml # Entry point
├── styles/
│ └── theme.rcss
└── scripts/
└── app.lua
```
---
## Overview
The app package format is the foundation of the entire ecosystem. Every tool (CLI, portal, device) needs to understand this format.
---
## Questions to Answer
1. What files comprise an app package?
2. How is the manifest structured?
3. How are apps signed for integrity?
4. How are updates handled (full vs delta)?
5. What metadata is required (name, version, permissions, icons)?
---
## Package Format Options
### Option A: ZIP Archive
```
myapp.zip
├── manifest.json
├── icon.png
└── assets/
├── main.rml
├── styles.rcss
└── app.lua
```
| Pros | Cons |
|------|------|
| Simple, standard tooling | No built-in signing |
| Easy to inspect | No metadata in filename |
| Wide compatibility | Need separate signature file |
### Option B: Custom Format (.mosis)
```
myapp.mosis (binary format)
┌─────────────────────────┐
│ Magic bytes (4) │
│ Version (2) │
│ Manifest length (4) │
│ Manifest JSON │
│ Signature (256) │
│ Compressed payload │
└─────────────────────────┘
```
| Pros | Cons |
|------|------|
| Signature built-in | Custom tooling needed |
| Efficient metadata access | Harder to inspect |
| Single file | More complex implementation |
### Option C: Signed ZIP (Recommended)
```
myapp.mosis/ (actually a ZIP)
├── manifest.json
├── META-INF/
│ ├── MANIFEST.MF # File hashes
│ └── SIGNATURE.SF # Signed manifest
├── icon.png
└── assets/
└── ...
```
| Pros | Cons |
|------|------|
| Standard ZIP tooling | Slightly more complex |
| JAR/APK-style signing | |
| Easy inspection | |
| Proven approach | |
---
## Proposed Structure
```
com.developer.appname-1.0.0.mosis
├── manifest.json # App metadata
├── META-INF/
│ ├── CERT.PEM # Developer certificate
│ ├── CERT.SIG # Signature of MANIFEST.MF
│ └── MANIFEST.MF # SHA-256 hashes of all files
├── icons/
│ ├── icon-32.png
│ ├── icon-64.png
│ └── icon-128.png
├── assets/
│ ├── main.rml # Entry point
│ ├── screens/
│ │ ├── home.rml
│ │ └── settings.rml
│ ├── styles/
│ │ └── theme.rcss
│ └── scripts/
│ ├── main.lua
│ └── utils.lua
└── locales/ # Optional i18n
├── en.json
└── es.json
```
---
## Manifest Schema
### Required Fields
```json
{
"$schema": "https://mosis.dev/schemas/manifest-v1.json",
"id": "com.developer.appname",
"name": "My App",
"version": "1.0.0",
"version_code": 1,
"entry": "assets/main.rml",
"min_mosis_version": "1.0.0"
}
```
### Full Schema
```json
{
"$schema": "https://mosis.dev/schemas/manifest-v1.json",
"id": "com.developer.appname",
"name": "My App",
"version": "1.0.0",
"version_code": 1,
"description": "A short description of the app",
"entry": "assets/main.rml",
"author": {
"name": "Developer Name",
"email": "dev@example.com",
"url": "https://developer.com"
},
"permissions": [
"storage",
"network",
"camera"
],
"icons": {
"32": "icons/icon-32.png",
"64": "icons/icon-64.png",
"128": "icons/icon-128.png"
},
"min_mosis_version": "1.0.0",
"target_mosis_version": "1.2.0",
"category": "utilities",
"tags": ["productivity", "tools"],
"orientation": "portrait",
"background_color": "#FFFFFF",
"locales": ["en", "es", "fr"],
"default_locale": "en"
}
```
### Field Definitions
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `id` | string | Yes | Unique package identifier (reverse domain) |
| `name` | string | Yes | Display name (max 30 chars) |
| `version` | string | Yes | Semantic version (X.Y.Z) |
| `version_code` | integer | Yes | Incremental build number |
| `entry` | string | Yes | Path to entry RML file |
| `min_mosis_version` | string | Yes | Minimum Mosis version required |
| `description` | string | No | Short description (max 80 chars) |
| `author` | object | No | Author information |
| `permissions` | array | No | Required permissions |
| `icons` | object | No | Icon paths by size |
| `category` | string | No | App store category |
| `tags` | array | No | Searchable tags |
| `orientation` | string | No | portrait, landscape, any |
| `background_color` | string | No | Hex color for loading |
| `locales` | array | No | Supported locales |
---
## Signing Mechanism
### Algorithm
- **Key type**: Ed25519 (fast, secure, small signatures)
- **Hash**: SHA-256 for file manifests
- **Format**: PEM for keys, base64 for signatures
### MANIFEST.MF Format
```
Manifest-Version: 1.0
Created-By: mosis-cli 1.0.0
Name: manifest.json
SHA-256-Digest: base64encodedHash==
Name: assets/main.rml
SHA-256-Digest: base64encodedHash==
Name: assets/scripts/main.lua
SHA-256-Digest: base64encodedHash==
```
### Signing Flow
```
1. Generate MANIFEST.MF with SHA-256 of each file
2. Sign MANIFEST.MF with developer's Ed25519 private key
3. Store signature in META-INF/CERT.SIG
4. Include developer's public key in META-INF/CERT.PEM
```
### Verification Flow
```
1. Extract META-INF/MANIFEST.MF
2. Verify signature using CERT.PEM
3. Verify CERT.PEM is registered with developer account
4. Verify each file hash matches MANIFEST.MF
```
---
## Size Limits
| Limit | Value | Rationale |
|-------|-------|-----------|
| Max package size | 50 MB | Reasonable for mobile |
| Max individual file | 10 MB | Prevent abuse |
| Max files count | 1000 | Prevent zip bombs |
| Max path length | 256 chars | Filesystem compat |
| Max manifest size | 64 KB | Prevent abuse |
---
## Validation Rules
### Manifest Validation
- [ ] Valid JSON
- [ ] All required fields present
- [ ] `id` matches reverse domain pattern: `^[a-z][a-z0-9]*(\.[a-z][a-z0-9]*)+$`
- [ ] `version` matches semver pattern
- [ ] `version_code` is positive integer
- [ ] `entry` file exists in package
- [ ] All `permissions` are valid permission names
- [ ] All `icons` paths exist and are valid images
### Package Validation
- [ ] Valid ZIP format
- [ ] manifest.json at root
- [ ] No path traversal (../)
- [ ] No absolute paths
- [ ] No symlinks
- [ ] Under size limits
- [ ] No duplicate files
- [ ] Valid file extensions only
### Signature Validation
- [ ] MANIFEST.MF present
- [ ] CERT.SIG present
- [ ] CERT.PEM present
- [ ] Signature valid for MANIFEST.MF
- [ ] All file hashes match
- [ ] Certificate registered with developer account (for store)
---
## File Extension Rules
### Allowed Extensions
| Category | Extensions |
|----------|------------|
| UI | .rml |
| Styles | .rcss |
| Scripts | .lua |
| Images | .png, .jpg, .jpeg, .tga, .webp |
| Fonts | .ttf, .otf |
| Data | .json |
| Localization | .json (in locales/) |
| Audio | .ogg, .wav, .mp3 |
### Forbidden
- Executables: .exe, .dll, .so, .dylib
- Scripts: .sh, .bat, .ps1, .py, .js
- Archives: .zip, .tar, .gz (nested)
---
## Update Handling
### Full Update
- Download complete new package
- Verify signature
- Atomic replacement of app directory
- Preserve user data in separate location
### Delta Updates (Future)
- bsdiff-style patches
- Reduces bandwidth for minor updates
- More complex implementation
- Consider for v2
---
## Deliverables
- [x] Package format decided (Signed ZIP with .mosis extension)
- [x] Signing algorithm decided (Ed25519)
- [ ] JSON Schema for manifest validation
- [ ] Go package: `pkg/package/manifest.go` (parsing/validation)
- [ ] Go package: `pkg/package/validator.go` (package validation)
- [ ] Go package: `pkg/package/signer.go` (Ed25519 signing/verification)
- [ ] Integration with mosis-cli `build` and `sign` commands
---
## Test Cases
| Test | Description |
|------|-------------|
| ValidPackage | Accept well-formed package |
| InvalidManifest | Reject malformed JSON |
| MissingRequired | Reject missing required fields |
| BadSignature | Reject invalid signature |
| TamperedFile | Reject if file hash mismatch |
| PathTraversal | Reject ../ in paths |
| OversizePackage | Reject over size limit |
| BadExtension | Reject forbidden file types |
| DuplicateFiles | Reject duplicate entries |
---
## Open Questions
1. ~~Should we support multiple entry points (e.g., widget vs full app)?~~ → Single entry point for v1
2. ~~Should icons be required or have defaults?~~ → Required (32, 64, 128 sizes)
3. ~~Delta updates in v1 or defer to v2?~~ → Defer to v2 (full updates only)
4. ~~Support for app bundles (multiple apps in one package)?~~ → No, one app per package
---
## References
- [Android APK format](https://developer.android.com/guide/components/fundamentals)
- [JAR signing](https://docs.oracle.com/javase/tutorial/deployment/jar/signing.html)
- [Ed25519 specification](https://ed25519.cr.yp.to/)

367
docs/DEV_PORTAL_M02_WEB_STACK.md Normal file
View File

@@ -0,0 +1,367 @@
# Milestone 2: Web Stack Selection
**Status**: Decided
**Goal**: Choose backend technologies for the developer portal and app store API.
## Decision
**Go** with the following stack:
```
Language: Go 1.22+
Framework: Chi (lightweight, idiomatic)
Database: SQLite via modernc.org/sqlite (pure Go, no CGO)
Migrations: golang-migrate
Validation: go-playground/validator
Auth: Custom JWT + OAuth2
Deployment: Single Docker container on Synology NAS
```
### Rationale
1. **NAS-friendly** - Tiny Docker image (~15MB), low RAM (~30-50MB)
2. **Cross-compilation** - Easy build for ARM64 or AMD64 Synology models
3. **Pure Go SQLite** - `modernc.org/sqlite` requires no CGO, cross-compiles easily
4. **Single container** - Go binary + SQLite + Litestream in one image
5. **Standard library** - HTTP, JSON, crypto all built-in
### Target Deployment
```
Synology NAS (Docker)
├── mosis-portal container (~15MB image)
│ ├── Go binary
│ ├── SQLite database (WAL mode)
│ └── Litestream backup
└── Volumes
├── /volume1/mosis/data/portal.db
├── /volume1/mosis/backups/
└── /volume1/mosis/packages/
```
---
## Overview
The web stack powers the developer portal, app store API, and telemetry ingestion. This decision affects development speed, hosting costs, and long-term maintenance.
---
## Requirements
### Functional
- REST API for developer portal
- File upload handling (app packages)
- Authentication (OAuth2, API keys)
- Database integration
- Background jobs (review queue, notifications)
- WebSocket support (optional, for real-time updates)
### Non-Functional
- Handle 1000+ developers initially
- Scale to 10,000+ apps
- 99.9% uptime target
- < 200ms API response time (p95)
- Secure by default
---
## Options Analysis
### Option A: Node.js + TypeScript
#### Stack
```
Runtime: Node.js 20 LTS
Language: TypeScript
Framework: Fastify or Hono
ORM: Prisma or Drizzle
Validation: Zod
Auth: Lucia or custom JWT
```
#### Pros
| Advantage | Details |
|-----------|---------|
| Fast development | Large ecosystem, familiar syntax |
| Type safety | TypeScript catches errors early |
| Easy hiring | Many JS/TS developers |
| Ecosystem | npm has libraries for everything |
| Hosting | Vercel, Railway, Render, any VPS |
#### Cons
| Disadvantage | Details |
|--------------|---------|
| Single-threaded | Need clustering for CPU tasks |
| Memory usage | Higher than compiled languages |
| Callback complexity | Async can get messy |
#### Example Structure
```
src/
├── index.ts
├── routes/
│ ├── auth.ts
│ ├── apps.ts
│ └── telemetry.ts
├── services/
│ ├── auth.service.ts
│ └── storage.service.ts
├── db/
│ ├── schema.ts
│ └── client.ts
└── utils/
```
#### Hosting Cost Estimate
| Service | Cost/month |
|---------|------------|
| Railway (starter) | $5-20 |
| Vercel Pro | $20 |
| VPS (4GB) | $20-40 |
---
### Option B: Go
#### Stack
```
Language: Go 1.22+
Framework: Chi, Echo, or Gin
ORM: sqlc or GORM
Validation: go-playground/validator
Auth: Custom JWT
```
#### Pros
| Advantage | Details |
|-----------|---------|
| Performance | Fast, low memory |
| Single binary | Easy deployment |
| Concurrency | Goroutines handle load well |
| Standard library | HTTP, JSON, crypto built-in |
#### Cons
| Disadvantage | Details |
|--------------|---------|
| Verbose | Error handling boilerplate |
| Smaller ecosystem | Fewer ready-made solutions |
| Learning curve | Different paradigm |
#### Example Structure
```
cmd/
├── server/
│ └── main.go
internal/
├── api/
│ ├── handlers/
│ ├── middleware/
│ └── routes.go
├── domain/
│ ├── app.go
│ └── developer.go
├── repository/
│ └── postgres/
└── service/
```
#### Hosting Cost Estimate
| Service | Cost/month |
|---------|------------|
| Fly.io (shared) | $5-15 |
| Cloud Run | Pay per use |
| VPS (2GB) | $10-20 |
---
### Option C: Rust + Axum
#### Stack
```
Language: Rust
Framework: Axum
ORM: sqlx or SeaORM
Validation: validator crate
Auth: Custom JWT
```
#### Pros
| Advantage | Details |
|-----------|---------|
| Maximum performance | Fastest option |
| Memory safety | No runtime errors |
| Single binary | Easy deployment |
| Long-term reliability | Compiler catches bugs |
#### Cons
| Disadvantage | Details |
|--------------|---------|
| Steep learning curve | Borrow checker takes time |
| Slower development | More code to write |
| Smaller ecosystem | Fewer web libraries |
| Compile times | Can be slow |
#### Example Structure
```
src/
├── main.rs
├── api/
│ ├── mod.rs
│ ├── handlers/
│ └── middleware/
├── domain/
├── repository/
└── service/
```
#### Hosting Cost Estimate
| Service | Cost/month |
|---------|------------|
| Fly.io | $5-10 |
| Shuttle | Free tier available |
| VPS (1GB) | $5-10 |
---
### Option D: .NET (ASP.NET Core)
#### Stack
```
Language: C#
Framework: ASP.NET Core 8
ORM: Entity Framework Core
Validation: FluentValidation
Auth: ASP.NET Identity
```
#### Pros
| Advantage | Details |
|-----------|---------|
| Enterprise-ready | Battle-tested at scale |
| Great tooling | Visual Studio, debugging |
| Performance | Very fast (Kestrel) |
| Full-featured | Auth, validation, DI built-in |
#### Cons
| Disadvantage | Details |
|--------------|---------|
| Heavier runtime | Larger container images |
| Microsoft ecosystem | May feel locked in |
| Verbose | More boilerplate |
#### Hosting Cost Estimate
| Service | Cost/month |
|---------|------------|
| Azure App Service | $15-50 |
| VPS (4GB) | $20-40 |
---
## Evaluation Matrix
| Criteria | Weight | Node.js | Go | Rust | .NET |
|----------|--------|---------|-----|------|------|
| Dev speed | 25% | 9 | 7 | 5 | 7 |
| Performance | 20% | 6 | 9 | 10 | 8 |
| Hosting cost | 15% | 7 | 9 | 9 | 6 |
| Ecosystem | 15% | 9 | 7 | 6 | 8 |
| Team familiarity | 15% | ? | ? | ? | ? |
| Long-term maintenance | 10% | 7 | 8 | 9 | 8 |
| **Weighted Score** | | **7.4** | **7.8** | **7.2** | **7.3** |
*Team familiarity needs to be filled in based on actual experience*
---
## Framework Comparison
### Node.js Frameworks
| Framework | Req/sec | Features | Learning Curve |
|-----------|---------|----------|----------------|
| Express | 15k | Minimal, flexible | Easy |
| Fastify | 45k | Fast, schema validation | Medium |
| Hono | 50k | Ultra-light, edge-ready | Easy |
| NestJS | 20k | Full-featured, Angular-like | Steep |
### Go Frameworks
| Framework | Req/sec | Features | Learning Curve |
|-----------|---------|----------|----------------|
| net/http | 60k | Standard library | Medium |
| Chi | 55k | Lightweight router | Easy |
| Gin | 50k | Popular, middleware | Easy |
| Echo | 55k | Similar to Gin | Easy |
---
## Prototype Plan
### Phase 1: Build minimal API in top 2 choices
Endpoints to implement:
```
POST /auth/register
POST /auth/login
GET /apps
POST /apps
POST /apps/:id/versions
```
### Phase 2: Benchmark
- Requests per second
- Memory usage under load
- Development time tracking
- Code complexity comparison
### Phase 3: Decision
Based on:
- Benchmark results
- Developer experience during prototype
- Hosting cost analysis
---
## Recommendation
**Primary: Go with Chi/Echo**
- Best balance of performance and simplicity
- Low hosting costs
- Single binary deployment
- Good enough ecosystem for our needs
**Fallback: Node.js with Hono/Fastify**
- If Go feels too slow to develop in
- Larger ecosystem for edge cases
- More developers familiar with it
---
## Deliverables
- [x] Final selection with rationale
- [ ] Project structure template
- [ ] Development environment setup guide
- [ ] Base Go project scaffolding
---
## Open Questions
1. Do we need GraphQL or is REST sufficient?
2. Do we need real-time features (WebSocket)?
3. What's the team's current language experience?
4. Any preference for specific cloud providers?
---
## References
- [TechEmpower Benchmarks](https://www.techempower.com/benchmarks/)
- [Go vs Node.js comparison](https://blog.logrocket.com/node-js-vs-golang/)
- [Fastify benchmarks](https://fastify.dev/benchmarks/)

527
docs/DEV_PORTAL_M03_DATABASE.md Normal file
View File

@@ -0,0 +1,527 @@
# Milestone 3: Database Selection
**Status**: Decided
**Goal**: Choose database for developer accounts, app metadata, and analytics.
## Decision
**SQLite + Litestream** for self-hosted deployment on Synology NAS.
```
Database: SQLite 3.x (WAL mode)
Driver: modernc.org/sqlite (pure Go, no CGO)
Backup: Litestream continuous replication
Storage: Synology volume (/volume1/mosis/)
```
### Rationale
1. **Single container** - No separate database service needed
2. **Minimal resources** - ~50MB RAM, perfect for NAS
3. **Zero ops** - No connection pooling, no tuning
4. **Continuous backup** - Litestream replicates to local storage
5. **Point-in-time recovery** - Restore to any moment
6. **Sufficient scale** - Handles 1000s of developers easily
### Architecture
```
┌─────────────────────────────────────────┐
│ Synology NAS │
│ ┌─────────────────────────────────┐ │
│ │ mosis-portal container │ │
│ │ ├── Go binary │ │
│ │ ├── SQLite (portal.db) │ │
│ │ └── Litestream │ │
│ └──────────────┬──────────────────┘ │
│ │ │
│ ┌──────────────▼──────────────────┐ │
│ │ /volume1/mosis/ │ │
│ │ ├── data/portal.db │ │
│ │ ├── data/portal.db-wal │ │
│ │ ├── backups/ (litestream) │ │
│ │ └── packages/ (app uploads) │ │
│ └─────────────────────────────────┘ │
└─────────────────────────────────────────┘
```
### Litestream Configuration
```yaml
dbs:
- path: /data/portal.db
replicas:
- type: file
path: /backups/portal
retention: 720h # 30 days
```
---
## Overview
The database stores all persistent data: developer accounts, app metadata, versions, telemetry events, and audit logs.
---
## Requirements
### Data Characteristics
| Data Type | Volume | Access Pattern | Consistency |
|-----------|--------|----------------|-------------|
| Developers | 10K rows | Read-heavy, low write | Strong |
| Apps | 100K rows | Read-heavy | Strong |
| Versions | 500K rows | Read-heavy | Strong |
| API Keys | 50K rows | Read-heavy | Strong |
| Telemetry | 100M+ rows | Write-heavy, append | Eventual OK |
| Audit Logs | 10M+ rows | Write-heavy, append | Eventual OK |
### Query Patterns
- Get developer by email
- List apps by developer
- Get app with latest version
- Search apps by name/tags
- Aggregate telemetry by app/day
- Time-range queries on events
---
## Options Analysis
### Option A: PostgreSQL
#### Characteristics
```
Type: Relational (SQL)
ACID: Full
JSON: Native JSONB support
Full-text: Built-in tsvector
Scaling: Vertical + read replicas
```
#### Pros
| Advantage | Details |
|-----------|---------|
| Battle-tested | Decades of reliability |
| ACID compliance | Strong consistency |
| JSON support | JSONB for flexible data |
| Full-text search | No separate search engine needed |
| Extensions | PostGIS, pg_trgm, etc. |
| Tooling | pgAdmin, great ORMs |
#### Cons
| Disadvantage | Details |
|--------------|---------|
| Ops overhead | Need connection pooling |
| Scaling writes | Vertical scaling limits |
| Time-series | Not optimized for telemetry |
#### Hosting Options
| Provider | Free Tier | Paid |
|----------|-----------|------|
| Supabase | 500MB | $25/mo |
| Neon | 512MB | $19/mo |
| Railway | 1GB | $5/mo |
| AWS RDS | - | $15/mo+ |
| Self-hosted | - | VPS cost |
---
### Option B: SQLite + Litestream
#### Characteristics
```
Type: Embedded relational
ACID: Full
Scaling: Single writer
Backup: Litestream to S3
```
#### Pros
| Advantage | Details |
|-----------|---------|
| Zero ops | No separate DB server |
| Fast reads | In-process, no network |
| Simple backup | Litestream handles replication |
| Low cost | Just storage costs |
| Portable | Easy local development |
#### Cons
| Disadvantage | Details |
|--------------|---------|
| Single writer | Limits write concurrency |
| No horizontal scale | One server only |
| Limited features | No full-text (without FTS5) |
#### Cost Estimate
| Component | Cost/month |
|-----------|------------|
| S3 storage (10GB) | $0.25 |
| Compute | Included in app server |
---
### Option C: PostgreSQL + TimescaleDB
#### Characteristics
```
Type: Time-series extension
Base: PostgreSQL
Scaling: Automatic partitioning
Compression: Native
```
#### Pros
| Advantage | Details |
|-----------|---------|
| Best of both | Relational + time-series |
| Auto-partition | Handles telemetry scale |
| Compression | 90%+ compression ratio |
| Continuous aggregates | Pre-computed rollups |
#### Cons
| Disadvantage | Details |
|--------------|---------|
| Complexity | More to manage |
| Cost | Higher than plain Postgres |
| Learning curve | New concepts |
---
### Option D: Hybrid Approach
```
PostgreSQL → Developers, Apps, Versions, API Keys
ClickHouse/QuestDB → Telemetry, Analytics
Redis → Caching, Sessions
```
#### Pros
| Advantage | Details |
|-----------|---------|
| Right tool for job | Optimized for each use case |
| Scale independently | Telemetry won't affect main DB |
| Performance | Best possible for each workload |
#### Cons
| Disadvantage | Details |
|--------------|---------|
| Complexity | Multiple systems to manage |
| Cost | More infrastructure |
| Consistency | Cross-DB transactions hard |
---
## Schema Design (SQLite)
### Core Tables
```sql
-- Developers
CREATE TABLE developers (
id TEXT PRIMARY KEY, -- UUID as text
email TEXT UNIQUE NOT NULL,
name TEXT NOT NULL,
password_hash TEXT,
oauth_provider TEXT,
oauth_id TEXT,
verified INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
-- API Keys
CREATE TABLE api_keys (
id TEXT PRIMARY KEY,
developer_id TEXT NOT NULL REFERENCES developers(id) ON DELETE CASCADE,
name TEXT NOT NULL,
key_hash TEXT NOT NULL,
key_prefix TEXT NOT NULL, -- For display: "mk_abc..."
permissions TEXT DEFAULT '[]', -- JSON array
last_used_at TEXT,
expires_at TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
-- Apps
CREATE TABLE apps (
id TEXT PRIMARY KEY,
developer_id TEXT NOT NULL REFERENCES developers(id) ON DELETE CASCADE,
package_id TEXT UNIQUE NOT NULL, -- com.dev.app
name TEXT NOT NULL,
description TEXT,
category TEXT,
tags TEXT DEFAULT '[]', -- JSON array
status TEXT DEFAULT 'draft', -- draft, published, suspended
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
-- App Versions
CREATE TABLE app_versions (
id TEXT PRIMARY KEY,
app_id TEXT NOT NULL REFERENCES apps(id) ON DELETE CASCADE,
version_code INTEGER NOT NULL,
version_name TEXT NOT NULL,
package_url TEXT NOT NULL,
package_size INTEGER NOT NULL,
signature TEXT NOT NULL,
permissions TEXT DEFAULT '[]', -- JSON array
min_mosis_version TEXT,
release_notes TEXT,
status TEXT DEFAULT 'draft', -- draft, review, approved, published, rejected
review_notes TEXT,
published_at TEXT,
created_at TEXT DEFAULT (datetime('now')),
UNIQUE(app_id, version_code)
);
-- Developer Signing Keys
CREATE TABLE signing_keys (
id TEXT PRIMARY KEY,
developer_id TEXT NOT NULL REFERENCES developers(id) ON DELETE CASCADE,
name TEXT NOT NULL,
public_key TEXT NOT NULL,
fingerprint TEXT NOT NULL,
is_active INTEGER DEFAULT 1,
created_at TEXT DEFAULT (datetime('now'))
);
```
### Telemetry Tables
```sql
-- Telemetry Events (append-only, partition by month via separate tables)
CREATE TABLE telemetry_events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
app_id TEXT NOT NULL,
device_id TEXT NOT NULL, -- Hashed for privacy
event_type TEXT NOT NULL,
event_data TEXT, -- JSON string
mosis_version TEXT,
timestamp TEXT NOT NULL -- ISO8601 format
);
-- Crash Reports
CREATE TABLE crash_reports (
id TEXT PRIMARY KEY,
app_id TEXT NOT NULL,
app_version TEXT NOT NULL,
device_id TEXT NOT NULL,
crash_type TEXT NOT NULL,
message TEXT,
stack_trace TEXT,
context TEXT, -- JSON string
mosis_version TEXT,
timestamp TEXT NOT NULL,
created_at TEXT DEFAULT (datetime('now'))
);
-- Daily aggregates (computed by background job)
CREATE TABLE telemetry_daily (
app_id TEXT NOT NULL,
date TEXT NOT NULL, -- YYYY-MM-DD
event_type TEXT NOT NULL,
count INTEGER NOT NULL,
unique_devices INTEGER NOT NULL,
PRIMARY KEY (app_id, date, event_type)
);
-- Audit Logs
CREATE TABLE audit_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
developer_id TEXT,
action TEXT NOT NULL,
resource_type TEXT,
resource_id TEXT,
details TEXT, -- JSON string
ip_address TEXT,
user_agent TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
```
**Note**: For high-volume telemetry, consider:
- Separate SQLite database file for telemetry (isolates write load)
- Monthly table rotation with application-level partitioning
- Aggressive data retention (delete events older than 90 days)
### Indexes
```sql
-- Developers
CREATE INDEX idx_developers_email ON developers(email);
CREATE INDEX idx_developers_oauth ON developers(oauth_provider, oauth_id);
-- API Keys
CREATE INDEX idx_api_keys_developer ON api_keys(developer_id);
CREATE INDEX idx_api_keys_prefix ON api_keys(key_prefix);
-- Apps
CREATE INDEX idx_apps_developer ON apps(developer_id);
CREATE INDEX idx_apps_package ON apps(package_id);
CREATE INDEX idx_apps_status ON apps(status);
CREATE INDEX idx_apps_name ON apps(name); -- For LIKE searches
-- Versions
CREATE INDEX idx_versions_app ON app_versions(app_id);
CREATE INDEX idx_versions_status ON app_versions(status);
-- Signing Keys
CREATE INDEX idx_signing_keys_developer ON signing_keys(developer_id);
CREATE INDEX idx_signing_keys_fingerprint ON signing_keys(fingerprint);
-- Telemetry
CREATE INDEX idx_telemetry_app ON telemetry_events(app_id, timestamp);
CREATE INDEX idx_telemetry_type ON telemetry_events(event_type, timestamp);
-- Crashes
CREATE INDEX idx_crashes_app ON crash_reports(app_id, timestamp);
CREATE INDEX idx_crashes_type ON crash_reports(crash_type);
-- Audit Logs
CREATE INDEX idx_audit_developer ON audit_logs(developer_id);
CREATE INDEX idx_audit_created ON audit_logs(created_at);
```
**Full-text Search**: For app search, use SQLite FTS5:
```sql
-- Create FTS5 virtual table for app search
CREATE VIRTUAL TABLE apps_fts USING fts5(
name,
description,
tags,
content='apps',
content_rowid='rowid'
);
-- Triggers to keep FTS in sync
CREATE TRIGGER apps_ai AFTER INSERT ON apps BEGIN
INSERT INTO apps_fts(rowid, name, description, tags)
VALUES (NEW.rowid, NEW.name, NEW.description, NEW.tags);
END;
CREATE TRIGGER apps_ad AFTER DELETE ON apps BEGIN
INSERT INTO apps_fts(apps_fts, rowid, name, description, tags)
VALUES ('delete', OLD.rowid, OLD.name, OLD.description, OLD.tags);
END;
CREATE TRIGGER apps_au AFTER UPDATE ON apps BEGIN
INSERT INTO apps_fts(apps_fts, rowid, name, description, tags)
VALUES ('delete', OLD.rowid, OLD.name, OLD.description, OLD.tags);
INSERT INTO apps_fts(rowid, name, description, tags)
VALUES (NEW.rowid, NEW.name, NEW.description, NEW.tags);
END;
```
---
## Migration Strategy
### Approach: Incremental Migrations
```
migrations/
├── 001_create_developers.sql
├── 002_create_apps.sql
├── 003_create_versions.sql
├── 004_create_telemetry.sql
└── ...
```
### Tools
- **Go**: golang-migrate, goose
- **Node.js**: Prisma Migrate, Drizzle Kit
- **Rust**: sqlx migrate, refinery
### Rollback Strategy
- Every migration has up/down
- Test rollbacks in staging
- Keep migrations small and focused
---
## Backup Strategy
### PostgreSQL
```bash
# Daily full backup
pg_dump -Fc $DATABASE_URL > backup_$(date +%Y%m%d).dump
# Continuous WAL archiving to S3
archive_command = 'aws s3 cp %p s3://backups/wal/%f'
```
### SQLite + Litestream
```yaml
# litestream.yml
dbs:
- path: /data/mosis.db
replicas:
- url: s3://backups/mosis
retention: 720h # 30 days
```
### Recovery Time Objectives
| Scenario | RTO | RPO |
|----------|-----|-----|
| Hardware failure | 1 hour | 5 minutes |
| Data corruption | 4 hours | 1 hour |
| Disaster recovery | 24 hours | 24 hours |
---
## Recommendation
### For MVP/Early Stage
**SQLite + Litestream**
- Simplest to operate
- Lowest cost
- Good enough for initial scale
- Easy migration to PostgreSQL later
### For Production Scale
**PostgreSQL + TimescaleDB**
- Handles all data types well
- Time-series for telemetry
- Proven at scale
- Good tooling ecosystem
### Hybrid (If needed later)
```
PostgreSQL → Core data (developers, apps)
TimescaleDB → Telemetry (same cluster, extension)
Redis → Caching, rate limiting
```
---
## Deliverables
- [x] Final database selection (SQLite + Litestream)
- [x] Complete schema design (core + telemetry + FTS5)
- [ ] Migration scripts (golang-migrate)
- [x] Backup/restore procedures (Litestream to local storage)
- [x] ~~Connection pooling setup~~ (not needed for SQLite)
- [ ] Monitoring queries
---
## Open Questions
1. ~~Expected telemetry volume per day?~~ → Start simple, optimize if needed
2. ~~How long to retain raw telemetry?~~ → 90 days raw, daily aggregates indefinitely
3. ~~Need for real-time analytics vs batch?~~ → Batch is sufficient for MVP
4. ~~Multi-region requirements?~~ → Single NAS deployment for now
---
## References
- [PostgreSQL JSONB performance](https://www.postgresql.org/docs/current/datatype-json.html)
- [TimescaleDB vs InfluxDB](https://www.timescale.com/blog/timescaledb-vs-influxdb/)
- [Litestream documentation](https://litestream.io/)
- [SQLite at scale](https://www.sqlite.org/whentouse.html)

493
docs/DEV_PORTAL_M04_AUTH.md Normal file
View File

@@ -0,0 +1,493 @@
# Milestone 4: Authentication System
**Status**: Decided
**Goal**: Secure developer authentication and app signing infrastructure.
## Decision
**Custom JWT + OAuth2** with Go standard library crypto:
```
OAuth2: golang.org/x/oauth2 (GitHub, Google)
JWT: github.com/golang-jwt/jwt/v5
Signing: crypto/ed25519 (stdlib)
Password Hash: golang.org/x/crypto/argon2
API Key Hash: golang.org/x/crypto/bcrypt
```
### Rationale
1. **Go stdlib crypto** - Ed25519 built into Go, no external deps
2. **Simple JWT** - golang-jwt is battle-tested, minimal
3. **Stateless tokens** - No token store needed (SQLite handles refresh token revocation)
4. **OAuth-first** - GitHub OAuth for most developers, minimal password handling
---
## Overview
Authentication covers two areas:
1. **Developer authentication** - Login to portal, API access
2. **App signing** - Package integrity and developer verification
---
## Developer Authentication
### Methods Required
| Method | Use Case | Priority |
|--------|----------|----------|
| OAuth2 (GitHub) | Primary login | P0 |
| OAuth2 (Google) | Alternative login | P1 |
| Email + Password | Fallback | P2 |
| API Keys | CLI tools, CI/CD | P0 |
### OAuth2 Flow
```
┌─────────┐ ┌─────────┐ ┌──────────┐ ┌─────────┐
│ Browser │────►│ Portal │────►│ Provider │────►│ Callback│
└─────────┘ └─────────┘ │(GitHub) │ └────┬────┘
└──────────┘ │
▼
┌─────────────┐
│ Create/Link │
│ Account │
└─────────────┘
```
### OAuth2 Implementation
#### GitHub OAuth
```
Authorization URL: https://github.com/login/oauth/authorize
Token URL: https://github.com/login/oauth/access_token
User Info: https://api.github.com/user
Scopes: read:user, user:email
```
#### Google OAuth
```
Authorization URL: https://accounts.google.com/o/oauth2/v2/auth
Token URL: https://oauth2.googleapis.com/token
User Info: https://www.googleapis.com/oauth2/v2/userinfo
Scopes: openid, email, profile
```
### Session Management
#### JWT Tokens
```json
{
"sub": "dev_uuid",
"email": "dev@example.com",
"iat": 1704067200,
"exp": 1704153600,
"type": "access"
}
```
| Token Type | Lifetime | Storage |
|------------|----------|---------|
| Access Token | 1 hour | Memory/Cookie |
| Refresh Token | 30 days | HttpOnly Cookie |
#### Token Refresh Flow
```
1. Access token expires
2. Client sends refresh token
3. Server validates refresh token
4. Issue new access + refresh tokens
5. Invalidate old refresh token
```
### API Key Authentication
#### Key Format
```
mk_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
│ │ └── 32 random bytes (base62)
│ └── Environment (live/test)
└── Prefix (mosis key)
```
#### Key Storage
```sql
-- Only store hash, never the key itself
INSERT INTO api_keys (
developer_id,
name,
key_hash, -- bcrypt or argon2
key_prefix, -- "mk_live_abc" for display
permissions
) VALUES (...);
```
#### Key Permissions
```json
{
"permissions": [
"apps:read",
"apps:write",
"versions:upload",
"telemetry:read"
]
}
```
### Rate Limiting
| Endpoint Category | Limit | Window |
|-------------------|-------|--------|
| Auth endpoints | 10 | 1 minute |
| API (authenticated) | 1000 | 1 hour |
| API (per key) | 100 | 1 minute |
| Upload | 10 | 1 hour |
---
## App Signing
### Key Generation
#### Algorithm: Ed25519
```
Private key: 32 bytes (256 bits)
Public key: 32 bytes (256 bits)
Signature: 64 bytes (512 bits)
```
#### Why Ed25519?
- Fast signing and verification
- Small key and signature sizes
- No configuration choices (secure by default)
- Widely supported
### Key Generation Flow
```bash
# Developer generates keypair locally
mosis keys generate
# Output:
# Private key saved to: ~/.mosis/signing_key.pem
# Public key saved to: ~/.mosis/signing_key.pub
#
# Fingerprint: SHA256:xxxxx
#
# IMPORTANT: Keep your private key secure!
# Upload your public key to the developer portal.
```
### Key Format
#### Private Key (PEM)
```
-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEIGxxxxx...
-----END PRIVATE KEY-----
```
#### Public Key (PEM)
```
-----BEGIN PUBLIC KEY-----
MCowBQYDK2VwAyEAxxxxx...
-----END PUBLIC KEY-----
```
### Signing Flow
```
1. Build package (ZIP all files)
2. Generate MANIFEST.MF with SHA-256 hashes
3. Sign MANIFEST.MF with private key
4. Store signature in META-INF/CERT.SIG
5. Include public key in META-INF/CERT.PEM
```
#### MANIFEST.MF Example
```
Manifest-Version: 1.0
Created-By: mosis-cli 1.0.0
Package-Id: com.developer.myapp
Version-Code: 1
Name: manifest.json
SHA-256-Digest: K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=
Name: assets/main.rml
SHA-256-Digest: uU0nuZNNPgilLlLX2n2r+sSE7+N6U4DukIj3rOLvzek=
```
### Verification Flow
```
1. Extract MANIFEST.MF from package
2. Extract CERT.SIG (signature)
3. Extract CERT.PEM (public key)
4. Verify signature of MANIFEST.MF using public key
5. Verify CERT.PEM matches registered developer key
6. Verify each file hash matches MANIFEST.MF entry
```
### Key Registration
```
Developer Portal:
├── Go to Settings > Signing Keys
├── Click "Add Key"
├── Paste public key (PEM format)
├── Verify fingerprint matches local
└── Key is now registered
Server stores:
├── public_key (PEM text)
├── fingerprint (SHA256 of public key)
├── created_at
└── is_active
```
### Key Rotation
```
1. Generate new keypair
2. Register new public key in portal
3. Sign new versions with new key
4. (Optional) Revoke old key after transition period
```
### Trust Model
```
┌─────────────────────────────────────────────────┐
│ Trust Chain │
├─────────────────────────────────────────────────┤
│ │
│ Developer │
│ │ │
│ ▼ │
│ Private Key ──signs──► Package │
│ │ │
│ ▼ │
│ Public Key ──registered──► Portal │
│ │ │
│ ▼ │
│ Portal ──verifies──► Signature │
│ │ │
│ ▼ │
│ Device ──trusts──► Portal-verified packages │
│ │
└─────────────────────────────────────────────────┘
```
---
## Security Considerations
### Password Storage (if used)
```
Algorithm: Argon2id
Memory: 64 MB
Iterations: 3
Parallelism: 4
Salt: 16 bytes random
```
### Token Security
- Access tokens: Short-lived, in-memory only
- Refresh tokens: HttpOnly, Secure, SameSite=Strict
- API keys: Hashed with bcrypt, shown once on creation
### Key Security
- Private keys never leave developer's machine
- Public keys verified via fingerprint
- Key compromise: Revoke immediately, re-sign apps
### Audit Logging
```sql
INSERT INTO auth_audit_log (
developer_id,
action, -- login, logout, key_create, key_revoke
ip_address,
user_agent,
success,
failure_reason,
timestamp
) VALUES (...);
```
---
## API Endpoints
### Authentication
```
POST /auth/oauth/github # Start GitHub OAuth
GET /auth/oauth/github/callback # GitHub callback
POST /auth/oauth/google # Start Google OAuth
GET /auth/oauth/google/callback # Google callback
POST /auth/refresh # Refresh tokens
POST /auth/logout # Invalidate tokens
GET /auth/me # Get current user
```
### API Keys
```
GET /api-keys # List keys
POST /api-keys # Create key
DELETE /api-keys/:id # Revoke key
```
### Signing Keys
```
GET /signing-keys # List keys
POST /signing-keys # Register key
DELETE /signing-keys/:id # Revoke key
GET /signing-keys/:id/verify # Verify a signature
```
---
## Implementation (Go)
### Dependencies
```go
import (
// OAuth2
"golang.org/x/oauth2"
"golang.org/x/oauth2/github"
"golang.org/x/oauth2/google"
// JWT
"github.com/golang-jwt/jwt/v5"
// Cryptography (all stdlib)
"crypto/ed25519"
"crypto/rand"
"crypto/sha256"
// Password/Key hashing
"golang.org/x/crypto/argon2"
"golang.org/x/crypto/bcrypt"
)
```
### OAuth2 Config
```go
var githubOAuth = &oauth2.Config{
ClientID: os.Getenv("GITHUB_CLIENT_ID"),
ClientSecret: os.Getenv("GITHUB_CLIENT_SECRET"),
Endpoint: github.Endpoint,
Scopes: []string{"read:user", "user:email"},
RedirectURL: "https://portal.mosis.dev/auth/github/callback",
}
```
### JWT Generation
```go
func generateAccessToken(developerID string) (string, error) {
claims := jwt.MapClaims{
"sub": developerID,
"type": "access",
"iat": time.Now().Unix(),
"exp": time.Now().Add(time.Hour).Unix(),
}
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
return token.SignedString([]byte(os.Getenv("JWT_SECRET")))
}
```
### Ed25519 Signing
```go
func signManifest(manifest []byte, privateKey ed25519.PrivateKey) []byte {
return ed25519.Sign(privateKey, manifest)
}
func verifySignature(manifest, signature []byte, publicKey ed25519.PublicKey) bool {
return ed25519.Verify(publicKey, manifest, signature)
}
```
### API Key Hashing
```go
func hashAPIKey(key string) (string, error) {
hash, err := bcrypt.GenerateFromPassword([]byte(key), bcrypt.DefaultCost)
return string(hash), err
}
func verifyAPIKey(key, hash string) bool {
return bcrypt.CompareHashAndPassword([]byte(hash), []byte(key)) == nil
}
```
---
## Deliverables
- [x] Auth approach decided (OAuth2 + JWT + API Keys)
- [x] Crypto libraries selected (Go stdlib + golang-jwt)
- [ ] OAuth2 integration (GitHub) - P0
- [ ] OAuth2 integration (Google) - P1
- [ ] JWT token management
- [ ] API key generation and validation
- [ ] Ed25519 key generation (CLI tool)
- [ ] Signature creation and verification
- [ ] Key registration API
- [ ] Audit logging
---
## Test Cases
| Test | Description |
|------|-------------|
| OAuthLogin | Complete OAuth flow |
| TokenRefresh | Refresh expired access token |
| InvalidToken | Reject tampered JWT |
| APIKeyAuth | Authenticate with API key |
| KeyGeneration | Generate valid Ed25519 keypair |
| SignPackage | Sign and verify package |
| InvalidSignature | Reject tampered package |
| KeyRevocation | Revoked key fails verification |
---
## Open Questions
1. ~~Support for hardware security keys (YubiKey)?~~ → Defer to post-MVP
2. ~~Multi-factor authentication for portal?~~ → Defer to post-MVP
3. Team accounts with role-based access? → Consider for v1.1
4. ~~Key escrow for enterprise customers?~~ → Not needed for self-hosted
---
## References
- [OAuth 2.0 RFC 6749](https://tools.ietf.org/html/rfc6749)
- [JWT RFC 7519](https://tools.ietf.org/html/rfc7519)
- [Ed25519 paper](https://ed25519.cr.yp.to/ed25519-20110926.pdf)
- [OWASP Authentication Cheatsheet](https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html)

592
docs/DEV_PORTAL_M05_FRONTEND.md Normal file
View File

@@ -0,0 +1,592 @@
# Milestone 5: Developer Portal Frontend
**Status**: Decided
**Goal**: Web interface for developer account and app management.
## Decision
**htmx + Go Templates** for server-rendered UI from the single Go container:
```
Rendering: Go html/template
Interactivity: htmx (partial page updates)
Styling: Tailwind CSS (compiled at build time)
Charts: Chart.js (lightweight)
Icons: Heroicons or Lucide
Build: Embed static assets in Go binary
```
### Rationale
1. **Single container** - No separate Node.js server needed
2. **Server-rendered** - All HTML generated by Go templates
3. **htmx for interactivity** - AJAX without JavaScript framework
4. **Embedded assets** - CSS/JS bundled into Go binary via `embed`
5. **Simple deployment** - Just the Go binary, nothing else
6. **Low resource usage** - Perfect for Synology NAS
### Architecture
```
┌─────────────────────────────────────────┐
│ mosis-portal container │
│ ┌────────────────────────────────────┐ │
│ │ Go binary │ │
│ │ ├── Chi router │ │
│ │ ├── html/template rendering │ │
│ │ ├── Embedded static assets │ │
│ │ │ ├── tailwind.css (built) │ │
│ │ │ ├── htmx.min.js │ │
│ │ │ └── chart.min.js │ │
│ │ └── SQLite database │ │
│ └────────────────────────────────────┘ │
└─────────────────────────────────────────┘
```
---
## Overview
The developer portal is the primary interface for developers to manage their accounts, create apps, submit versions, and view analytics.
---
## Pages Required
### Public Pages
| Page | URL | Purpose |
|------|-----|---------|
| Landing | `/` | Marketing, sign up CTA |
| Sign In | `/login` | OAuth + password login |
| Sign Up | `/register` | Create account |
| Docs | `/docs/*` | Documentation (separate site?) |
### Authenticated Pages
| Page | URL | Purpose |
|------|-----|---------|
| Dashboard | `/dashboard` | App list, quick stats |
| App Details | `/apps/:id` | Single app overview |
| App Settings | `/apps/:id/settings` | Edit app metadata |
| App Versions | `/apps/:id/versions` | Version history |
| Submit Version | `/apps/:id/versions/new` | Upload new version |
| App Analytics | `/apps/:id/analytics` | Telemetry dashboard |
| Create App | `/apps/new` | New app wizard |
| API Keys | `/settings/keys` | Manage API keys |
| Signing Keys | `/settings/signing` | Manage signing keys |
| Profile | `/settings/profile` | Account settings |
---
## Wireframes
### Dashboard
```
┌─────────────────────────────────────────────────────────────┐
│ [Logo] Dashboard Apps Docs Settings [Avatar ▼] │
├─────────────────────────────────────────────────────────────┤
│ │
│ Welcome back, Developer! │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Total Apps │ │ Downloads │ │ Active Users│ │
│ │ 12 │ │ 45,230 │ │ 1,234 │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │
│ Your Apps [+ New App] │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ [Icon] My Calculator v1.2.0 Published 1.2K ↓ │ │
│ │ [Icon] Notes App v2.0.1 Published 5.4K ↓ │ │
│ │ [Icon] Weather Widget v0.9.0 In Review --- │ │
│ └──────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
### App Details
```
┌─────────────────────────────────────────────────────────────┐
│ ← Back to Apps │
├─────────────────────────────────────────────────────────────┤
│ │
│ [Icon] My Calculator │
│ com.developer.calculator │
│ ● Published │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Overview │ │ Versions │ │Analytics │ │ Settings │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │
│ Latest Version: 1.2.0 [Submit New Version] │
│ Published: Jan 15, 2024 │
│ Downloads: 1,234 │
│ │
│ ┌────────────────────────────────────────┐ │
│ │ Downloads over time │ │
│ │ [Chart: line graph] │ │
│ └────────────────────────────────────────┘ │
│ │
│ Recent Crashes [View All →] │
│ • attempt to index nil (v1.2.0) - 23 reports │
│ • memory limit exceeded (v1.1.0) - 5 reports │
│ │
└─────────────────────────────────────────────────────────────┘
```
### Submit Version
```
┌─────────────────────────────────────────────────────────────┐
│ Submit New Version - My Calculator │
├─────────────────────────────────────────────────────────────┤
│ │
│ Step 2 of 3: Upload Package │
│ ○ Details ● Upload ○ Review │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ ┌─────────┐ │ │
│ │ │ .mosis │ Drop your package here │ │
│ │ │ 📦 │ or click to browse │ │
│ │ └─────────┘ │ │
│ │ │ │
│ │ Max size: 50 MB │ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ ✓ Package validated │
│ ✓ Signature verified │
│ ✓ Permissions: storage, network │
│ │
│ [Back] [Continue to Review] │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## Tech Stack Options
### Option A: Next.js 14
```
Framework: Next.js 14 (App Router)
UI: shadcn/ui + Tailwind CSS
State: React Query + Zustand
Forms: React Hook Form + Zod
Charts: Recharts or Tremor
Auth: NextAuth.js
```
| Pros | Cons |
|------|------|
| SSR for SEO landing page | Complexity |
| Great developer experience | React knowledge required |
| Full-stack capability | Heavier bundle |
| Large ecosystem | |
### Option B: SvelteKit
```
Framework: SvelteKit
UI: Skeleton UI or custom
State: Svelte stores
Forms: Superforms
Charts: Chart.js or LayerCake
Auth: Lucia
```
| Pros | Cons |
|------|------|
| Fast, small bundles | Smaller ecosystem |
| Simple state management | Fewer UI libraries |
| Great DX | Less hiring pool |
### Option C: Astro + React Islands
```
Framework: Astro
UI: React components (islands)
State: Nanostores
Forms: React Hook Form
Charts: Recharts
Auth: Custom
```
| Pros | Cons |
|------|------|
| Ultra-fast static pages | More setup |
| Partial hydration | Newer approach |
| Use React where needed | Less documented patterns |
### Option D: htmx + Go Templates
```
Framework: Go templates + htmx
UI: Tailwind CSS
State: Server-side
Forms: Native HTML
Charts: Chart.js
Auth: Server sessions
```
| Pros | Cons |
|------|------|
| Simple, fast | Limited interactivity |
| No JS framework | Less polished UX |
| Server-rendered | Complex UI harder |
---
## UI Component Library
### Option A: shadcn/ui
```
Base: Radix UI primitives
Styling: Tailwind CSS
Approach: Copy-paste components
```
| Pros | Cons |
|------|------|
| High quality | React only |
| Full control | Manual updates |
| Accessible | |
### Option B: Tailwind UI
```
Base: Headless UI
Styling: Tailwind CSS
Approach: Copy-paste templates
```
| Pros | Cons |
|------|------|
| Beautiful designs | Paid ($299) |
| Production-ready | Templates, not components |
### Option C: Mantine
```
Base: Custom components
Styling: CSS-in-JS or CSS
Approach: npm package
```
| Pros | Cons |
|------|------|
| Complete solution | Opinionated |
| Many components | Larger bundle |
| Good docs | |
---
## Key Features (htmx Implementation)
### File Upload with Progress
```html
<!-- htmx file upload with progress indicator -->
<form hx-post="/api/apps/{{.App.ID}}/versions"
hx-encoding="multipart/form-data"
hx-target="#upload-result"
hx-indicator="#upload-spinner">
<div class="dropzone" id="dropzone">
<input type="file" name="package" accept=".mosis" required
class="hidden" id="file-input">
<label for="file-input" class="cursor-pointer">
<svg><!-- upload icon --></svg>
<p>Drop your .mosis package here or click to browse</p>
<p class="text-sm text-gray-500">Max size: 50 MB</p>
</label>
</div>
<div id="upload-spinner" class="htmx-indicator">
Uploading...
</div>
<button type="submit" class="btn btn-primary">Upload</button>
</form>
<div id="upload-result"></div>
```
### Dynamic App List
```html
<!-- Server renders this, htmx updates it -->
<div id="app-list"
hx-get="/partials/apps"
hx-trigger="load, newApp from:body"
hx-swap="innerHTML">
{{range .Apps}}
<div class="app-card">
<img src="{{.IconURL}}" alt="{{.Name}}">
<h3>{{.Name}}</h3>
<span class="badge {{.StatusClass}}">{{.Status}}</span>
<a href="/apps/{{.ID}}" hx-boost="true">View →</a>
</div>
{{end}}
</div>
```
### Form with Validation
```html
<!-- Create app form with server-side validation -->
<form hx-post="/apps" hx-target="#form-result" hx-swap="outerHTML">
<div class="form-group">
<label for="name">App Name</label>
<input type="text" name="name" id="name" required
hx-post="/api/validate/name"
hx-trigger="blur"
hx-target="next .error">
<span class="error"></span>
</div>
<div class="form-group">
<label for="package_id">Package ID</label>
<input type="text" name="package_id" id="package_id"
placeholder="com.yourname.appname" required
hx-post="/api/validate/package-id"
hx-trigger="blur"
hx-target="next .error">
<span class="error"></span>
</div>
<button type="submit" class="btn btn-primary">Create App</button>
</form>
```
### Analytics Chart
```html
<!-- Chart.js for analytics -->
<div class="chart-container">
<canvas id="downloads-chart"></canvas>
</div>
<script>
// Data injected from Go template
const chartData = {{.ChartDataJSON}};
new Chart(document.getElementById('downloads-chart'), {
type: 'line',
data: {
labels: chartData.labels,
datasets: [{
label: 'Downloads',
data: chartData.values,
borderColor: '#6366f1',
tension: 0.3
}]
}
});
</script>
```
---
## Go Template Structure
### Base Layout
```go
// templates/layouts/base.html
{{define "base"}}
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{{.Title}} - Mosis Developer Portal</title>
<link href="/static/tailwind.css" rel="stylesheet">
<script src="/static/htmx.min.js"></script>
</head>
<body class="bg-gray-50" hx-boost="true">
{{template "navbar" .}}
<main class="container mx-auto py-8">
{{template "content" .}}
</main>
</body>
</html>
{{end}}
```
### Page Template
```go
// templates/pages/dashboard.html
{{define "content"}}
<h1 class="text-2xl font-bold mb-6">Dashboard</h1>
<div class="grid grid-cols-3 gap-6 mb-8">
{{template "stat-card" dict "Label" "Total Apps" "Value" .Stats.TotalApps}}
{{template "stat-card" dict "Label" "Downloads" "Value" .Stats.Downloads}}
{{template "stat-card" dict "Label" "Active Users" "Value" .Stats.ActiveUsers}}
</div>
<div class="flex justify-between items-center mb-4">
<h2 class="text-xl font-semibold">Your Apps</h2>
<a href="/apps/new" class="btn btn-primary">+ New App</a>
</div>
<div id="app-list" hx-get="/partials/apps" hx-trigger="load">
Loading...
</div>
{{end}}
```
---
## Authentication Flow
### Login Page (Go Template)
```html
{{define "content"}}
<div class="min-h-screen flex items-center justify-center">
<div class="card w-96 bg-white shadow-lg rounded-lg p-6">
<h1 class="text-2xl font-bold text-center mb-6">Sign in to Mosis</h1>
<div class="space-y-4">
<a href="/auth/github" class="btn btn-github w-full flex items-center justify-center gap-2">
<svg><!-- GitHub icon --></svg>
Continue with GitHub
</a>
<a href="/auth/google" class="btn btn-google w-full flex items-center justify-center gap-2">
<svg><!-- Google icon --></svg>
Continue with Google
</a>
<div class="divider">or</div>
<form hx-post="/auth/login" hx-target="#login-error" class="space-y-4">
<input type="email" name="email" placeholder="Email"
class="input input-bordered w-full" required>
<input type="password" name="password" placeholder="Password"
class="input input-bordered w-full" required>
<div id="login-error" class="text-red-500 text-sm"></div>
<button type="submit" class="btn btn-primary w-full">Sign in</button>
</form>
</div>
</div>
</div>
{{end}}
```
### Auth Middleware (Go)
```go
// middleware/auth.go
func RequireAuth(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
session, err := sessionStore.Get(r, "session")
if err != nil || session.Values["developer_id"] == nil {
http.Redirect(w, r, "/login", http.StatusSeeOther)
return
}
next.ServeHTTP(w, r)
})
}
// Usage in router
r.Group(func(r chi.Router) {
r.Use(RequireAuth)
r.Get("/dashboard", handlers.Dashboard)
r.Get("/apps", handlers.AppList)
r.Get("/apps/{id}", handlers.AppDetail)
})
```
---
## Responsive Design
### Breakpoints
| Size | Width | Target |
|------|-------|--------|
| sm | 640px | Mobile landscape |
| md | 768px | Tablet |
| lg | 1024px | Desktop |
| xl | 1280px | Large desktop |
### Mobile Navigation
```tsx
// Hamburger menu for mobile
<Sheet>
<SheetTrigger className="md:hidden">
<MenuIcon />
</SheetTrigger>
<SheetContent side="left">
<Navigation />
</SheetContent>
</Sheet>
```
---
## Accessibility
### Requirements
- [ ] Keyboard navigation
- [ ] Screen reader support
- [ ] Color contrast (WCAG AA)
- [ ] Focus indicators
- [ ] Alt text for images
- [ ] Form labels
- [ ] Error announcements
### Testing
```bash
# Lighthouse audit
npx lighthouse http://localhost:3000 --view
# axe-core
npm install @axe-core/react
```
---
## Deliverables
- [x] Framework selection (htmx + Go Templates)
- [x] UI component library selection (Tailwind CSS + Chart.js)
- [ ] Design system (colors, typography)
- [x] Page wireframes (see above)
- [x] Authentication flow (server sessions + OAuth)
- [ ] Dashboard implementation
- [ ] App management pages
- [ ] Version submission flow
- [ ] Settings pages
- [ ] Responsive design
- [ ] Accessibility audit
---
## Open Questions
1. ~~Dark mode support?~~ → Defer to post-MVP (Tailwind makes it easy to add later)
2. ~~Internationalization (i18n)?~~ → English only for MVP
3. ~~Custom domain for docs vs integrated?~~ → Integrated into same Go server at /docs
4. Email notifications UI? → Consider for v1.1
---
## References
- [shadcn/ui](https://ui.shadcn.com/)
- [Next.js App Router](https://nextjs.org/docs/app)
- [Tailwind CSS](https://tailwindcss.com/)
- [React Query](https://tanstack.com/query/latest)

715
docs/DEV_PORTAL_M06_API.md Normal file
View File

@@ -0,0 +1,715 @@
# Milestone 6: App Store Backend API
**Status**: Decided
**Goal**: REST API for app submission, review, and distribution.
## Decision
**Go + Chi router** with JSON REST API:
```
Framework: Chi (lightweight, idiomatic)
Validation: go-playground/validator/v10
OpenAPI: ogen (generated from spec)
Middleware: Custom auth, rate limiting, logging
Database: SQLite via repository pattern
```
### Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ mosis-portal container │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Chi Router │ │
│ │ /v1/auth/* /v1/apps/* /v1/store/* /v1/keys/* │ │
│ └──────────────────────┬─────────────────────────────────────┘ │
│ │ │
│ ┌──────────┬───────────┼───────────┬──────────┐ │
│ │ Handlers │ Services │ Repos │ SQLite │ │
│ └──────────┴───────────┴───────────┴──────────┘ │
│ │
│ /volume1/mosis/data/portal.db │
│ /volume1/mosis/packages/ │
└─────────────────────────────────────────────────────────────────┘
```
---
## Overview
The backend API serves the developer portal, CLI tools, and device-side app management. It handles authentication, app lifecycle, file storage, and telemetry ingestion.
---
## API Design Principles
1. **RESTful** - Standard HTTP methods and status codes
2. **JSON** - Request and response bodies in JSON
3. **Versioned** - `/v1/` prefix for breaking changes
4. **Consistent** - Same patterns across all endpoints
5. **Documented** - OpenAPI specification
---
## Base URL
```
Production: https://api.mosis.dev/v1
Staging: https://api.staging.mosis.dev/v1
Local: http://localhost:8080/v1
```
---
## Authentication
### Headers
```
Authorization: Bearer <jwt_token>
# or
X-API-Key: mk_live_xxxxxxxx
```
### Scopes
| Scope | Description |
|-------|-------------|
| `apps:read` | Read app metadata |
| `apps:write` | Create/update apps |
| `versions:upload` | Upload new versions |
| `versions:publish` | Publish versions |
| `telemetry:read` | Read analytics |
| `keys:manage` | Manage API keys |
---
## Endpoints
### Authentication
```yaml
POST /v1/auth/oauth/github:
summary: Start GitHub OAuth flow
response: { redirect_url: string }
GET /v1/auth/oauth/github/callback:
summary: GitHub OAuth callback
query:
code: string
state: string
response: { access_token, refresh_token, user }
POST /v1/auth/oauth/google:
summary: Start Google OAuth flow
response: { redirect_url: string }
GET /v1/auth/oauth/google/callback:
summary: Google OAuth callback
POST /v1/auth/refresh:
summary: Refresh access token
body: { refresh_token: string }
response: { access_token, refresh_token }
POST /v1/auth/logout:
summary: Invalidate tokens
auth: required
GET /v1/auth/me:
summary: Get current user
auth: required
response: Developer
```
### Apps
```yaml
GET /v1/apps:
summary: List developer's apps
auth: required
query:
status: draft | published | suspended
page: number
limit: number
response: { apps: App[], total: number }
POST /v1/apps:
summary: Create new app
auth: required
body:
package_id: string # com.developer.appname
name: string
description?: string
category?: string
response: App
GET /v1/apps/:id:
summary: Get app details
auth: required
response: App
PATCH /v1/apps/:id:
summary: Update app metadata
auth: required
body:
name?: string
description?: string
category?: string
tags?: string[]
response: App
DELETE /v1/apps/:id:
summary: Delete app (if no published versions)
auth: required
response: { success: true }
```
### App Versions
```yaml
GET /v1/apps/:id/versions:
summary: List app versions
auth: required
query:
status: draft | review | approved | published | rejected
page: number
limit: number
response: { versions: AppVersion[], total: number }
POST /v1/apps/:id/versions:
summary: Create new version (get upload URL)
auth: required
body:
version_name: string # 1.0.0
version_code: number # 1
release_notes?: string
response:
version: AppVersion
upload_url: string # Presigned S3 URL
upload_expires: string # ISO timestamp
PUT /v1/apps/:id/versions/:vid/upload-complete:
summary: Mark upload as complete, trigger validation
auth: required
response: AppVersion
GET /v1/apps/:id/versions/:vid:
summary: Get version details
auth: required
response: AppVersion
POST /v1/apps/:id/versions/:vid/submit:
summary: Submit version for review
auth: required
response: AppVersion # status: review
POST /v1/apps/:id/versions/:vid/publish:
summary: Publish approved version
auth: required
response: AppVersion # status: published
DELETE /v1/apps/:id/versions/:vid:
summary: Delete draft version
auth: required
response: { success: true }
```
### Public App Store
```yaml
GET /v1/store/apps:
summary: Browse/search published apps
auth: none
query:
q: string # Search query
category: string
sort: popular | recent | name
page: number
limit: number
response: { apps: PublicApp[], total: number }
GET /v1/store/apps/:package_id:
summary: Get app store listing
auth: none
response: PublicApp
GET /v1/store/apps/:package_id/download:
summary: Get download URL for latest version
auth: none (or device token)
response:
download_url: string
version: string
size: number
signature: string
GET /v1/store/apps/:package_id/versions/:version_code/download:
summary: Get download URL for specific version
auth: none
response: { download_url, version, size, signature }
```
### API Keys
```yaml
GET /v1/keys:
summary: List API keys
auth: required
response: { keys: APIKey[] }
POST /v1/keys:
summary: Create API key
auth: required
body:
name: string
permissions: string[]
expires_at?: string
response:
key: APIKey
secret: string # Only shown once!
DELETE /v1/keys/:id:
summary: Revoke API key
auth: required
response: { success: true }
```
### Signing Keys
```yaml
GET /v1/signing-keys:
summary: List signing keys
auth: required
response: { keys: SigningKey[] }
POST /v1/signing-keys:
summary: Register signing key
auth: required
body:
name: string
public_key: string # PEM format
response: SigningKey
DELETE /v1/signing-keys/:id:
summary: Revoke signing key
auth: required
response: { success: true }
```
### Telemetry
```yaml
POST /v1/telemetry/events:
summary: Submit telemetry events (batch)
auth: device token or API key
body:
events:
- app_id: string
event_type: string
event_data: object
timestamp: string
response: { received: number }
POST /v1/telemetry/crash:
summary: Submit crash report
auth: device token or API key
body:
app_id: string
app_version: string
crash_type: string
message: string
stack_trace: string
context: object
timestamp: string
response: { id: string }
GET /v1/apps/:id/analytics:
summary: Get app analytics
auth: required
query:
start_date: string
end_date: string
metrics: downloads | active_users | crashes
response:
data:
- date: string
downloads: number
active_users: number
crashes: number
GET /v1/apps/:id/crashes:
summary: Get crash reports
auth: required
query:
version?: string
page: number
limit: number
response: { crashes: CrashReport[], total: number }
```
---
## Data Models (Go)
### Developer
```go
type Developer struct {
ID string `json:"id"`
Email string `json:"email"`
Name string `json:"name"`
AvatarURL *string `json:"avatar_url,omitempty"`
Verified bool `json:"verified"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
```
### App
```go
type AppStatus string
const (
AppStatusDraft AppStatus = "draft"
AppStatusPublished AppStatus = "published"
AppStatusSuspended AppStatus = "suspended"
)
type App struct {
ID string `json:"id"`
PackageID string `json:"package_id"`
Name string `json:"name"`
Description *string `json:"description,omitempty"`
Category *string `json:"category,omitempty"`
Tags []string `json:"tags"`
Status AppStatus `json:"status"`
IconURL *string `json:"icon_url,omitempty"`
LatestVersion *AppVersion `json:"latest_version,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
```
### AppVersion
```go
type VersionStatus string
const (
VersionStatusDraft VersionStatus = "draft"
VersionStatusUploading VersionStatus = "uploading"
VersionStatusValidating VersionStatus = "validating"
VersionStatusReview VersionStatus = "review"
VersionStatusApproved VersionStatus = "approved"
VersionStatusPublished VersionStatus = "published"
VersionStatusRejected VersionStatus = "rejected"
)
type AppVersion struct {
ID string `json:"id"`
AppID string `json:"app_id"`
VersionName string `json:"version_name"`
VersionCode int `json:"version_code"`
PackageURL *string `json:"package_url,omitempty"`
PackageSize *int64 `json:"package_size,omitempty"`
Signature *string `json:"signature,omitempty"`
Permissions []string `json:"permissions"`
MinMosisVersion *string `json:"min_mosis_version,omitempty"`
ReleaseNotes *string `json:"release_notes,omitempty"`
Status VersionStatus `json:"status"`
ReviewNotes *string `json:"review_notes,omitempty"`
PublishedAt *time.Time `json:"published_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
```
### PublicApp
```go
type PublicApp struct {
PackageID string `json:"package_id"`
Name string `json:"name"`
Description *string `json:"description,omitempty"`
Category *string `json:"category,omitempty"`
Tags []string `json:"tags"`
IconURL *string `json:"icon_url,omitempty"`
AuthorName string `json:"author_name"`
LatestVersion string `json:"latest_version"`
DownloadCount int64 `json:"download_count"`
Rating *float64 `json:"rating,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
```
### APIKey
```go
type APIKey struct {
ID string `json:"id"`
Name string `json:"name"`
KeyPrefix string `json:"key_prefix"` // "mk_live_abc..."
Permissions []string `json:"permissions"`
LastUsedAt *time.Time `json:"last_used_at,omitempty"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
```
### CrashReport
```go
type CrashReport struct {
ID string `json:"id"`
AppID string `json:"app_id"`
AppVersion string `json:"app_version"`
CrashType string `json:"crash_type"`
Message string `json:"message"`
StackTrace string `json:"stack_trace"`
Context map[string]interface{} `json:"context"`
Occurrences int `json:"occurrences"`
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
}
```
---
## Error Handling
### Error Response Format
```json
{
"error": {
"code": "VALIDATION_ERROR",
"message": "Invalid package_id format",
"details": {
"field": "package_id",
"constraint": "Must match pattern: ^[a-z][a-z0-9]*(\\.[a-z][a-z0-9]*)+$"
}
}
}
```
### Error Codes
| Code | HTTP Status | Description |
|------|-------------|-------------|
| `UNAUTHORIZED` | 401 | Missing or invalid auth |
| `FORBIDDEN` | 403 | Insufficient permissions |
| `NOT_FOUND` | 404 | Resource not found |
| `VALIDATION_ERROR` | 400 | Invalid request body |
| `CONFLICT` | 409 | Resource already exists |
| `RATE_LIMITED` | 429 | Too many requests |
| `INTERNAL_ERROR` | 500 | Server error |
---
## Rate Limiting
### Headers
```
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 999
X-RateLimit-Reset: 1704067200
```
### Limits
| Endpoint Category | Limit | Window |
|-------------------|-------|--------|
| Auth | 10 | 1 minute |
| Read | 1000 | 1 hour |
| Write | 100 | 1 hour |
| Upload | 10 | 1 hour |
| Telemetry | 10000 | 1 hour |
---
## Pagination
### Request
```
GET /v1/apps?page=2&limit=20
```
### Response
```json
{
"apps": [...],
"pagination": {
"page": 2,
"limit": 20,
"total": 45,
"total_pages": 3
}
}
```
---
## Webhooks (Future)
```yaml
POST /v1/webhooks:
summary: Register webhook
body:
url: string
events: string[] # version.published, crash.new
secret: string
Webhook Payload:
headers:
X-Mosis-Signature: sha256=xxx
body:
event: string
data: object
timestamp: string
```
---
## OpenAPI Specification
Full OpenAPI 3.0 spec will be generated and hosted at:
- `https://api.mosis.dev/v1/openapi.json`
- `https://api.mosis.dev/v1/docs` (Swagger UI)
---
## Implementation Structure (Go)
```
cmd/
└── portal/
└── main.go # Entry point, wire dependencies
internal/
├── api/
│ ├── router.go # Chi router setup
│ ├── middleware/
│ │ ├── auth.go # JWT/API key validation
│ │ ├── ratelimit.go # Token bucket rate limiter
│ │ ├── logging.go # Request/response logging
│ │ └── recovery.go # Panic recovery
│ └── handlers/
│ ├── auth.go # OAuth2 + token endpoints
│ ├── apps.go # App CRUD
│ ├── versions.go # Version upload/publish
│ ├── store.go # Public store API
│ ├── keys.go # API/signing keys
│ └── telemetry.go # Event ingestion
├── service/
│ ├── app.go # Business logic
│ ├── version.go
│ ├── auth.go
│ └── storage.go # File storage operations
├── repository/
│ ├── sqlite/ # SQLite implementations
│ │ ├── app.go
│ │ ├── version.go
│ │ ├── developer.go
│ │ └── migrations/ # SQL migrations
│ └── interfaces.go # Repository interfaces
└── domain/
├── app.go # Domain types
├── version.go
└── developer.go
pkg/
├── validator/ # Custom validation rules
└── signing/ # Ed25519 operations
```
### Chi Router Setup
```go
func NewRouter(
authHandler *handlers.AuthHandler,
appHandler *handlers.AppHandler,
storeHandler *handlers.StoreHandler,
) chi.Router {
r := chi.NewRouter()
// Global middleware
r.Use(middleware.RequestID)
r.Use(middleware.RealIP)
r.Use(middleware.Logger)
r.Use(middleware.Recoverer)
r.Use(middleware.Timeout(30 * time.Second))
// API v1
r.Route("/v1", func(r chi.Router) {
// Public routes
r.Group(func(r chi.Router) {
r.Post("/auth/oauth/github", authHandler.GitHubOAuth)
r.Get("/auth/oauth/github/callback", authHandler.GitHubCallback)
r.Post("/auth/refresh", authHandler.Refresh)
r.Get("/store/apps", storeHandler.ListApps)
r.Get("/store/apps/{packageID}", storeHandler.GetApp)
})
// Protected routes
r.Group(func(r chi.Router) {
r.Use(middleware.RequireAuth)
r.Get("/auth/me", authHandler.Me)
r.Route("/apps", func(r chi.Router) {
r.Get("/", appHandler.List)
r.Post("/", appHandler.Create)
r.Route("/{appID}", func(r chi.Router) {
r.Get("/", appHandler.Get)
r.Patch("/", appHandler.Update)
r.Delete("/", appHandler.Delete)
r.Route("/versions", func(r chi.Router) {
r.Get("/", appHandler.ListVersions)
r.Post("/", appHandler.CreateVersion)
})
})
})
})
})
return r
}
```
---
## Deliverables
- [ ] OpenAPI specification
- [ ] Authentication middleware
- [ ] Rate limiting middleware
- [ ] Auth endpoints
- [ ] Apps CRUD endpoints
- [ ] Versions endpoints with upload flow
- [ ] Store public endpoints
- [ ] API keys management
- [ ] Signing keys management
- [ ] Telemetry ingestion
- [ ] Error handling
- [ ] Request validation
- [ ] Integration tests
---
## Open Questions
1. ~~GraphQL alongside REST?~~ → REST only for simplicity
2. WebSocket for real-time review status? → Consider for v1.1
3. ~~Batch operations for bulk updates?~~ → Not needed for MVP
4. ~~API versioning strategy (URL vs header)?~~ → URL prefix (/v1/)
---
## References
- [REST API Design Guidelines](https://github.com/microsoft/api-guidelines)
- [OpenAPI 3.0 Specification](https://swagger.io/specification/)
- [HTTP Status Codes](https://httpstatuses.com/)

627
docs/DEV_PORTAL_M07_STORAGE.md Normal file
View File

@@ -0,0 +1,627 @@
# Milestone 7: CDN & Storage
**Status**: Decided
**Goal**: Scalable storage for app packages and assets with global distribution.
## Decision
**Local Synology filesystem** as primary storage, with optional Cloudflare R2 for CDN:
```
Primary: Synology volume (/volume1/mosis/)
CDN: Cloudflare R2 (optional, for global distribution)
Serving: Go binary serves files directly (local)
Backup: Synology Hyper Backup / rsync
```
### Rationale
1. **Self-hosted** - All data stays on premises
2. **Zero cost** - No cloud storage fees
3. **Simple** - Go binary serves files directly via http.FileServer
4. **Fast local** - NAS has gigabit+ internal network
5. **Optional CDN** - Can sync to R2 if global distribution needed
### Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ Synology NAS │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ mosis-portal container │ │
│ │ │ │
│ │ Go binary ──serves──► /packages/, /assets/ │ │
│ │ │ │ │
│ │ └── SQLite (/data/portal.db) │ │
│ └──────────────────────────┬─────────────────────────────────┘ │
│ │ │
│ /volume1/mosis/ │ bind mount │
│ ├── data/portal.db │ │
│ ├── packages/ ◄───────────┘ │
│ │ └── {dev_id}/{app_id}/{version}/package.mosis │
│ ├── assets/ │
│ │ └── {app_id}/icon-{size}.png │
│ └── backups/ │
│ └── litestream replicas │
└─────────────────────────────────────────────────────────────────┘
│
(optional sync)
▼
┌──────────────────┐
│ Cloudflare R2 │
│ (CDN for global │
│ distribution) │
└──────────────────┘
```
---
## Overview
Storage handles app packages, icons, screenshots, and serves downloads to devices. For self-hosted Synology NAS deployment, local filesystem is the primary storage.
---
## Requirements
### Functional
- Store app packages (.mosis files)
- Store icons (multiple sizes)
- Store screenshots (optional)
- Serve downloads globally
- Support presigned upload URLs
- Version retention (keep old versions)
### Non-Functional
| Requirement | Target |
|-------------|--------|
| Upload speed | < 30s for 50MB |
| Download latency | < 100ms (p50) |
| Availability | 99.9% |
| Durability | 99.999999999% (11 nines) |
| Cost | Minimize egress fees |
---
## Storage Structure
```
/packages/
/{developer_id}/
/{app_id}/
/{version_code}/
package.mosis
manifest.json (extracted for quick access)
/assets/
/{app_id}/
icon-32.png
icon-64.png
icon-128.png
screenshots/
1.png
2.png
3.png
/temp/
/{upload_id}/
package.mosis (pending validation)
```
---
## Options Analysis
### Option A: Cloudflare R2
```
Storage: Object storage (S3-compatible)
CDN: Cloudflare network (automatic)
Egress: FREE (zero egress fees)
Pricing: $0.015/GB storage, $4.50/million requests
```
| Pros | Cons |
|------|------|
| No egress fees | Newer service |
| Global CDN included | Fewer regions than S3 |
| S3-compatible API | Less tooling |
| Workers integration | |
#### Cost Estimate (10K apps, 100GB)
| Component | Monthly Cost |
|-----------|--------------|
| Storage (100GB) | $1.50 |
| Requests (1M) | $4.50 |
| Egress | $0 |
| **Total** | **~$6** |
---
### Option B: AWS S3 + CloudFront
```
Storage: S3 Standard
CDN: CloudFront
Egress: $0.085-0.12/GB (varies by region)
Pricing: $0.023/GB storage
```
| Pros | Cons |
|------|------|
| Most mature | Egress costs add up |
| Best tooling | Complex pricing |
| All regions | Need CloudFront config |
| IAM integration | |
#### Cost Estimate (10K apps, 100GB, 1TB egress)
| Component | Monthly Cost |
|-----------|--------------|
| Storage (100GB) | $2.30 |
| CloudFront (1TB) | $85 |
| Requests | ~$5 |
| **Total** | **~$92** |
---
### Option C: Backblaze B2 + Cloudflare
```
Storage: Backblaze B2
CDN: Cloudflare (free egress via Bandwidth Alliance)
Egress: FREE (through Cloudflare)
Pricing: $0.005/GB storage
```
| Pros | Cons |
|------|------|
| Cheapest storage | Two services to manage |
| Free egress via CF | B2 API slightly different |
| Good reliability | Need CF proxy setup |
#### Cost Estimate (10K apps, 100GB)
| Component | Monthly Cost |
|-----------|--------------|
| Storage (100GB) | $0.50 |
| Egress (via CF) | $0 |
| Requests | ~$0.40 |
| **Total** | **~$1** |
---
### Option D: Self-hosted MinIO
```
Storage: MinIO on VPS
CDN: Cloudflare proxy
Egress: VPS bandwidth
Pricing: VPS cost only
```
| Pros | Cons |
|------|------|
| Full control | Ops overhead |
| S3-compatible | Need to manage |
| Predictable cost | Scaling complexity |
#### Cost Estimate
| Component | Monthly Cost |
|-----------|--------------|
| VPS (500GB SSD) | $20-50 |
| Cloudflare | $0 (free tier) |
| **Total** | **~$20-50** |
---
## Recommendation
**Primary: Local Synology Filesystem**
- Zero recurring costs
- All data on premises
- Simple Go file serving
- Synology's built-in backup tools
**Optional: Cloudflare R2 for CDN**
- If global distribution needed
- Sync packages via cron/background job
- Zero egress fees
- S3-compatible API
---
## Upload Flow
### Local Filesystem Approach
```
┌────────┐ ┌─────────┐ ┌─────────────────┐
│ Client │───►│ API │───►│ Local Filesystem│
└────────┘ └─────────┘ └─────────────────┘
│ │ │
│ 1. Upload package (multipart) │
│─────────────►│ │
│ │ │
│ │ 2. Save to temp │
│ │─────────────────►│
│ │ │
│ │ 3. Validate │
│ │ 4. Move to final│
│ │─────────────────►│
│ │ │
│ 5. Confirm │ │
│◄─────────────┤ │
```
### Go Implementation (Local Storage)
```go
// Upload handler - direct file upload
func (h *VersionHandler) Upload(w http.ResponseWriter, r *http.Request) {
appID := chi.URLParam(r, "appID")
devID := r.Context().Value("developer_id").(string)
// Parse multipart form (max 50MB)
r.ParseMultipartForm(50 << 20)
file, header, err := r.FormFile("package")
if err != nil {
respondError(w, http.StatusBadRequest, "INVALID_FILE", "No package file")
return
}
defer file.Close()
// Create version record
version := &domain.Version{
ID: uuid.New().String(),
AppID: appID,
VersionCode: parseInt(r.FormValue("version_code")),
VersionName: r.FormValue("version_name"),
Status: domain.VersionStatusUploading,
}
// Save to temp directory
tempPath := filepath.Join(h.storagePath, "temp", version.ID, "package.mosis")
os.MkdirAll(filepath.Dir(tempPath), 0755)
dst, err := os.Create(tempPath)
if err != nil {
respondError(w, http.StatusInternalServerError, "STORAGE_ERROR", err.Error())
return
}
defer dst.Close()
size, err := io.Copy(dst, file)
version.PackageSize = size
// Validate package (signature, manifest, etc.)
if err := h.validator.Validate(tempPath); err != nil {
os.Remove(tempPath)
respondError(w, http.StatusBadRequest, "VALIDATION_ERROR", err.Error())
return
}
// Move to final location
finalPath := filepath.Join(h.storagePath, "packages",
devID, appID, fmt.Sprintf("%d", version.VersionCode), "package.mosis")
os.MkdirAll(filepath.Dir(finalPath), 0755)
os.Rename(tempPath, finalPath)
version.PackageURL = finalPath
version.Status = domain.VersionStatusDraft
h.repo.Save(version)
respondJSON(w, http.StatusCreated, version)
}
```
### Optional: Presigned URL for R2 CDN
If global distribution is needed, sync to R2 and generate presigned download URLs:
```go
// Sync to R2 after publishing (background job)
func (s *SyncService) SyncToR2(version *domain.Version) error {
localPath := version.PackageURL
r2Key := fmt.Sprintf("packages/%s/%s/%d/package.mosis",
version.DeveloperID, version.AppID, version.VersionCode)
file, _ := os.Open(localPath)
defer file.Close()
_, err := s.r2.PutObject(r2Key, file)
return err
}
```
---
## Download Flow
### Local File Serving
```go
// Serve package downloads directly from filesystem
func (h *StoreHandler) Download(w http.ResponseWriter, r *http.Request) {
packageID := chi.URLParam(r, "packageID")
// Get latest published version
version, err := h.repo.GetLatestPublished(packageID)
if err != nil {
respondError(w, http.StatusNotFound, "NOT_FOUND", "App not found")
return
}
// Serve file directly
filePath := version.PackageURL
file, err := os.Open(filePath)
if err != nil {
respondError(w, http.StatusNotFound, "FILE_NOT_FOUND", "Package not available")
return
}
defer file.Close()
stat, _ := file.Stat()
// Set headers
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%s-%s.mosis",
packageID, version.VersionName))
w.Header().Set("Content-Length", fmt.Sprintf("%d", stat.Size()))
w.Header().Set("X-Mosis-Version", version.VersionName)
w.Header().Set("X-Mosis-Signature", version.Signature)
// Stream file
io.Copy(w, file)
}
// Alternative: Return download info + serve via static file handler
func (h *StoreHandler) GetDownloadInfo(w http.ResponseWriter, r *http.Request) {
packageID := chi.URLParam(r, "packageID")
version, _ := h.repo.GetLatestPublished(packageID)
respondJSON(w, http.StatusOK, map[string]interface{}{
"download_url": fmt.Sprintf("/downloads/%s/%s/%d/package.mosis",
version.DeveloperID, version.AppID, version.VersionCode),
"version": version.VersionName,
"size": version.PackageSize,
"signature": version.Signature,
})
}
```
### Static File Server
```go
// In router setup - serve packages directory
r.Handle("/downloads/*", http.StripPrefix("/downloads/",
http.FileServer(http.Dir("/volume1/mosis/packages"))))
```
### Caching (via Nginx or Cloudflare Tunnel)
```
Cache-Control: public, max-age=86400
```
- Packages are immutable (version code = unique)
- Put Nginx in front for caching if needed
- Or use Cloudflare Tunnel for edge caching
---
## Icon/Screenshot Handling
### Upload (Go)
```go
// Icons uploaded with app creation/update
func (h *AppHandler) UploadIcon(w http.ResponseWriter, r *http.Request) {
appID := chi.URLParam(r, "appID")
r.ParseMultipartForm(10 << 20) // 10MB max
file, _, err := r.FormFile("icon")
if err != nil {
respondError(w, http.StatusBadRequest, "INVALID_FILE", "No icon file")
return
}
defer file.Close()
// Decode and validate
img, _, err := image.Decode(file)
if err != nil {
respondError(w, http.StatusBadRequest, "INVALID_IMAGE", "Cannot decode image")
return
}
bounds := img.Bounds()
if bounds.Dx() != bounds.Dy() {
respondError(w, http.StatusBadRequest, "INVALID_DIMENSIONS", "Icon must be square")
return
}
// Generate multiple sizes
sizes := []int{32, 64, 128}
assetsDir := filepath.Join(h.storagePath, "assets", appID)
os.MkdirAll(assetsDir, 0755)
for _, size := range sizes {
resized := resize.Resize(uint(size), uint(size), img, resize.Lanczos3)
outPath := filepath.Join(assetsDir, fmt.Sprintf("icon-%d.png", size))
out, _ := os.Create(outPath)
png.Encode(out, resized)
out.Close()
}
respondJSON(w, http.StatusOK, map[string]string{"status": "uploaded"})
}
```
### Serving
```go
// Serve assets via static file handler
r.Handle("/assets/*", http.StripPrefix("/assets/",
http.FileServer(http.Dir("/volume1/mosis/assets"))))
```
URLs:
```
https://portal.mosis.local/assets/{app_id}/icon-64.png
```
- Public read access
- Long cache TTL (icons rarely change)
- Nginx caching if needed
---
## Retention Policy
### Packages
| Status | Retention |
|--------|-----------|
| Published (current) | Forever |
| Published (old) | 1 year |
| Draft | 30 days |
| Rejected | 7 days |
| Failed validation | 24 hours |
### Temp Uploads
- Delete after 1 hour if not completed
- Lifecycle rule on `/temp/` prefix
### Implementation
```yaml
# R2 Lifecycle Rules
rules:
- prefix: "temp/"
expiration_days: 1
- prefix: "packages/"
tags:
status: draft
expiration_days: 30
```
---
## Backup Strategy
### Synology Built-in Options
- **Hyper Backup** - Backup to external drive, another NAS, or cloud
- **Snapshot Replication** - Point-in-time snapshots (Btrfs)
- **rsync** - Script-based backup to remote location
### Recommended Setup
```bash
# Cron job: Daily backup of packages to external drive
0 3 * * * rsync -av /volume1/mosis/packages/ /volumeUSB1/mosis-backup/packages/
# Or use Synology Hyper Backup with versioning
```
### Metadata Backup
- Package metadata in SQLite (portal.db)
- Litestream handles continuous replication
- Can regenerate file paths from DB
### Recovery
```bash
# Restore from backup
rsync -av /volumeUSB1/mosis-backup/packages/ /volume1/mosis/packages/
# Restore database (via Litestream)
litestream restore -o /data/portal.db /backups/portal/
```
---
## Monitoring
### Metrics to Track
| Metric | Source |
|--------|--------|
| Upload success rate | API logs |
| Download latency | Cloudflare analytics |
| Storage usage | R2 dashboard |
| Bandwidth | R2 dashboard |
| Error rate | API logs |
### Alerts
- Upload failure rate > 5%
- Download error rate > 1%
- Storage > 80% of budget
---
## Security
### Access Control
```
Packages:
- Write: API only (presigned URLs)
- Read: Public (presigned URLs)
Assets:
- Write: API only
- Read: Public (direct CDN)
Temp:
- Write: Presigned URLs (15 min)
- Read: None (API internal only)
```
### Signed URLs
```go
// Presigned URL with expiration
url := r2.PresignPut(key, 15*time.Minute, PutOptions{
ContentType: "application/octet-stream",
ContentLength: maxSize,
})
```
---
## Deliverables
- [x] Storage approach decided (local Synology filesystem)
- [ ] Upload flow implementation (multipart to local)
- [ ] Download serving (http.FileServer)
- [ ] Icon/screenshot upload and resize
- [ ] Temp file cleanup (cron job)
- [ ] Backup setup (Hyper Backup or rsync)
- [ ] (Optional) R2 sync for CDN
---
## Open Questions
1. ~~Multi-region storage for lower latency?~~ → Use R2 sync if needed
2. ~~Package compression (gzip/brotli)?~~ → Defer, .mosis is already ZIP
3. Delta updates storage structure? → Consider for v1.1
4. ~~Screenshot requirements (dimensions, count)?~~ → Max 5, 1280x720 or 720x1280
---
## References
- [Cloudflare R2 Documentation](https://developers.cloudflare.com/r2/)
- [S3 Presigned URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html)
- [Backblaze B2 + Cloudflare](https://www.backblaze.com/b2/docs/cloud_to_cloud.html)

596
docs/DEV_PORTAL_M08_TELEMETRY.md Normal file
View File

@@ -0,0 +1,596 @@
# Milestone 8: Telemetry System
**Status**: Decided
**Goal**: Collect app usage analytics and crash reports while respecting privacy.
## Decision
**SQLite with background aggregation** for self-hosted Synology NAS:
```
Storage: SQLite (separate telemetry.db to isolate write load)
Aggregation: Go background goroutine (hourly/daily rollups)
Retention: Raw events 7 days, aggregates indefinitely
Privacy: Hashed device IDs, no PII, opt-out available
```
### Rationale
1. **Simple** - No separate time-series database needed
2. **SQLite scales** - Can handle thousands of events/day easily
3. **Background jobs** - Go goroutines for aggregation, cleanup
4. **Separate DB** - Telemetry writes don't affect main portal.db
5. **Privacy-first** - Minimal collection, hashed IDs
### Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ mosis-portal container │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Go Binary │ │
│ │ ┌─────────────┐ ┌────────────────┐ │ │
│ │ │ API Handler │───►│ Telemetry Svc │ │ │
│ │ │ POST /v1/ │ │ - Buffer events│ │ │
│ │ │ telemetry/* │ │ - Batch insert │ │ │
│ │ └─────────────┘ └───────┬────────┘ │ │
│ │ │ │ │
│ │ ┌─────────────────────────▼────────────────────────────┐ │ │
│ │ │ Background Workers │ │ │
│ │ │ • Hourly aggregation (event counts, unique devices) │ │ │
│ │ │ • Daily cleanup (delete raw events > 7 days) │ │ │
│ │ │ • Crash grouping (fingerprint + dedup) │ │ │
│ │ └───────────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────┬─────────────────────────────┘ │
│ │ │
│ /volume1/mosis/data/ │ │
│ ├── portal.db (main) │ │
│ └── telemetry.db ◄────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
---
## Overview
Telemetry provides developers with insights into app usage, performance, and crashes. Must balance usefulness with user privacy.
---
## Privacy Principles
1. **Minimal collection** - Only what's necessary
2. **No PII by default** - Anonymized device IDs
3. **Transparency** - Users know what's collected
4. **Opt-out available** - Users can disable
5. **Data retention limits** - Auto-delete old data
6. **GDPR compliance** - Export/delete on request
---
## Event Types
### Automatic Events (Default)
| Event | Description | Data |
|-------|-------------|------|
| `app_start` | App launched | version, mosis_version |
| `app_stop` | App closed | duration_seconds |
| `app_crash` | Unhandled error | crash_type, message |
| `lua_error` | Lua runtime error | message, stack (no user data) |
### Performance Events (Default)
| Event | Description | Data |
|-------|-------------|------|
| `perf_frame` | Frame time (sampled) | avg_ms, p95_ms |
| `perf_memory` | Memory usage | used_mb, limit_mb |
| `perf_startup` | Startup time | duration_ms |
### Usage Events (Opt-in)
| Event | Description | Data |
|-------|-------------|------|
| `screen_view` | Screen navigation | screen_name |
| `button_click` | UI interaction | element_id |
| `feature_used` | Feature usage | feature_name |
---
## Data Schema
### Event Payload
```json
{
"app_id": "com.developer.myapp",
"app_version": "1.2.0",
"mosis_version": "1.0.0",
"device_id": "sha256_hashed_id",
"session_id": "uuid",
"events": [
{
"type": "app_start",
"timestamp": "2024-01-15T10:30:00Z",
"data": {}
},
{
"type": "screen_view",
"timestamp": "2024-01-15T10:30:05Z",
"data": {
"screen_name": "home"
}
}
]
}
```
### Crash Report Payload
```json
{
"app_id": "com.developer.myapp",
"app_version": "1.2.0",
"mosis_version": "1.0.0",
"device_id": "sha256_hashed_id",
"timestamp": "2024-01-15T10:35:00Z",
"crash": {
"type": "lua_error",
"message": "attempt to index nil value 'user'",
"stack_trace": "main.lua:42: in function 'loadUser'\nmain.lua:15: in main chunk",
"context": {
"screen": "profile.rml",
"memory_mb": 45,
"uptime_seconds": 300
}
}
}
```
### Device ID Hashing
```lua
-- On device
local raw_id = get_android_id() -- or similar
local hashed = sha256(raw_id .. "mosis_salt_" .. app_id)
-- Result: "a3f2b1c4d5e6..."
-- Cannot reverse to original device ID
-- Different per app (can't track across apps)
```
---
## Collection Architecture
```
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Device │────►│ Batch │────►│ API │────►│ Storage │
│ │ │ Queue │ │ │ │ │
└──────────┘ └──────────┘ └──────────┘ └──────────┘
│
│ Every 60s or
│ on app close
▼
┌──────────┐
│ Upload │
└──────────┘
```
### Client-Side Batching
```lua
-- TelemetryManager on device
local events = {}
local last_flush = os.time()
function track(event_type, data)
if not telemetry_enabled then return end
table.insert(events, {
type = event_type,
timestamp = os.date("!%Y-%m-%dT%H:%M:%SZ"),
data = data or {}
})
-- Flush if batch is large or time elapsed
if #events >= 50 or (os.time() - last_flush) > 60 then
flush()
end
end
function flush()
if #events == 0 then return end
local payload = {
app_id = APP_ID,
app_version = APP_VERSION,
device_id = HASHED_DEVICE_ID,
events = events
}
-- Async HTTP POST
http.post(TELEMETRY_URL, json.encode(payload))
events = {}
last_flush = os.time()
end
```
---
## Storage (SQLite)
### Telemetry Database Schema
```sql
-- telemetry.db (separate from portal.db)
-- Raw events (7-day retention)
CREATE TABLE events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
app_id TEXT NOT NULL,
device_id TEXT NOT NULL, -- SHA256 hashed
session_id TEXT,
event_type TEXT NOT NULL,
event_data TEXT, -- JSON string
app_version TEXT,
mosis_version TEXT,
timestamp TEXT NOT NULL -- ISO8601
);
CREATE INDEX idx_events_app_time ON events(app_id, timestamp);
CREATE INDEX idx_events_type ON events(event_type, timestamp);
-- Hourly aggregates (computed by background job)
CREATE TABLE hourly_stats (
app_id TEXT NOT NULL,
hour TEXT NOT NULL, -- YYYY-MM-DDTHH
event_type TEXT NOT NULL,
count INTEGER NOT NULL,
unique_devices INTEGER NOT NULL,
PRIMARY KEY (app_id, hour, event_type)
);
-- Daily aggregates (computed from hourly)
CREATE TABLE daily_stats (
app_id TEXT NOT NULL,
date TEXT NOT NULL, -- YYYY-MM-DD
event_type TEXT NOT NULL,
count INTEGER NOT NULL,
unique_devices INTEGER NOT NULL,
PRIMARY KEY (app_id, date, event_type)
);
-- Crash groups (deduplicated by fingerprint)
CREATE TABLE crash_groups (
id TEXT PRIMARY KEY,
app_id TEXT NOT NULL,
fingerprint TEXT NOT NULL,
crash_type TEXT NOT NULL,
message TEXT,
sample_stack_trace TEXT,
first_seen TEXT NOT NULL,
last_seen TEXT NOT NULL,
occurrence_count INTEGER DEFAULT 1,
affected_versions TEXT, -- JSON array
status TEXT DEFAULT 'open',
UNIQUE(app_id, fingerprint)
);
CREATE INDEX idx_crashes_app ON crash_groups(app_id, status);
```
### Go Background Workers
```go
// Start background workers
func (s *TelemetryService) StartWorkers(ctx context.Context) {
// Hourly aggregation
go s.runPeriodic(ctx, time.Hour, s.aggregateHourly)
// Daily aggregation (run at 2am)
go s.runDaily(ctx, 2, s.aggregateDaily)
// Cleanup old events (run at 3am)
go s.runDaily(ctx, 3, s.cleanupOldEvents)
}
func (s *TelemetryService) aggregateHourly(ctx context.Context) error {
hour := time.Now().Add(-time.Hour).Format("2006-01-02T15")
_, err := s.db.ExecContext(ctx, `
INSERT OR REPLACE INTO hourly_stats (app_id, hour, event_type, count, unique_devices)
SELECT
app_id,
strftime('%Y-%m-%dT%H', timestamp) as hour,
event_type,
COUNT(*) as count,
COUNT(DISTINCT device_id) as unique_devices
FROM events
WHERE strftime('%Y-%m-%dT%H', timestamp) = ?
GROUP BY app_id, hour, event_type
`, hour)
return err
}
func (s *TelemetryService) cleanupOldEvents(ctx context.Context) error {
cutoff := time.Now().AddDate(0, 0, -7).Format(time.RFC3339)
_, err := s.db.ExecContext(ctx,
"DELETE FROM events WHERE timestamp < ?", cutoff)
return err
}
```
---
## Aggregation
### Pre-computed Metrics
| Metric | Granularity | Retention |
|--------|-------------|-----------|
| Daily active users | Day | 2 years |
| Event counts | Day | 1 year |
| Crash counts | Day | 1 year |
| Session duration | Day | 90 days |
| Performance percentiles | Day | 90 days |
### Aggregation Queries
```sql
-- Daily active users
SELECT
DATE_TRUNC('day', time) as day,
COUNT(DISTINCT device_id) as dau
FROM telemetry_events
WHERE app_id = $1
AND event_type = 'app_start'
AND time > NOW() - INTERVAL '30 days'
GROUP BY day
ORDER BY day;
-- Crash rate by version
SELECT
app_version,
COUNT(*) FILTER (WHERE event_type = 'app_crash') as crashes,
COUNT(*) FILTER (WHERE event_type = 'app_start') as starts,
ROUND(
100.0 * COUNT(*) FILTER (WHERE event_type = 'app_crash') /
NULLIF(COUNT(*) FILTER (WHERE event_type = 'app_start'), 0),
2
) as crash_rate
FROM telemetry_events
WHERE app_id = $1
AND time > NOW() - INTERVAL '7 days'
GROUP BY app_version;
```
---
## Crash Grouping
### Stack Trace Fingerprinting
```go
func fingerprintCrash(crash CrashReport) string {
// Normalize stack trace
normalized := normalizeStackTrace(crash.StackTrace)
// Hash key components
key := fmt.Sprintf("%s:%s:%s",
crash.CrashType,
crash.Message,
normalized,
)
return sha256(key)[:16]
}
func normalizeStackTrace(stack string) string {
// Remove line numbers (they change with code updates)
// Remove memory addresses
// Keep function names and file names
re := regexp.MustCompile(`:\d+:`)
return re.ReplaceAllString(stack, ":?:")
}
```
### Crash Groups Table
```sql
CREATE TABLE crash_groups (
id UUID PRIMARY KEY,
app_id TEXT NOT NULL,
fingerprint TEXT NOT NULL,
crash_type TEXT NOT NULL,
message TEXT,
sample_stack_trace TEXT,
first_seen TIMESTAMPTZ NOT NULL,
last_seen TIMESTAMPTZ NOT NULL,
occurrence_count INT DEFAULT 1,
affected_versions TEXT[],
status TEXT DEFAULT 'open', -- open, resolved, ignored
UNIQUE(app_id, fingerprint)
);
```
---
## Developer Dashboard
### Metrics View
```
┌─────────────────────────────────────────────────────────────┐
│ Analytics - My Calculator │
├─────────────────────────────────────────────────────────────┤
│ │
│ Date Range: [Last 30 days ▼] │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Daily Users │ │ Crashes │ │ Crash-free │ │
│ │ 1,234 │ │ 23 │ │ 98.1% │ │
│ │ ▲ +12% │ │ ▼ -45% │ │ ▲ +2% │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────┐ │
│ │ Daily Active Users │ │
│ │ [Line chart showing DAU over time] │ │
│ └────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────┐ │
│ │ Version Distribution │ │
│ │ [Pie chart: v1.2.0: 60%, v1.1.0: 30%, v1.0.0: 10%]│ │
│ └────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
### Crashes View
```
┌─────────────────────────────────────────────────────────────┐
│ Crashes - My Calculator │
├─────────────────────────────────────────────────────────────┤
│ │
│ Filter: [All versions ▼] [Open ▼] │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ ● attempt to index nil value 'user' │ │
│ │ lua_error • 156 occurrences • v1.2.0 │ │
│ │ First: Jan 10 • Last: Jan 15 │ │
│ │ [View] │ │
│ ├──────────────────────────────────────────────────────┤ │
│ │ ● memory limit exceeded │ │
│ │ sandbox_error • 23 occurrences • v1.1.0, v1.2.0 │ │
│ │ First: Jan 5 • Last: Jan 14 │ │
│ │ [View] │ │
│ └──────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## API Endpoints
```yaml
# Ingestion (from devices)
POST /v1/telemetry/events:
auth: device_token or api_key
body: { app_id, device_id, events[] }
response: { received: number }
POST /v1/telemetry/crash:
auth: device_token or api_key
body: { app_id, device_id, crash }
response: { id: string }
# Dashboard (for developers)
GET /v1/apps/:id/analytics/overview:
auth: required
query: { start_date, end_date }
response: { dau, crashes, crash_free_rate, ... }
GET /v1/apps/:id/analytics/events:
auth: required
query: { start_date, end_date, event_type }
response: { data: [{ date, count, unique_devices }] }
GET /v1/apps/:id/crashes:
auth: required
query: { version, status, page, limit }
response: { crashes: CrashGroup[], total }
GET /v1/apps/:id/crashes/:fingerprint:
auth: required
response: { crash_group, recent_occurrences[] }
PATCH /v1/apps/:id/crashes/:fingerprint:
auth: required
body: { status: 'resolved' | 'ignored' }
response: { crash_group }
```
---
## Data Retention
| Data Type | Retention | Reason |
|-----------|-----------|--------|
| Raw events | 7 days | Debugging |
| Daily aggregates | 2 years | Trends |
| Crash reports | 90 days | Investigation |
| Crash groups | Forever | Issue tracking |
### Cleanup Job
```sql
-- Run daily
DELETE FROM telemetry_events
WHERE time < NOW() - INTERVAL '7 days';
DELETE FROM crash_reports
WHERE timestamp < NOW() - INTERVAL '90 days';
```
---
## Privacy Controls
### User Settings
```
Settings > Privacy > Analytics
├── [✓] Send crash reports (helps developers fix bugs)
├── [ ] Send usage analytics (how you use apps)
└── [Request Data Deletion]
```
### GDPR Endpoints
```yaml
# User requests their data
GET /v1/privacy/export:
auth: user_token
response: { download_url } # JSON export of all data
# User requests deletion
DELETE /v1/privacy/data:
auth: user_token
response: { status: 'scheduled' } # Delete within 30 days
```
---
## Deliverables
- [x] Storage approach decided (SQLite with separate telemetry.db)
- [ ] Event schema specification
- [ ] Client-side batching (Lua TelemetryManager)
- [ ] Ingestion API endpoints (Go + Chi)
- [ ] SQLite schema and migrations
- [ ] Background aggregation workers (Go goroutines)
- [ ] Crash grouping logic
- [ ] Developer analytics dashboard (htmx)
- [ ] Privacy controls (opt-out in manifest)
- [ ] Data retention cleanup job
- [ ] GDPR export/delete endpoints
---
## Open Questions
1. Real-time crash alerts? → Consider email notifications for v1.1
2. ~~Sampling for high-volume apps?~~ → Not needed for self-hosted scale
3. ~~Custom events API for developers?~~ → Yes, via manifest opt-in
4. ~~Benchmarks/comparisons with similar apps?~~ → Defer to post-MVP
---
## References
- [GDPR Requirements](https://gdpr.eu/)
- [TimescaleDB Best Practices](https://docs.timescale.com/timescaledb/latest/)
- [Sentry Crash Grouping](https://docs.sentry.io/product/data-management-settings/event-grouping/)

526
docs/DEV_PORTAL_M09_REVIEW.md Normal file
View File

@@ -0,0 +1,526 @@
# Milestone 9: App Review System
**Status**: Decided
**Goal**: Automated and manual review process for app submissions.
## Decision
**Go validation workers + SQLite** for self-hosted review pipeline:
```
Validation: Go workers with concurrent file processing
Storage: SQLite (review state in portal.db)
Queue: In-memory channel + SQLite persistence
UI: htmx server-rendered pages (admin section)
```
### Rationale
1. **Go concurrency** - Process multiple files in parallel with goroutines
2. **Single binary** - No separate queue service needed
3. **Simple state** - Review state in SQLite alongside app data
4. **htmx admin UI** - Server-rendered review queue, no SPA needed
### Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ mosis-portal container │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Go Binary │ │
│ │ ┌─────────────┐ ┌────────────────┐ │ │
│ │ │ Upload API │───►│ Review Service │ │ │
│ │ │ POST /v1/ │ │ - Queue submit │ │ │
│ │ │ versions │ │ - Track state │ │ │
│ │ └─────────────┘ └───────┬────────┘ │ │
│ │ │ │ │
│ │ ┌─────────────────────────▼────────────────────────────┐ │ │
│ │ │ Validation Worker Pool │ │ │
│ │ │ • Tier 1: Package validation (ZIP, manifest, sig) │ │ │
│ │ │ • Tier 2: Content validation (RML, RCSS, Lua) │ │ │
│ │ │ • Tier 3: Security analysis (patterns, perms) │ │ │
│ │ │ • Tier 4: Quality checks (description, icons) │ │ │
│ │ └───────────────────────────────────────────────────────┘ │ │
│ │ │ │ │
│ │ ┌─────────────────────────▼────────────────────────────┐ │ │
│ │ │ Admin Review UI (htmx) │ │ │
│ │ │ • /admin/review-queue │ │ │
│ │ │ • /admin/review/:id │ │ │
│ │ └───────────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────┬─────────────────────────────┘ │
│ │ │
│ /volume1/mosis/ │ │
│ ├── data/portal.db ◄───────────┘ │
│ └── packages/{dev}/{app}/{ver}/ (validation target) │
└─────────────────────────────────────────────────────────────────┘
```
---
## Overview
The review system ensures apps meet quality and security standards before publication. Balances automation with manual review for edge cases.
---
## Review Pipeline
```
┌─────────┐ ┌───────────┐ ┌─────────┐ ┌──────────┐ ┌───────────┐
│ Submit │──►│ Automated │──►│ Manual │──►│ Approved │──►│ Published │
│ │ │ Checks │ │ Review │ │ │ │ │
└─────────┘ └───────────┘ └─────────┘ └──────────┘ └───────────┘
│ │
▼ ▼
┌─────────┐ ┌─────────┐
│ Failed │ │Rejected │
│(auto-fix)│ │(feedback)│
└─────────┘ └─────────┘
```
---
## Automated Checks
### Tier 1: Package Validation (Blocking)
| Check | Description | Action on Fail |
|-------|-------------|----------------|
| Valid ZIP | Package is valid ZIP format | Reject |
| Manifest exists | manifest.json at root | Reject |
| Manifest valid | JSON parses, required fields | Reject |
| Signature valid | Package signed with registered key | Reject |
| Entry exists | Entry point file exists | Reject |
| Size limits | Under max package size | Reject |
| No forbidden files | No .exe, .dll, etc. | Reject |
### Tier 2: Content Validation (Blocking)
| Check | Description | Action on Fail |
|-------|-------------|----------------|
| RML valid | All RML files parse | Reject |
| RCSS valid | All RCSS files parse | Reject |
| Lua syntax | All Lua files parse | Reject |
| Icons valid | Icons are valid images | Reject |
| Icon sizes | Required icon sizes present | Reject |
| Path safety | No path traversal attempts | Reject |
### Tier 3: Security Analysis (Warning/Flag)
| Check | Description | Action on Fail |
|-------|-------------|----------------|
| Dangerous patterns | Known malicious Lua patterns | Flag for review |
| Excessive permissions | Unusual permission combos | Flag for review |
| Obfuscated code | Heavily obfuscated Lua | Flag for review |
| External URLs | Hardcoded external URLs | Flag for review |
| Large assets | Unusually large files | Warning |
### Tier 4: Quality Checks (Warning)
| Check | Description | Action on Fail |
|-------|-------------|----------------|
| Description length | Meaningful description | Warning |
| Release notes | Non-empty release notes | Warning |
| Icon quality | Not placeholder/blank | Warning |
| Localization | Locale files complete | Warning |
---
## Implementation
### Validation Worker
```go
type ValidationResult struct {
Passed bool
Errors []ValidationError
Warnings []ValidationWarning
Flags []ReviewFlag
}
type ValidationError struct {
Code string
Message string
File string
Line int
}
func ValidatePackage(packagePath string) ValidationResult {
result := ValidationResult{Passed: true}
// Tier 1: Package validation
if err := validateZip(packagePath); err != nil {
result.AddError("INVALID_ZIP", err.Error())
return result
}
manifest, err := extractManifest(packagePath)
if err != nil {
result.AddError("INVALID_MANIFEST", err.Error())
return result
}
if err := validateSignature(packagePath, manifest); err != nil {
result.AddError("INVALID_SIGNATURE", err.Error())
return result
}
// Tier 2: Content validation
files, _ := listFiles(packagePath)
for _, file := range files {
switch filepath.Ext(file) {
case ".rml":
if err := validateRML(file); err != nil {
result.AddError("INVALID_RML", err.Error(), file)
}
case ".rcss":
if err := validateRCSS(file); err != nil {
result.AddError("INVALID_RCSS", err.Error(), file)
}
case ".lua":
if err := validateLua(file); err != nil {
result.AddError("INVALID_LUA", err.Error(), file)
}
if flags := analyzeLuaSecurity(file); len(flags) > 0 {
result.Flags = append(result.Flags, flags...)
}
}
}
// Tier 3: Security analysis
if hasDangerousPatterns(files) {
result.AddFlag("DANGEROUS_PATTERNS", "Code contains suspicious patterns")
}
// Tier 4: Quality checks
if len(manifest.Description) < 10 {
result.AddWarning("SHORT_DESCRIPTION", "Description is very short")
}
result.Passed = len(result.Errors) == 0
return result
}
```
### Dangerous Pattern Detection
```go
var dangerousPatterns = []struct {
Pattern *regexp.Regexp
Reason string
}{
{
regexp.MustCompile(`loadstring\s*\(`),
"Dynamic code execution",
},
{
regexp.MustCompile(`debug\s*\.\s*\w+`),
"Debug library usage",
},
{
regexp.MustCompile(`os\s*\.\s*execute`),
"OS command execution",
},
{
regexp.MustCompile(`io\s*\.\s*\w+`),
"Direct I/O operations",
},
{
regexp.MustCompile(`ffi\s*\.\s*\w+`),
"FFI usage",
},
{
regexp.MustCompile(`package\s*\.\s*loadlib`),
"Native library loading",
},
}
func analyzeLuaSecurity(content string) []ReviewFlag {
var flags []ReviewFlag
for _, dp := range dangerousPatterns {
if dp.Pattern.MatchString(content) {
flags = append(flags, ReviewFlag{
Type: "SECURITY",
Reason: dp.Reason,
})
}
}
return flags
}
```
---
## Manual Review
### When Required
| Trigger | Reason |
|---------|--------|
| New developer | First app submission |
| Dangerous permissions | camera, microphone, contacts, location |
| Security flags | Automated checks flagged concerns |
| User reports | Existing app reported |
| Appeal | Developer contests rejection |
### Review Queue UI
```
┌─────────────────────────────────────────────────────────────┐
│ Review Queue [14 pending] │
├─────────────────────────────────────────────────────────────┤
│ │
│ Filter: [All ▼] [Flagged first ▼] │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ 🚩 Weather Pro v2.0.0 │ │
│ │ com.newdev.weather • New developer │ │
│ │ Permissions: location, network │ │
│ │ Flags: First submission │ │
│ │ Submitted: 2 hours ago │ │
│ │ [Review] [Auto-approve]│ │
│ ├──────────────────────────────────────────────────────┤ │
│ │ 🚩 Photo Editor v1.5.0 │ │
│ │ com.trusted.photos • Verified developer │ │
│ │ Permissions: camera, storage │ │
│ │ Flags: DANGEROUS_PATTERNS (1) │ │
│ │ Submitted: 5 hours ago │ │
│ │ [Review] [Auto-approve]│ │
│ └──────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
### Review Detail View
```
┌─────────────────────────────────────────────────────────────┐
│ Review: Weather Pro v2.0.0 │
├─────────────────────────────────────────────────────────────┤
│ │
│ Package: com.newdev.weather │
│ Developer: newdev@example.com (New - first app) │
│ Submitted: Jan 15, 2024 10:30 AM │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Automated Checks │ │
│ │ ✓ Package validation passed │ │
│ │ ✓ Content validation passed │ │
│ │ ⚠ Warning: SHORT_DESCRIPTION │ │
│ │ 🚩 Flag: First submission from new developer │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ Permissions Requested: │
│ • location (fine) - Used for weather forecasts │
│ • network - Required for API calls │
│ │
│ Files: [Expand to browse] │
│ ├── manifest.json │
│ ├── assets/ │
│ │ ├── main.rml │
│ │ └── scripts/ │
│ │ └── weather.lua [View source] │
│ └── icons/ │
│ │
│ Reviewer Notes: │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ [Approve] [Reject with feedback] [Request changes] │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## Review States
```
┌─────────┐
│ Draft │
└────┬────┘
│ submit
▼
┌─────────┐
┌───────│Uploaded │
│ └────┬────┘
│ │ validation
│ ▼
│ ┌──────────┐
failed │ │Validating│
│ └────┬─────┘
│ │
│ ┌───────┴───────┐
│ │ │
▼ ▼ ▼
┌────────┐ ┌─────────┐
│ Failed │ │In Review│
└────────┘ └────┬────┘
│
┌───────────┼───────────┐
│ │ │
▼ ▼ ▼
┌──────────┐ ┌────────┐ ┌──────────┐
│ Approved │ │Rejected│ │ Changes │
└────┬─────┘ └────────┘ │ Requested│
│ └──────────┘
│ publish
▼
┌──────────┐
│Published │
└──────────┘
```
---
## Rejection Feedback
### Feedback Template
```json
{
"reason": "SECURITY_CONCERN",
"message": "Your app contains patterns that raise security concerns.",
"details": [
{
"file": "scripts/main.lua",
"line": 42,
"issue": "Usage of loadstring() is not allowed",
"suggestion": "Use static Lua code instead of dynamic evaluation"
}
],
"can_resubmit": true,
"appeal_available": true
}
```
### Common Rejection Reasons
| Code | Description |
|------|-------------|
| `SECURITY_CONCERN` | Security issues found |
| `QUALITY_ISSUE` | Doesn't meet quality standards |
| `POLICY_VIOLATION` | Violates content policy |
| `METADATA_ISSUE` | Incorrect/misleading metadata |
| `PERMISSION_ABUSE` | Unnecessary permissions |
| `COPYRIGHT` | Copyright/trademark issues |
---
## Appeal Process
```
1. Developer receives rejection
2. Developer clicks "Appeal" (within 14 days)
3. Provides justification
4. Different reviewer examines
5. Final decision (approve/uphold rejection)
```
---
## Review SLA
| Submission Type | Target Time |
|-----------------|-------------|
| Auto-approved | Instant |
| New developer | 24 hours |
| Flagged | 48 hours |
| Appeal | 72 hours |
---
## API Endpoints
```yaml
# Developer endpoints
POST /v1/apps/:id/versions/:vid/submit:
summary: Submit for review
response: { version, estimated_review_time }
GET /v1/apps/:id/versions/:vid/review-status:
summary: Get review status
response: { status, feedback?, estimated_completion }
POST /v1/apps/:id/versions/:vid/appeal:
summary: Appeal rejection
body: { justification }
response: { appeal_id, status }
# Internal review endpoints (admin only)
GET /v1/admin/review-queue:
summary: Get pending reviews
response: { items[], total }
GET /v1/admin/review/:version_id:
summary: Get review details
response: { version, validation_result, flags }
POST /v1/admin/review/:version_id/approve:
summary: Approve version
body: { notes? }
response: { version }
POST /v1/admin/review/:version_id/reject:
summary: Reject version
body: { reason, message, details[] }
response: { version }
```
---
## Metrics
### Review Performance
| Metric | Target |
|--------|--------|
| Auto-approval rate | > 70% |
| Average review time | < 24 hours |
| Rejection rate | < 20% |
| Appeal overturn rate | < 10% |
### Dashboard
```sql
-- Review stats
SELECT
DATE_TRUNC('week', submitted_at) as week,
COUNT(*) as submissions,
COUNT(*) FILTER (WHERE status = 'published') as approved,
COUNT(*) FILTER (WHERE status = 'rejected') as rejected,
AVG(EXTRACT(EPOCH FROM (reviewed_at - submitted_at))/3600) as avg_hours
FROM app_versions
WHERE submitted_at > NOW() - INTERVAL '30 days'
GROUP BY week;
```
---
## Deliverables
- [x] Review approach decided (Go workers + SQLite + htmx admin)
- [ ] Validation worker implementation (Go concurrent file processing)
- [ ] Dangerous pattern database (regex patterns in code)
- [ ] Review queue UI (htmx server-rendered)
- [ ] Reviewer tools (file browser, source viewer)
- [ ] Rejection feedback system
- [ ] Appeal workflow
- [ ] Review metrics queries
---
## Open Questions
1. ~~Automated approval for trusted developers?~~ → Yes, after 3+ approved apps
2. ~~Community moderators?~~ → Defer to post-MVP (single admin for now)
3. Content policy document? → Create during M12 Docs
4. ~~Rate limiting resubmissions?~~ → Max 3 resubmits per day per app
---
## References
- [Apple App Review Guidelines](https://developer.apple.com/app-store/review/guidelines/)
- [Google Play Policy](https://play.google.com/about/developer-content-policy/)

639
docs/DEV_PORTAL_M10_DEVICE.md Normal file
View File

@@ -0,0 +1,639 @@
# Milestone 10: Device-Side App Management
**Status**: Decided
**Goal**: Install, update, and manage apps on Mosis devices.
## Decision
**C++ AppManager + Lua bindings** running on MosisService:
```
AppManager: C++ class managing installation/updates
Storage: Local device storage (/data/mosis/apps/)
Updates: Background service checking Portal API
UI: App Store system app (RML/Lua)
API: Connects to Portal at portal.mosis.dev (or self-hosted)
```
### Rationale
1. **Native C++** - AppManager runs in MosisService process for performance
2. **Background updates** - UpdateService thread checks Portal API periodically
3. **System app** - App Store is a privileged RML/Lua app with special permissions
4. **Ed25519 verification** - All packages verified before installation
### API Integration
```
Device Portal (Synology NAS)
┌──────────────┐ ┌──────────────────────┐
│ MosisService │ │ mosis-portal │
│ │ │ │
│ UpdateService├──────GET /store/apps────►│ Chi API Router │
│ │ /updates?pkgs=... │ │
│ │◄─────{updates: [...]}───┤ SQLite portal.db │
│ │ │ │
│ AppManager ├──────GET /packages/...──►│ /volume1/mosis/ │
│ │◄─────[package.mosis]────┤ packages/{dev}/... │
└──────────────┘ └──────────────────────┘
```
---
## Overview
Device-side app management handles the full lifecycle of apps on user devices: discovery, installation, updates, and removal.
---
## Components
```
┌─────────────────────────────────────────────────────────────┐
│ MosisService │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ AppManager │ │ UpdateService │ │ AppStore UI │ │
│ │ (C++ class) │ │ (Background) │ │ (System App) │ │
│ └───────────────┘ └───────────────┘ └───────────────┘ │
│ │ │ │ │
│ └───────────────────┼───────────────────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ │ LuaSandboxManager│ │
│ └─────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
```
---
## Storage Layout
```
/data/mosis/
├── config/
│ ├── device.json # Device ID, settings
│ └── apps.json # Installed apps registry
├── apps/
│ └── {package_id}/
│ ├── package/ # Extracted app files
│ │ ├── manifest.json
│ │ └── assets/
│ ├── data/ # App data (VirtualFS)
│ ├── cache/ # App cache
│ └── db/ # SQLite databases
├── downloads/ # Temp download location
└── backups/ # App data backups (before update)
```
---
## AppManager Class
### Interface
```cpp
namespace mosis {
struct InstalledApp {
std::string package_id;
std::string name;
std::string version_name;
int version_code;
std::string install_path;
std::vector<std::string> permissions;
std::chrono::system_clock::time_point installed_at;
std::chrono::system_clock::time_point updated_at;
int64_t package_size;
int64_t data_size;
};
struct InstallProgress {
enum class Stage {
Downloading,
Verifying,
Extracting,
Registering,
Complete,
Failed
};
Stage stage;
float progress; // 0.0 - 1.0
std::string error;
};
using ProgressCallback = std::function<void(const InstallProgress&)>;
class AppManager {
public:
explicit AppManager(const std::string& data_root);
~AppManager();
// Installation
bool Install(const std::string& package_url,
const std::string& signature,
ProgressCallback callback);
bool InstallFromFile(const std::string& package_path,
ProgressCallback callback);
// Uninstallation
bool Uninstall(const std::string& package_id, bool keep_data = false);
// Updates
bool Update(const std::string& package_id,
const std::string& package_url,
const std::string& signature,
ProgressCallback callback);
// Queries
std::vector<InstalledApp> GetInstalledApps() const;
std::optional<InstalledApp> GetApp(const std::string& package_id) const;
bool IsInstalled(const std::string& package_id) const;
// Data management
int64_t GetAppDataSize(const std::string& package_id) const;
bool ClearAppData(const std::string& package_id);
bool ClearAppCache(const std::string& package_id);
bool BackupAppData(const std::string& package_id);
bool RestoreAppData(const std::string& package_id);
// Integration with sandbox
void SetSandboxManager(LuaSandboxManager* manager);
private:
std::string m_data_root;
LuaSandboxManager* m_sandbox_manager = nullptr;
mutable std::mutex m_mutex;
std::map<std::string, InstalledApp> m_installed_apps;
bool VerifyPackage(const std::string& path, const std::string& signature);
bool ExtractPackage(const std::string& path, const std::string& dest);
void LoadInstalledApps();
void SaveInstalledApps();
};
} // namespace mosis
```
### Implementation
```cpp
bool AppManager::Install(const std::string& package_url,
const std::string& signature,
ProgressCallback callback) {
callback({InstallProgress::Stage::Downloading, 0.0f, ""});
// Download package
std::string download_path = m_data_root + "/downloads/" + GenerateUUID();
if (!DownloadFile(package_url, download_path, [&](float p) {
callback({InstallProgress::Stage::Downloading, p, ""});
})) {
callback({InstallProgress::Stage::Failed, 0.0f, "Download failed"});
return false;
}
callback({InstallProgress::Stage::Verifying, 0.0f, ""});
// Verify signature
if (!VerifyPackage(download_path, signature)) {
std::filesystem::remove(download_path);
callback({InstallProgress::Stage::Failed, 0.0f, "Signature verification failed"});
return false;
}
// Extract manifest to get package_id
auto manifest = ExtractManifest(download_path);
if (!manifest) {
std::filesystem::remove(download_path);
callback({InstallProgress::Stage::Failed, 0.0f, "Invalid manifest"});
return false;
}
callback({InstallProgress::Stage::Extracting, 0.0f, ""});
// Check if already installed (update path)
std::string install_path = m_data_root + "/apps/" + manifest->package_id;
if (std::filesystem::exists(install_path + "/package")) {
// Backup existing data
BackupAppData(manifest->package_id);
// Remove old package
std::filesystem::remove_all(install_path + "/package");
}
// Extract package
std::filesystem::create_directories(install_path + "/package");
if (!ExtractPackage(download_path, install_path + "/package")) {
callback({InstallProgress::Stage::Failed, 0.0f, "Extraction failed"});
return false;
}
// Clean up download
std::filesystem::remove(download_path);
callback({InstallProgress::Stage::Registering, 0.0f, ""});
// Create data directories
std::filesystem::create_directories(install_path + "/data");
std::filesystem::create_directories(install_path + "/cache");
std::filesystem::create_directories(install_path + "/db");
// Register app
InstalledApp app{
.package_id = manifest->package_id,
.name = manifest->name,
.version_name = manifest->version,
.version_code = manifest->version_code,
.install_path = install_path,
.permissions = manifest->permissions,
.installed_at = std::chrono::system_clock::now(),
.updated_at = std::chrono::system_clock::now(),
.package_size = std::filesystem::file_size(download_path)
};
{
std::lock_guard<std::mutex> lock(m_mutex);
m_installed_apps[manifest->package_id] = app;
SaveInstalledApps();
}
callback({InstallProgress::Stage::Complete, 1.0f, ""});
return true;
}
bool AppManager::Uninstall(const std::string& package_id, bool keep_data) {
std::lock_guard<std::mutex> lock(m_mutex);
auto it = m_installed_apps.find(package_id);
if (it == m_installed_apps.end()) {
return false;
}
// Stop app if running
if (m_sandbox_manager && m_sandbox_manager->IsAppRunning(package_id)) {
m_sandbox_manager->StopApp(package_id);
}
// Remove files
std::string install_path = it->second.install_path;
std::filesystem::remove_all(install_path + "/package");
std::filesystem::remove_all(install_path + "/cache");
if (!keep_data) {
std::filesystem::remove_all(install_path + "/data");
std::filesystem::remove_all(install_path + "/db");
std::filesystem::remove_all(install_path);
}
// Unregister
m_installed_apps.erase(it);
SaveInstalledApps();
return true;
}
```
---
## Update Service
### Background Update Checker
```cpp
class UpdateService {
public:
UpdateService(AppManager* app_manager, const std::string& api_base);
// Start background checking
void Start(std::chrono::hours interval = std::chrono::hours(24));
void Stop();
// Manual check
std::vector<UpdateInfo> CheckForUpdates();
// Download and install update
bool ApplyUpdate(const std::string& package_id, ProgressCallback callback);
// Settings
void SetAutoUpdate(bool enabled);
void SetWifiOnly(bool wifi_only);
private:
void CheckLoop();
AppManager* m_app_manager;
std::string m_api_base;
std::thread m_check_thread;
std::atomic<bool> m_running{false};
bool m_auto_update = false;
bool m_wifi_only = true;
};
```
### Update Check Flow
```
1. Get list of installed apps
2. Call API: GET /store/apps/updates?packages=com.a,com.b,com.c
3. API returns list of available updates
4. If auto-update enabled and on WiFi:
- Download and install in background
- Notify user of completed updates
5. If manual:
- Show notification with update count
- User opens App Store to review
```
---
## App Store System App
### UI Screens
```
Home
├── Featured apps
├── Categories
├── Search
└── My Apps
├── Installed
├── Updates available
└── Previously installed
App Detail
├── Icon, name, developer
├── Screenshots
├── Description
├── Permissions
├── Reviews (future)
└── [Install] / [Update] / [Open]
Settings
├── Auto-update (WiFi only)
├── Storage usage
└── Clear all caches
```
### RML Structure
```xml
<!-- app_store/main.rml -->
<rml>
<head>
<link type="text/rcss" href="../../ui/theme.rcss"/>
<link type="text/rcss" href="store.rcss"/>
<script src="store.lua"/>
</head>
<body>
<div class="app-bar">
<h1>App Store</h1>
<div class="search-icon" onclick="showSearch()"/>
</div>
<div id="content">
<!-- Dynamic content loaded by Lua -->
</div>
<div class="bottom-nav">
<div class="nav-item active" onclick="showHome()">
<img src="icons/home.tga"/>
<span>Home</span>
</div>
<div class="nav-item" onclick="showCategories()">
<img src="icons/category.tga"/>
<span>Categories</span>
</div>
<div class="nav-item" onclick="showMyApps()">
<img src="icons/apps.tga"/>
<span>My Apps</span>
</div>
</div>
</body>
</rml>
```
### Lua Store Logic
```lua
-- store.lua
local api = require("store_api")
local ui = require("ui")
local state = {
screen = "home",
featured = {},
categories = {},
installed = {},
updates = {}
}
function init()
-- Load installed apps
state.installed = mosis.apps.getInstalled()
-- Fetch featured apps
api.getFeatured(function(apps)
state.featured = apps
render()
end)
-- Check for updates
checkUpdates()
end
function checkUpdates()
local package_ids = {}
for _, app in ipairs(state.installed) do
table.insert(package_ids, app.package_id)
end
api.checkUpdates(package_ids, function(updates)
state.updates = updates
render()
end)
end
function installApp(package_id)
local app = findApp(package_id)
if not app then return end
showProgress(app.name)
mosis.apps.install(app.download_url, app.signature, function(progress)
updateProgress(progress.stage, progress.progress)
if progress.stage == "complete" then
hideProgress()
showToast(app.name .. " installed")
state.installed = mosis.apps.getInstalled()
render()
elseif progress.stage == "failed" then
hideProgress()
showError("Installation failed: " .. progress.error)
end
end)
end
function openApp(package_id)
mosis.apps.launch(package_id)
end
function uninstallApp(package_id)
showConfirm("Uninstall " .. getAppName(package_id) .. "?", function(confirmed)
if confirmed then
mosis.apps.uninstall(package_id)
state.installed = mosis.apps.getInstalled()
render()
end
end)
end
```
---
## Lua API for Apps
### Exposed to System Apps
```lua
-- mosis.apps namespace (system apps only)
-- Get installed apps
local apps = mosis.apps.getInstalled()
-- Returns: [{package_id, name, version_name, version_code, installed_at}]
-- Install from store
mosis.apps.install(url, signature, callback)
-- callback(progress): {stage, progress, error}
-- Uninstall
mosis.apps.uninstall(package_id)
-- Launch app
mosis.apps.launch(package_id)
-- Get app info
local info = mosis.apps.getInfo(package_id)
-- Storage management
local size = mosis.apps.getDataSize(package_id)
mosis.apps.clearCache(package_id)
mosis.apps.clearData(package_id)
```
### Exposed to All Apps
```lua
-- mosis.app namespace (current app only)
-- Get own package info
local info = mosis.app.info()
-- Returns: {package_id, name, version_name, version_code}
-- Check for update
mosis.app.checkUpdate(function(available, new_version)
if available then
showUpdatePrompt(new_version)
end
end)
-- Open store page for self
mosis.app.openStorePage()
```
---
## Permissions for App Management
```lua
-- Required permission to use mosis.apps.*
permissions = {"system.app_management"}
-- Only granted to:
-- - App Store system app
-- - Settings system app
-- - Other OEM system apps
```
---
## Installation Intents
### From Deep Links
```
mosis://store/app/com.developer.myapp
mosis://store/install?url=...&sig=...
```
### From App Store
```lua
-- User taps Install button
installApp("com.developer.myapp")
```
### From ADB (Development)
```bash
adb shell am broadcast -a com.omixlab.mosis.INSTALL_APP \
--es package_path "/sdcard/myapp.mosis"
```
---
## Security
### Package Verification
1. Verify ZIP integrity
2. Verify Ed25519 signature
3. Verify signer is registered developer
4. Verify app not in blocklist
5. Verify permissions are declared
### Installation Sources
| Source | Allowed |
|--------|---------|
| Official store | Always |
| Developer sideload | If enabled in settings |
| Unknown APK | Never (MosisService only) |
### Sandboxing
- All apps run in LuaSandbox
- File access limited to app's data directory
- Network access requires permission
- Hardware access requires permission + user gesture
---
## Deliverables
- [x] Architecture decided (C++ AppManager + Lua bindings)
- [ ] AppManager C++ class
- [ ] UpdateService background checker
- [ ] App Store system app (RML/Lua)
- [ ] Lua API bindings (mosis.apps, mosis.app)
- [ ] Installation progress UI
- [ ] Uninstall confirmation UI
- [ ] Storage management UI
- [ ] Deep link handling
---
## Open Questions
1. ~~App backup to cloud?~~ → Defer to post-MVP (local backups only)
2. ~~Family sharing / multiple devices?~~ → Defer to post-MVP
3. ~~Enterprise MDM integration?~~ → Not needed for self-hosted
4. ~~Sideloading policy?~~ → Enabled via Settings toggle (developer mode)
---
## References
- [Android PackageManager](https://developer.android.com/reference/android/content/pm/PackageManager)
- [iOS App Installation](https://developer.apple.com/documentation/devicemanagement/installing_apps)

722
docs/DEV_PORTAL_M11_CLI.md Normal file
View File

@@ -0,0 +1,722 @@
# Milestone 11: Developer CLI Tool
**Status**: Decided
**Goal**: Command-line tool for app development workflow.
## Decision
**Go + Cobra** for single-binary cross-platform CLI:
```
Framework: Cobra (github.com/spf13/cobra)
Distribution: Single binary (no runtime dependencies)
Signing: crypto/ed25519 (stdlib)
Auth: OAuth2 device flow + API key storage
Portal URL: Configurable (default: self-hosted Synology)
```
### Rationale
1. **Consistency** - Same language as Portal backend (Go)
2. **Single binary** - No Node.js/Python runtime needed
3. **Fast** - Compiles to native code, instant startup
4. **Cross-platform** - Build for Windows, macOS, Linux from one codebase
5. **Cobra ecosystem** - Shell completions, man pages, help generation
### Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ mosis CLI │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ Cobra Commands ││
│ │ ├── init → Template generation ││
│ │ ├── build → ZIP package creation ││
│ │ ├── sign → Ed25519 signing (crypto/ed25519) ││
│ │ ├── run → Launch mosis-designer subprocess ││
│ │ ├── login → OAuth2 device flow → ~/.mosis/credentials ││
│ │ └── publish → HTTP client → Portal API ││
│ └───────────────────────────────────────────────────────────────┘│
│ │ │
│ ~/.mosis/ │ │
│ ├── config.json │ Portal (Synology NAS) │
│ ├── credentials │ ┌──────────────────────────┐ │
│ ├── signing_key.pem ────────┼──│ POST /v1/versions │ │
│ └── signing_key.pub │ │ POST /auth/device │ │
│ │ └──────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
---
## Overview
The CLI tool (`mosis`) streamlines the developer workflow: project creation, building, testing, signing, and publishing.
---
## Commands Overview
```
mosis
├── init Create new app project
├── validate Validate manifest and assets
├── build Create .mosis package
├── sign Sign package with developer key
├── run Run in local designer/emulator
├── test Run automated tests
├── login Authenticate with portal
├── logout Clear authentication
├── publish Upload and submit for review
├── status Check review status
├── keys
│ ├── generate Generate signing keypair
│ ├── list List registered keys
│ └── register Upload public key to portal
└── config
├── get Get config value
└── set Set config value
```
---
## Command Details
### `mosis init`
Create a new app project with boilerplate.
```bash
$ mosis init
? App name: My Calculator
? Package ID: com.myname.calculator
? Description: A simple calculator app
? Author name: John Doe
? Author email: john@example.com
Creating project structure...
✓ Created manifest.json
✓ Created assets/main.rml
✓ Created assets/styles/theme.rcss
✓ Created assets/scripts/app.lua
✓ Created icons/ (placeholder icons)
Project created! Next steps:
cd my-calculator
mosis run # Preview in designer
mosis build # Create package
mosis publish # Submit to store
```
#### Generated Structure
```
my-calculator/
├── manifest.json
├── assets/
│ ├── main.rml
│ ├── styles/
│ │ └── theme.rcss
│ └── scripts/
│ └── app.lua
├── icons/
│ ├── icon-32.png
│ ├── icon-64.png
│ └── icon-128.png
└── .mosisignore # Files to exclude from package
```
---
### `mosis validate`
Validate project without building.
```bash
$ mosis validate
Validating manifest.json...
✓ Required fields present
✓ Package ID format valid
✓ Version format valid
Validating assets...
✓ Entry point exists: assets/main.rml
✓ All RML files valid (3 files)
✓ All RCSS files valid (2 files)
✓ All Lua files valid (4 files)
Validating icons...
✓ icon-32.png (32x32)
✓ icon-64.png (64x64)
✓ icon-128.png (128x128)
Checking permissions...
✓ Permissions declared: storage, network
All validations passed!
```
---
### `mosis build`
Create a .mosis package.
```bash
$ mosis build
Reading manifest.json...
Package: com.myname.calculator v1.0.0 (1)
Collecting files...
✓ manifest.json
✓ assets/main.rml
✓ assets/styles/theme.rcss
✓ assets/scripts/app.lua
✓ icons/icon-32.png
✓ icons/icon-64.png
✓ icons/icon-128.png
Creating package...
✓ Package created: dist/com.myname.calculator-1.0.0.mosis (45.2 KB)
⚠ Package is unsigned. Run 'mosis sign' before publishing.
```
#### Options
```bash
mosis build [options]
Options:
-o, --output <path> Output path (default: dist/)
--no-compress Skip compression
--include-source Include .lua source maps
```
---
### `mosis sign`
Sign a package with developer key.
```bash
$ mosis sign dist/com.myname.calculator-1.0.0.mosis
Using key: ~/.mosis/signing_key.pem
Fingerprint: SHA256:abc123...
Generating file hashes...
Signing MANIFEST.MF...
✓ Package signed: dist/com.myname.calculator-1.0.0.mosis
Signature details:
Algorithm: Ed25519
Key fingerprint: SHA256:abc123...
Files signed: 7
```
#### Options
```bash
mosis sign <package> [options]
Options:
-k, --key <path> Path to private key (default: ~/.mosis/signing_key.pem)
--verify Verify existing signature
```
---
### `mosis run`
Launch in desktop designer for testing.
```bash
$ mosis run
Starting Mosis Designer...
Loading: assets/main.rml
Designer running at http://localhost:8080
Press Ctrl+C to stop
[Hot reload enabled - changes auto-refresh]
```
#### Options
```bash
mosis run [options]
Options:
--entry <file> Override entry point
--port <number> Designer port (default: 8080)
--no-hot-reload Disable hot reload
--device <name> Emulate device (phone, tablet)
```
---
### `mosis login`
Authenticate with developer portal.
```bash
$ mosis login
Opening browser for authentication...
Waiting for authorization...
✓ Logged in as john@example.com
API key stored in ~/.mosis/credentials
```
#### Options
```bash
mosis login [options]
Options:
--api-key <key> Use API key instead of browser auth
--portal <url> Portal URL (default: https://portal.mosis.dev)
```
---
### `mosis publish`
Upload and submit for review.
```bash
$ mosis publish
Checking authentication...
✓ Logged in as john@example.com
Building package...
✓ Package created: dist/com.myname.calculator-1.0.0.mosis
Signing package...
✓ Package signed
Uploading...
████████████████████████████████ 100%
Submitting for review...
✓ Version 1.0.0 submitted
Review status: In Review
Estimated review time: 24 hours
Track status: mosis status
```
#### Options
```bash
mosis publish [options]
Options:
--package <path> Use existing package
--notes <text> Release notes
--notes-file <path> Release notes from file
--draft Upload without submitting for review
```
---
### `mosis status`
Check app/version status.
```bash
$ mosis status
App: My Calculator (com.myname.calculator)
Versions:
v1.0.0 (1) Published Jan 10, 2024 1,234 downloads
v1.1.0 (2) In Review Jan 15, 2024 Submitted 2h ago
Latest review:
Status: In Review
Submitted: Jan 15, 2024 10:30 AM
Estimated completion: Jan 16, 2024
Run 'mosis status --watch' to monitor in real-time.
```
---
### `mosis keys generate`
Generate Ed25519 signing keypair.
```bash
$ mosis keys generate
Generating Ed25519 keypair...
Private key saved to: ~/.mosis/signing_key.pem
Public key saved to: ~/.mosis/signing_key.pub
Fingerprint: SHA256:abc123def456...
⚠ IMPORTANT: Keep your private key secure!
- Never share or commit signing_key.pem
- Back it up securely
- If compromised, revoke immediately
Next step: Register your public key
mosis keys register
```
---
### `mosis keys register`
Upload public key to portal.
```bash
$ mosis keys register
Reading public key from ~/.mosis/signing_key.pub
Fingerprint: SHA256:abc123def456...
? Key name: MacBook Pro 2024
Uploading to portal...
✓ Key registered successfully
Your signing key is now active. Packages signed with this
key will be accepted for review.
```
---
## Configuration
### Config File Location
```
~/.mosis/
├── config.json # CLI configuration
├── credentials # Auth tokens (encrypted)
├── signing_key.pem # Private key
└── signing_key.pub # Public key
```
### Config Options
```json
{
"portal_url": "https://portal.mosis.dev",
"api_url": "https://api.mosis.dev",
"designer_path": "/usr/local/bin/mosis-designer",
"default_author": {
"name": "John Doe",
"email": "john@example.com"
}
}
```
---
## Implementation (Go + Cobra)
### Main Entry Point
```go
package main
import (
"fmt"
"os"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
func main() {
rootCmd := &cobra.Command{
Use: "mosis",
Short: "Mosis app development CLI",
Long: "CLI tool for building, signing, and publishing Mosis apps",
}
// Global flags
rootCmd.PersistentFlags().StringP("portal", "p", "", "Portal URL (default from config)")
rootCmd.PersistentFlags().BoolP("verbose", "v", false, "Verbose output")
// Commands
rootCmd.AddCommand(initCmd())
rootCmd.AddCommand(buildCmd())
rootCmd.AddCommand(signCmd())
rootCmd.AddCommand(runCmd())
rootCmd.AddCommand(loginCmd())
rootCmd.AddCommand(publishCmd())
rootCmd.AddCommand(statusCmd())
rootCmd.AddCommand(keysCmd())
rootCmd.AddCommand(configCmd())
if err := rootCmd.Execute(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
```
### Build Command Example
```go
func buildCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "build",
Short: "Create .mosis package",
RunE: func(cmd *cobra.Command, args []string) error {
manifest, err := readManifest("manifest.json")
if err != nil {
return fmt.Errorf("manifest.json not found\n\nAre you in a Mosis project directory?")
}
output, _ := cmd.Flags().GetString("output")
if output == "" {
output = fmt.Sprintf("dist/%s-%s.mosis", manifest.PackageID, manifest.Version)
}
fmt.Printf("Building %s v%s...\n", manifest.Name, manifest.Version)
files, err := collectFiles(manifest)
if err != nil {
return err
}
if err := createPackage(files, output); err != nil {
return err
}
info, _ := os.Stat(output)
fmt.Printf("✓ Package created: %s (%.1f KB)\n", output, float64(info.Size())/1024)
fmt.Println("\n⚠ Package is unsigned. Run 'mosis sign' before publishing.")
return nil
},
}
cmd.Flags().StringP("output", "o", "", "Output path (default: dist/)")
return cmd
}
```
### OAuth2 Device Flow (Login)
```go
func loginCmd() *cobra.Command {
return &cobra.Command{
Use: "login",
Short: "Authenticate with developer portal",
RunE: func(cmd *cobra.Command, args []string) error {
portalURL := viper.GetString("portal_url")
// Start device flow
resp, err := http.Post(portalURL+"/auth/device", "application/json", nil)
if err != nil {
return err
}
var device DeviceResponse
json.NewDecoder(resp.Body).Decode(&device)
fmt.Printf("Go to: %s\n", device.VerificationURI)
fmt.Printf("Enter code: %s\n\n", device.UserCode)
fmt.Println("Waiting for authorization...")
// Poll for token
token, err := pollForToken(portalURL, device.DeviceCode, device.Interval)
if err != nil {
return err
}
// Save credentials
if err := saveCredentials(token); err != nil {
return err
}
fmt.Printf("✓ Logged in as %s\n", token.Email)
return nil
},
}
}
```
### Ed25519 Signing
```go
func signPackage(packagePath, keyPath string) error {
// Read private key
keyPEM, err := os.ReadFile(keyPath)
if err != nil {
return fmt.Errorf("failed to read key: %w", err)
}
block, _ := pem.Decode(keyPEM)
privateKey, err := x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
return fmt.Errorf("invalid key format: %w", err)
}
ed25519Key := privateKey.(ed25519.PrivateKey)
// Generate MANIFEST.MF with file hashes
manifest, err := generateManifest(packagePath)
if err != nil {
return err
}
// Sign manifest
signature := ed25519.Sign(ed25519Key, manifest)
// Add signature to package
return addSignatureToPackage(packagePath, manifest, signature)
}
---
## Distribution
### npm (Node.js version)
```bash
npm install -g @mosis/cli
```
### Homebrew (macOS)
```bash
brew tap mosis/tap
brew install mosis
```
### Direct Download
```bash
# Linux/macOS
curl -fsSL https://mosis.dev/install.sh | sh
# Windows
irm https://mosis.dev/install.ps1 | iex
```
### Package Managers
| Platform | Package Manager | Command |
|----------|-----------------|---------|
| macOS | Homebrew | `brew install mosis` |
| Windows | Scoop | `scoop install mosis` |
| Linux | apt (deb) | `apt install mosis` |
| Any | npm | `npm install -g @mosis/cli` |
---
## Error Handling
### User-Friendly Errors
```bash
$ mosis build
Error: manifest.json not found
Are you in a Mosis project directory?
Run 'mosis init' to create a new project.
```
### Verbose Mode
```bash
$ mosis build --verbose
[DEBUG] Reading manifest from ./manifest.json
[DEBUG] Manifest parsed: {id: "com.example.app", ...}
[DEBUG] Collecting files from ./assets
[DEBUG] Found 7 files
[DEBUG] Creating ZIP archive
[DEBUG] Writing to dist/com.example.app-1.0.0.mosis
[DEBUG] Package size: 45234 bytes
✓ Package created
```
---
## CI/CD Integration
### GitHub Actions
```yaml
name: Build and Publish
on:
push:
tags:
- 'v*'
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Mosis CLI
run: npm install -g @mosis/cli
- name: Build and Sign
env:
MOSIS_SIGNING_KEY: ${{ secrets.MOSIS_SIGNING_KEY }}
run: |
echo "$MOSIS_SIGNING_KEY" > signing_key.pem
mosis build
mosis sign dist/*.mosis --key signing_key.pem
- name: Publish
env:
MOSIS_API_KEY: ${{ secrets.MOSIS_API_KEY }}
run: mosis publish --api-key "$MOSIS_API_KEY"
```
---
## Deliverables
- [x] CLI framework selected (Go + Cobra)
- [ ] `init` command (template generation)
- [ ] `validate` command
- [ ] `build` command (ZIP package creation)
- [ ] `sign` command (Ed25519 signing)
- [ ] `run` command (designer subprocess)
- [ ] `login/logout` commands (OAuth2 device flow)
- [ ] `publish` command (HTTP upload to Portal)
- [ ] `status` command
- [ ] `keys` subcommands (generate, register, list)
- [ ] Configuration management (viper)
- [ ] Cross-platform builds (goreleaser)
- [ ] CI/CD examples (GitHub Actions)
---
## Open Questions
1. ~~Should CLI auto-update itself?~~ → No, manual updates via package manager
2. ~~Offline mode for build/sign?~~ → Yes, build/sign work offline
3. ~~Plugin system for custom commands?~~ → Defer to post-MVP
4. IDE integrations (VS Code extension)? → Consider for v1.1
---
## References
- [Cobra CLI Framework](https://cobra.dev/)
- [oclif Framework](https://oclif.io/)
- [Clap for Rust](https://docs.rs/clap/)

668
docs/DEV_PORTAL_M12_DOCS.md Normal file
View File

@@ -0,0 +1,668 @@
# Milestone 12: Documentation Site
**Status**: Decided
**Goal**: Comprehensive documentation for Mosis app developers.
## Decision
**Hugo + Docsy theme** for self-hosted static documentation:
```
Framework: Hugo (Go-based static site generator)
Theme: Docsy (technical documentation theme)
Search: Pagefind (local, no external service)
Hosting: Synology NAS (nginx or Go static server)
```
### Rationale
1. **Go ecosystem** - Hugo is written in Go, consistent with Portal
2. **Fast builds** - Hugo compiles thousands of pages in seconds
3. **No runtime** - Generates static HTML, served directly from NAS
4. **Docsy theme** - Full-featured docs theme with versioning, search, i18n
5. **Self-contained** - Pagefind search works offline, no Algolia needed
### Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ Synology NAS │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ mosis-portal container │ │
│ │ ├── Go binary (API + Portal UI) │ │
│ │ ├── /static/docs/ → Hugo build output │ │
│ │ └── http.FileServer serves docs at /docs/* │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ /volume1/mosis/ │
│ └── docs/ │
│ ├── content/ (Markdown source) │
│ ├── static/ (Images, assets) │
│ └── public/ (Hugo build output → served) │
└─────────────────────────────────────────────────────────────────┘
Build pipeline:
docs/ (Markdown) ──► hugo build ──► public/ ──► Deploy to NAS
```
### URL Structure
```
https://portal.mosis.dev/docs/ # Docs home
https://portal.mosis.dev/docs/quickstart/ # Getting started
https://portal.mosis.dev/docs/api/ # API reference
https://portal.mosis.dev/docs/cli/ # CLI reference
```
---
## Overview
The documentation site is the primary resource for developers learning to build Mosis apps. Must be clear, searchable, and up-to-date.
---
## Information Architecture
```
docs.mosis.dev/
├── Getting Started
│ ├── Introduction
│ ├── Quick Start (5 min)
│ ├── Your First App
│ └── Project Structure
├── Guides
│ ├── UI Design
│ │ ├── RML Basics
│ │ ├── Styling with RCSS
│ │ ├── Layouts
│ │ └── Components
│ ├── Lua Scripting
│ │ ├── Basics
│ │ ├── Event Handling
│ │ ├── State Management
│ │ └── Async Operations
│ ├── Data & Storage
│ │ ├── Local Storage
│ │ ├── SQLite Database
│ │ └── Files
│ ├── Networking
│ │ ├── HTTP Requests
│ │ └── WebSockets
│ ├── Hardware
│ │ ├── Camera
│ │ ├── Microphone
│ │ ├── Location
│ │ └── Sensors
│ ├── Permissions
│ │ ├── Permission Model
│ │ ├── Requesting Permissions
│ │ └── User Gestures
│ └── Publishing
│ ├── Preparing for Release
│ ├── App Signing
│ └── Store Guidelines
├── API Reference
│ ├── Lua APIs
│ │ ├── mosis.storage
│ │ ├── mosis.db
│ │ ├── mosis.http
│ │ ├── mosis.ws
│ │ ├── mosis.camera
│ │ ├── mosis.microphone
│ │ ├── mosis.location
│ │ ├── mosis.sensors
│ │ └── ...
│ ├── RML Elements
│ ├── RCSS Properties
│ └── Manifest Schema
├── CLI Reference
│ ├── mosis init
│ ├── mosis build
│ ├── mosis sign
│ ├── mosis publish
│ └── ...
├── Best Practices
│ ├── Performance
│ ├── Security
│ ├── UX Guidelines
│ └── Accessibility
├── Troubleshooting
│ ├── Common Errors
│ ├── Debugging Tips
│ └── FAQ
└── Changelog
```
---
## Content Types
### Tutorials (Step-by-step)
```markdown
# Build a Weather App
In this tutorial, you'll build a weather app that:
- Fetches weather data from an API
- Displays current conditions
- Shows a 5-day forecast
- Requests location permission
**Time:** 30 minutes
**Prerequisites:** Completed Quick Start
## Step 1: Create the Project
```bash
mosis init weather-app
cd weather-app
```
## Step 2: Design the UI
Open `assets/main.rml` and add...
```
### Guides (Conceptual)
```markdown
# Understanding Permissions
Mosis uses a permission system to protect user privacy.
Apps must declare permissions in their manifest and
request them at runtime.
## Permission Categories
| Category | Description | Examples |
|----------|-------------|----------|
| Normal | Low risk, auto-granted | storage, network |
| Dangerous | User data, requires prompt | camera, location |
| Signature | System only | app_management |
## When Permissions Are Checked
Permissions are checked when your app calls...
```
### API Reference (Technical)
```markdown
# mosis.http
HTTP client for making network requests.
## Functions
### `mosis.http.get(url, options)`
Make a GET request.
**Parameters:**
- `url` (string): The URL to fetch
- `options` (table, optional):
- `headers` (table): Request headers
- `timeout` (number): Timeout in ms (default: 30000)
**Returns:** Promise that resolves to Response
**Example:**
```lua
local response = mosis.http.get("https://api.example.com/data")
if response.ok then
local data = json.decode(response.body)
print(data.name)
end
```
**Errors:**
- `NETWORK_ERROR`: Network unavailable
- `TIMEOUT`: Request timed out
- `INVALID_URL`: Malformed URL
```
---
## Tech Stack Options
### Option A: Docusaurus
```
Framework: Docusaurus 3
Language: MDX (Markdown + React)
Search: Algolia DocSearch
Deploy: Vercel/Cloudflare Pages
```
| Pros | Cons |
|------|------|
| Versioning built-in | React knowledge needed |
| Great search | Can be heavy |
| Plugin ecosystem | |
| Used by many OSS | |
### Option B: VitePress
```
Framework: VitePress
Language: Markdown + Vue
Search: Built-in local search
Deploy: Any static host
```
| Pros | Cons |
|------|------|
| Very fast | Fewer features |
| Vue-powered | Less ecosystem |
| Simple setup | |
### Option C: Astro Starlight
```
Framework: Astro + Starlight
Language: MDX
Search: Pagefind (local)
Deploy: Any static host
```
| Pros | Cons |
|------|------|
| Very fast | Newer |
| Great defaults | Less customizable |
| Built-in i18n | |
### Option D: MkDocs Material
```
Framework: MkDocs
Language: Markdown
Search: Built-in
Deploy: Any static host
```
| Pros | Cons |
|------|------|
| Simple | Less interactive |
| Great theme | Python-based |
| Fast builds | |
---
## Features Required
### Must Have
- [ ] Full-text search
- [ ] Syntax highlighting
- [ ] Mobile responsive
- [ ] Dark mode
- [ ] Version selector
- [ ] Edit on GitHub links
- [ ] Copy code buttons
- [ ] Table of contents
- [ ] Previous/Next navigation
### Nice to Have
- [ ] API playground
- [ ] Interactive examples
- [ ] Video tutorials
- [ ] Community translations
- [ ] Feedback widget
---
## Code Examples
### Runnable Examples
```html
<!-- Interactive code block -->
<CodePlayground language="lua">
local greeting = "Hello, Mosis!"
print(greeting)
</CodePlayground>
```
### Multi-file Examples
```markdown
:::code-group
```rml [main.rml]
<div id="counter">
<span id="count">0</span>
<button onclick="increment()">+</button>
</div>
```
```lua [app.lua]
local count = 0
function increment()
count = count + 1
document:GetElementById("count").inner_rml = tostring(count)
end
```
```rcss [styles.rcss]
#counter {
display: flex;
gap: 10px;
}
```
:::
```
---
## API Documentation Generation
### From Lua Annotations
```lua
--- Make an HTTP GET request.
--- @param url string The URL to fetch
--- @param options? HttpOptions Request options
--- @return HttpResponse response The response object
--- @example
--- local resp = mosis.http.get("https://api.example.com/data")
--- print(resp.body)
function mosis.http.get(url, options)
-- implementation
end
```
### Generated Output
```markdown
## `mosis.http.get(url, options?)`
Make an HTTP GET request.
### Parameters
| Name | Type | Description |
|------|------|-------------|
| url | string | The URL to fetch |
| options | HttpOptions? | Request options |
### Returns
`HttpResponse` - The response object
### Example
```lua
local resp = mosis.http.get("https://api.example.com/data")
print(resp.body)
```
```
---
## Versioning
### URL Structure
```
docs.mosis.dev/ # Latest stable
docs.mosis.dev/v1.0/ # Version 1.0
docs.mosis.dev/v1.1/ # Version 1.1
docs.mosis.dev/next/ # Development (unreleased)
```
### Version Dropdown
```
┌────────────────────┐
│ Version: 1.1 (latest) ▼ │
├────────────────────┤
│ 1.1 (latest) │
│ 1.0 │
│ next (unreleased) │
└────────────────────┘
```
---
## Search
### Algolia DocSearch (Free for OSS)
```javascript
// docusaurus.config.js
themeConfig: {
algolia: {
appId: 'YOUR_APP_ID',
apiKey: 'YOUR_SEARCH_KEY',
indexName: 'mosis',
},
}
```
### Local Search (Pagefind)
```javascript
// For Astro/VitePress
// Indexes at build time, searches client-side
// No external service needed
```
---
## Internationalization
### Directory Structure
```
docs/
├── en/
│ ├── getting-started/
│ └── guides/
├── es/
│ ├── getting-started/
│ └── guides/
└── zh/
├── getting-started/
└── guides/
```
### Language Switcher
```
┌──────────┐
│ 🌐 EN ▼ │
├──────────┤
│ English │
│ Español │
│ 中文 │
└──────────┘
```
---
## Content Guidelines
### Writing Style
1. **Be concise** - Get to the point quickly
2. **Use active voice** - "The function returns..." not "A value is returned..."
3. **Show, don't tell** - Code examples over explanations
4. **Assume beginner** - Don't assume prior knowledge
5. **Test all examples** - Every code block must work
### Code Style
```lua
-- Good: Clear, commented
local response = mosis.http.get(API_URL)
if response.ok then
local data = json.decode(response.body)
updateUI(data)
else
showError("Failed to load data")
end
-- Bad: Unclear, no error handling
local d = json.decode(mosis.http.get(u).body)
```
### Screenshots
- Use consistent device frame
- Show relevant UI only
- Add callouts for important areas
- Keep file sizes small (WebP)
---
## Deployment
### CI/CD Pipeline
```yaml
# .github/workflows/docs.yml
name: Deploy Docs
on:
push:
branches: [main]
paths:
- 'docs/**'
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 20
- name: Install and Build
run: |
cd docs
npm install
npm run build
- name: Deploy to Cloudflare Pages
uses: cloudflare/pages-action@v1
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}
projectName: mosis-docs
directory: docs/build
```
---
## Analytics
### Track (Privacy-Friendly)
- Page views (which docs are popular)
- Search queries (what are people looking for)
- 404 pages (missing content)
- Time on page (engagement)
### Tools
- Plausible (privacy-focused)
- Simple Analytics
- Cloudflare Analytics (free)
---
## Feedback
### Per-Page Feedback
```
┌────────────────────────────────────┐
│ Was this page helpful? │
│ │
│ [👍 Yes] [👎 No] │
│ │
│ [Edit this page on GitHub] │
└────────────────────────────────────┘
```
### Feedback Collection
```javascript
// Send to analytics or issue tracker
function submitFeedback(page, helpful, comment) {
fetch('/api/feedback', {
method: 'POST',
body: JSON.stringify({ page, helpful, comment })
})
}
```
---
## Deliverables
- [x] Framework selected (Hugo + Docsy)
- [x] Hosting decided (self-hosted on Synology NAS)
- [ ] Hugo project setup with Docsy theme
- [ ] Information architecture (directory structure)
- [ ] Getting Started content (Quick Start, First App)
- [ ] UI design guides (RML, RCSS)
- [ ] Lua scripting guides
- [ ] API reference (all namespaces)
- [ ] CLI reference (all commands)
- [ ] Best practices (performance, security)
- [ ] Pagefind search integration
- [ ] Deploy script (hugo build + copy to NAS)
---
## Content Prioritization
### Phase 1 (Launch)
1. Quick Start
2. Your First App
3. Project Structure
4. RML Basics
5. Lua Basics
6. API Reference (core APIs)
7. CLI Reference
### Phase 2
1. Complete API Reference
2. All hardware guides
3. Best practices
4. Troubleshooting
### Phase 3
1. Advanced guides
2. Video tutorials
3. Translations
4. Community contributions
---
## Open Questions
1. ~~Host docs separately or under main domain?~~ → Under main domain at /docs/
2. ~~Community wiki/contributions?~~ → Defer to post-MVP (GitHub PRs for docs)
3. Video tutorial platform (YouTube, embedded)? → Consider for v1.1
4. ~~Glossary/terminology page?~~ → Yes, include in Phase 2
---
## References
- [Docusaurus](https://docusaurus.io/)
- [VitePress](https://vitepress.dev/)
- [Astro Starlight](https://starlight.astro.build/)
- [Divio Documentation System](https://documentation.divio.com/)

714
docs/DEV_PORTAL_MILESTONES.md Normal file
View File

@@ -0,0 +1,714 @@
# Developer Portal & App Ecosystem Milestones
Planning document for the Mosis app development, distribution, and monitoring ecosystem.
---
## Overview
```
Developer Mosis Platform User Device
─────────────────────────────────────────────────────────────────────────────────
Register Account ──────────────► Developer Portal
│
Create App Project ────────────► App Registry
│
Design UI (RML/RCSS/Lua) ──────► Designer Tool (local)
│
Test Locally ──────────────────► Desktop Designer / Emulator
│
Submit for Review ─────────────► App Store Backend
│
Publish ───────────────────────► CDN / Distribution
│
Install ◄─────────────────── User Request
│
Run in Sandbox ◄──────────── Launch App
│
Telemetry/Crashes ──────────► Analytics Backend
```
---
## Decision Areas
| Area | Options | Status |
|------|---------|--------|
| Web Stack | Node/Express, Go, Rust/Axum, .NET | TBD |
| Database | PostgreSQL, SQLite, MongoDB | TBD |
| Auth | OAuth2/OIDC, API keys, JWT | TBD |
| CDN/Storage | S3, Cloudflare R2, self-hosted | TBD |
| Telemetry | Custom, PostHog, Plausible | TBD |
| Crash Reports | Sentry, custom, Crashlytics | TBD |
| App Format | ZIP, custom package, signed | TBD |
---
## Milestone 1: App Package Format
**Goal**: Define how apps are bundled, signed, and validated.
### Questions to Answer
1. What files comprise an app package?
2. How is the manifest structured?
3. How are apps signed for integrity?
4. How are updates handled (full vs delta)?
5. What metadata is required (name, version, permissions, icons)?
### Considerations
| Approach | Pros | Cons |
|----------|------|------|
| **ZIP archive** | Simple, standard tooling | No built-in signing |
| **Custom format (.mosis)** | Can embed signature, metadata | Custom tooling needed |
| **Signed ZIP** | Best of both, detached signature | Slightly more complex |
### Proposed Structure
```
myapp.mosis/
├── manifest.json # App metadata, permissions, entry point
├── signature.sig # Detached signature (optional for dev)
├── icon.png # App icon (multiple sizes?)
├── assets/
│ ├── main.rml # Entry point UI
│ ├── styles.rcss # Stylesheets
│ └── scripts/
│ └── app.lua # Lua code
└── locales/ # i18n (optional)
├── en.json
└── es.json
```
### Manifest Schema (Draft)
```json
{
"id": "com.developer.appname",
"name": "My App",
"version": "1.0.0",
"version_code": 1,
"entry": "assets/main.rml",
"permissions": ["storage", "network"],
"min_mosis_version": "1.0.0",
"author": {
"name": "Developer Name",
"email": "dev@example.com"
},
"icons": {
"32": "icon-32.png",
"64": "icon-64.png",
"128": "icon-128.png"
}
}
```
### Deliverables
- [ ] Manifest JSON schema specification
- [ ] Package format specification
- [ ] Signing mechanism (key format, algorithm)
- [ ] Package validation tool (CLI)
- [ ] Package creation tool (CLI or integrated in designer)
---
## Milestone 2: Web Stack Selection
**Goal**: Choose backend technologies for the developer portal and app store.
### Options Analysis
#### Option A: Node.js + Express/Fastify
| Aspect | Details |
|--------|---------|
| Language | TypeScript |
| Framework | Express, Fastify, or Hono |
| Pros | Large ecosystem, easy hiring, fast development |
| Cons | Single-threaded, callback complexity |
| Hosting | Vercel, Railway, any VPS |
#### Option B: Go
| Aspect | Details |
|--------|---------|
| Language | Go |
| Framework | Gin, Echo, or Chi |
| Pros | Fast, low memory, single binary deployment |
| Cons | Smaller ecosystem, verbose error handling |
| Hosting | Any VPS, Cloud Run |
#### Option C: Rust + Axum
| Aspect | Details |
|--------|---------|
| Language | Rust |
| Framework | Axum, Actix-web |
| Pros | Maximum performance, memory safety |
| Cons | Steep learning curve, slower development |
| Hosting | Any VPS, Fly.io |
#### Option D: .NET
| Aspect | Details |
|--------|---------|
| Language | C# |
| Framework | ASP.NET Core |
| Pros | Enterprise-ready, great tooling, fast |
| Cons | Heavier runtime, Microsoft ecosystem |
| Hosting | Azure, any VPS |
### Evaluation Criteria
1. **Development speed** - How fast can we iterate?
2. **Performance** - Can it handle scale?
3. **Hosting cost** - Monthly infrastructure cost
4. **Team familiarity** - Learning curve
5. **Ecosystem** - Libraries for auth, storage, etc.
### Deliverables
- [ ] Prototype API in top 2 candidates
- [ ] Benchmark comparison
- [ ] Final selection with rationale
---
## Milestone 3: Database Selection
**Goal**: Choose database for developer accounts, app metadata, analytics.
### Options Analysis
#### Option A: PostgreSQL
| Aspect | Details |
|--------|---------|
| Type | Relational |
| Pros | ACID, JSON support, mature, scalable |
| Cons | Requires management, connection pooling |
| Hosting | Supabase, Neon, RDS, self-hosted |
#### Option B: SQLite + Litestream
| Aspect | Details |
|--------|---------|
| Type | Embedded relational |
| Pros | Zero ops, fast reads, simple backup |
| Cons | Single-writer, limited concurrency |
| Hosting | Embedded in app server |
#### Option C: MongoDB
| Aspect | Details |
|--------|---------|
| Type | Document |
| Pros | Flexible schema, easy start |
| Cons | Less ACID, can get expensive |
| Hosting | Atlas, self-hosted |
### Data Models Preview
```
developers
├── id (UUID)
├── email
├── name
├── api_keys[]
├── created_at
└── verified
apps
├── id (UUID)
├── developer_id (FK)
├── package_id (com.dev.app)
├── name
├── description
├── versions[]
├── status (draft/review/published/suspended)
├── created_at
└── updated_at
app_versions
├── id (UUID)
├── app_id (FK)
├── version_code
├── version_name
├── package_url
├── signature
├── release_notes
├── status
└── published_at
telemetry_events
├── id
├── app_id
├── device_id (anonymized)
├── event_type
├── event_data (JSON)
├── timestamp
└── mosis_version
```
### Deliverables
- [ ] Schema design for all tables
- [ ] Migration strategy
- [ ] Backup/restore plan
- [ ] Final selection with rationale
---
## Milestone 4: Authentication System
**Goal**: Secure developer authentication and app signing.
### Developer Authentication
| Method | Use Case |
|--------|----------|
| OAuth2 (GitHub/Google) | Portal login |
| Email + Password | Alternative login |
| API Keys | CLI tools, CI/CD |
| JWT | Session tokens |
### App Signing
| Approach | Details |
|----------|---------|
| **Developer keypair** | Dev signs with private key, we verify with public |
| **Platform signing** | We sign after review (like iOS) |
| **Both** | Dev signs, we countersign after review |
### Key Management
- Developer generates keypair locally
- Public key uploaded to portal
- Private key never leaves developer machine
- Key rotation supported
### Deliverables
- [ ] OAuth2 integration (GitHub, Google)
- [ ] API key generation and management
- [ ] Developer keypair registration
- [ ] App signature verification
- [ ] JWT token handling
---
## Milestone 5: Developer Portal Frontend
**Goal**: Web interface for developer account and app management.
### Pages Required
| Page | Features |
|------|----------|
| **Landing** | Sign up, sign in, overview |
| **Dashboard** | App list, quick stats |
| **App Details** | Versions, analytics, settings |
| **Create App** | Wizard for new app |
| **Submit Version** | Upload, release notes, submit |
| **API Keys** | Generate, revoke keys |
| **Profile** | Account settings, keys |
| **Docs** | SDK docs, API reference |
### Tech Options
| Option | Pros | Cons |
|--------|------|------|
| Next.js | SSR, React, full-stack | Complexity |
| SvelteKit | Fast, simple, SSR | Smaller ecosystem |
| Astro + React | Static + islands | Newer |
| Plain HTML + htmx | Simple, fast | Limited interactivity |
### Deliverables
- [ ] Framework selection
- [ ] UI component library selection
- [ ] Page wireframes
- [ ] Implementation
---
## Milestone 6: App Store Backend API
**Goal**: REST/GraphQL API for app submission, review, and distribution.
### API Endpoints (Draft)
```
Auth
├── POST /auth/register
├── POST /auth/login
├── POST /auth/logout
├── GET /auth/me
Apps
├── GET /apps # List developer's apps
├── POST /apps # Create new app
├── GET /apps/:id # Get app details
├── PATCH /apps/:id # Update app metadata
├── DELETE /apps/:id # Delete app (if no published versions)
Versions
├── GET /apps/:id/versions # List versions
├── POST /apps/:id/versions # Upload new version
├── GET /apps/:id/versions/:vid # Get version details
├── POST /apps/:id/versions/:vid/submit # Submit for review
├── POST /apps/:id/versions/:vid/publish # Publish (after review)
Public (App Store)
├── GET /store/apps # Browse/search apps
├── GET /store/apps/:id # App store listing
├── GET /store/apps/:id/download # Download latest version
API Keys
├── GET /keys # List API keys
├── POST /keys # Generate new key
├── DELETE /keys/:id # Revoke key
Telemetry (device → server)
├── POST /telemetry/events # Batch event upload
├── POST /telemetry/crash # Crash report
```
### Deliverables
- [ ] OpenAPI specification
- [ ] Rate limiting strategy
- [ ] Authentication middleware
- [ ] Implementation
---
## Milestone 7: CDN & Storage
**Goal**: Scalable storage for app packages and assets.
### Requirements
1. Store app packages (.mosis files)
2. Serve downloads globally with low latency
3. Handle icons and screenshots
4. Version retention policy
5. Bandwidth cost management
### Options
| Option | Pros | Cons |
|--------|------|------|
| **Cloudflare R2** | No egress fees, global | Newer service |
| **AWS S3 + CloudFront** | Mature, reliable | Egress costs |
| **Backblaze B2 + Cloudflare** | Cheap storage, free egress via CF | More setup |
| **Self-hosted MinIO** | Full control | Ops burden |
### Storage Structure
```
/packages/
/{app_id}/
/{version_code}/
package.mosis
signature.sig
/assets/
/{app_id}/
icon-32.png
icon-64.png
icon-128.png
screenshots/
1.png
2.png
```
### Deliverables
- [ ] Storage provider selection
- [ ] CDN configuration
- [ ] Upload flow (presigned URLs vs direct)
- [ ] Download URL generation
- [ ] Retention/cleanup policy
---
## Milestone 8: Telemetry System
**Goal**: Collect app usage analytics and crash reports.
### Event Types
| Category | Events |
|----------|--------|
| **Lifecycle** | app_start, app_stop, app_crash |
| **Performance** | frame_time, memory_usage, lua_errors |
| **Usage** | screen_view, button_click (opt-in) |
| **System** | mosis_version, device_info |
### Privacy Considerations
1. **No PII by default** - Device ID is hashed, no user data
2. **Opt-in for detailed analytics** - User consent required
3. **Data retention** - Auto-delete after X days
4. **GDPR compliance** - Export/delete on request
5. **Aggregation** - Store aggregates, drop raw after processing
### Options
| Option | Pros | Cons |
|--------|------|------|
| **Custom** | Full control, no vendor lock | Build everything |
| **PostHog** | Self-hostable, feature-rich | Can be heavy |
| **Plausible** | Privacy-focused, simple | Limited features |
| **Segment + warehouse** | Flexible routing | Complex, costly |
### Crash Report Schema
```json
{
"app_id": "com.dev.app",
"app_version": "1.0.0",
"mosis_version": "1.0.0",
"device_id": "hashed",
"timestamp": "2024-01-15T10:30:00Z",
"crash_type": "lua_error",
"message": "attempt to index nil value",
"stack_trace": "...",
"context": {
"screen": "main.rml",
"memory_mb": 45,
"uptime_seconds": 120
}
}
```
### Deliverables
- [ ] Event schema specification
- [ ] Collection endpoint
- [ ] Storage strategy (time-series DB?)
- [ ] Dashboard for developers
- [ ] Privacy controls
---
## Milestone 9: App Review System
**Goal**: Automated and manual review process for app submissions.
### Automated Checks
| Check | Description |
|-------|-------------|
| **Manifest validation** | Required fields, valid permissions |
| **Package integrity** | Signature verification |
| **Static analysis** | Dangerous Lua patterns |
| **Asset validation** | Icons present, correct sizes |
| **Size limits** | Package under max size |
| **Duplicate detection** | Same app ID collision |
### Manual Review (Optional)
- Flag for manual review based on:
- New developer (first app)
- Dangerous permissions requested
- Automated check warnings
- User reports
### Review States
```
draft → submitted → in_review → approved → published
↘ rejected (with feedback)
```
### Deliverables
- [ ] Automated validation pipeline
- [ ] Review queue UI
- [ ] Rejection feedback system
- [ ] Appeal process
---
## Milestone 10: Device-Side App Management
**Goal**: Install, update, and manage apps on Mosis devices.
### Components
| Component | Location | Purpose |
|-----------|----------|---------|
| App Manager | MosisService | Install/uninstall/update apps |
| App Store Client | System app | Browse, search, install UI |
| Update Checker | Background service | Check for updates |
### Installation Flow
```
1. User taps "Install" in App Store
2. Download package from CDN
3. Verify signature
4. Extract to app directory
5. Register with LuaSandboxManager
6. Add to home screen
```
### Update Flow
```
1. Background check for updates (daily?)
2. Notify user of available updates
3. Download new version
4. Verify signature
5. Replace app files (atomic swap)
6. Restart app if running
```
### Storage Layout
```
/data/mosis/
/apps/
/com.dev.app/
/package/ # Extracted app files
/data/ # App data (VirtualFS)
/cache/ # App cache
/db/ # SQLite databases
```
### Deliverables
- [ ] AppManager class in MosisService
- [ ] App Store system app
- [ ] Update checking service
- [ ] Uninstall with data cleanup
---
## Milestone 11: Developer CLI Tool
**Goal**: Command-line tool for app development workflow.
### Commands
```bash
# Project management
mosis init # Create new app project
mosis validate # Validate manifest and assets
# Packaging
mosis build # Create .mosis package
mosis sign # Sign package with developer key
# Testing
mosis run # Run in local designer/emulator
mosis test # Run automated tests
# Publishing
mosis login # Authenticate with portal
mosis publish # Upload and submit for review
mosis status # Check review status
# Keys
mosis keys generate # Generate signing keypair
mosis keys register # Upload public key to portal
```
### Implementation Options
| Option | Pros | Cons |
|--------|------|------|
| Node.js (oclif) | Easy to build, npm distribution | Requires Node |
| Go | Single binary, fast | Slower development |
| Rust (clap) | Single binary, fast | Slower development |
### Deliverables
- [ ] CLI framework selection
- [ ] Core commands implementation
- [ ] Distribution (npm, homebrew, direct download)
- [ ] Documentation
---
## Milestone 12: Documentation Site
**Goal**: Comprehensive docs for developers.
### Sections
| Section | Content |
|---------|---------|
| **Getting Started** | Quick start, first app tutorial |
| **Guides** | UI design, Lua scripting, permissions |
| **API Reference** | All Lua APIs, manifest schema |
| **CLI Reference** | All commands and options |
| **Best Practices** | Performance, security, UX |
| **Troubleshooting** | Common issues, FAQ |
### Tech Options
| Option | Pros | Cons |
|--------|------|------|
| Docusaurus | React-based, versioning | Heavy |
| VitePress | Vue-based, fast | Less features |
| Astro Starlight | Fast, modern | Newer |
| MkDocs | Python, simple | Less customizable |
### Deliverables
- [ ] Framework selection
- [ ] Information architecture
- [ ] Content writing
- [ ] API docs generation from code
- [ ] Search integration
---
## Summary
| Phase | Milestones | Description |
|-------|------------|-------------|
| **Foundation** | 1-4 | Package format, web stack, database, auth |
| **Portal** | 5-6 | Developer portal frontend and API |
| **Distribution** | 7, 10 | CDN/storage, device-side app management |
| **Quality** | 8-9 | Telemetry, crash reports, review system |
| **Tooling** | 11-12 | CLI tool, documentation |
### Recommended Order
1. **Milestone 1** - Package format (needed by everything)
2. **Milestone 2** - Web stack selection
3. **Milestone 3** - Database selection
4. **Milestone 4** - Authentication
5. **Milestone 6** - Backend API
6. **Milestone 5** - Portal frontend
7. **Milestone 7** - CDN/storage
8. **Milestone 10** - Device-side management
9. **Milestone 11** - CLI tool
10. **Milestone 9** - Review system
11. **Milestone 8** - Telemetry
12. **Milestone 12** - Documentation
---
## Open Questions
1. **Monetization model?** - Free only, paid apps, subscriptions?
2. **Enterprise/self-hosted?** - Can companies run private app stores?
3. **App categories?** - Predefined or free-form tags?
4. **Rating/reviews?** - User reviews for apps?
5. **Analytics dashboard?** - What metrics do developers see?
6. **Localization?** - Multi-language portal and apps?
7. **Beta testing?** - TestFlight-like distribution?
8. **Team accounts?** - Multiple developers per app?
---
## Next Steps
Begin with Milestone 1 (App Package Format) to establish the foundation, then proceed with technology selections in Milestones 2-4 before building the portal.

348
docs/GAME-ENGINES.md Normal file
View File

@@ -0,0 +1,348 @@
# Game Engine Integrations
MosisService provides a virtual phone that game engines can display and interact with.
## Integration Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ MosisService │
│ (OpenGL ES rendering → AHardwareBuffer) │
└───────────────────────────┬─────────────────────────────────────┘
│ Binder IPC + Shared Memory
┌─────────────────┴─────────────────┐
▼ ▼
┌─────────────────────┐ ┌─────────────────────┐
│ MosisUnreal │ │ MosisVR │
│ (UE5.5 Plugin) │ │ (Unity Package) │
│ Vulkan Import │ │ Vulkan/OpenGL │
└─────────────────────┘ └─────────────────────┘
```
## MosisUnreal (Unreal Engine 5.5)
**Location**: `D:\Dev\Mosis\MosisUnreal\Plugins\MosisSDK\`
### Build Commands
```batch
:: Windows Editor Build
"D:\Epic\UE_5.5\Engine\Build\BatchFiles\Build.bat" ^
MosisUnrealEditor Win64 Development ^
-Project="D:\Dev\Mosis\MosisUnreal\MosisUnreal.uproject"
:: Android APK Build
"D:\Epic\UE_5.5\Engine\Build\BatchFiles\RunUAT.bat" ^
BuildCookRun ^
-project="D:\Dev\Mosis\MosisUnreal\MosisUnreal.uproject" ^
-platform=Android -clientconfig=Development ^
-build -cook -stage -pak -package -noP4
:: Clean Build (delete these folders first)
rmdir /s /q "Intermediate\Build"
rmdir /s /q "Binaries"
```
### Output Files
| Build | Output |
|-------|--------|
| Windows Editor | `Plugins/MosisSDK/Binaries/Win64/UnrealEditor-MosisSDK.dll` |
| Android APK | `Binaries/Android/MosisUnreal-arm64.apk` |
| Android OBB | `Binaries/Android/main.1.com.omixlab.MosisUnreal.obb` |
### Requirements
- Android SDK Platform 36 (for AIDL binder headers)
- Android Build Tools 36.1.0 (for AIDL compiler)
- `ANDROID_HOME` environment variable set
### Quest Deployment
**IMPORTANT**: Git Bash on Windows converts Unix paths to Windows paths. Use `MSYS_NO_PATHCONV=1` prefix for all adb push commands.
```bash
# Set Quest device ID (check with: adb devices -l)
QUEST=2G0YC5ZF7W01T0
# Install APK
adb -s $QUEST install -r D:/Dev/Mosis/MosisUnreal/Binaries/Android/MosisUnreal-arm64.apk
# Create OBB temp directory
adb -s $QUEST shell "mkdir -p /data/local/tmp/obb/com.omixlab.MosisUnreal"
# Push OBB to temp location (MUST use MSYS_NO_PATHCONV=1)
MSYS_NO_PATHCONV=1 adb -s $QUEST push D:/Dev/Mosis/MosisUnreal/Binaries/Android/main.1.com.omixlab.MosisUnreal.obb /data/local/tmp/obb/com.omixlab.MosisUnreal/
# Move OBB to final location
adb -s $QUEST shell "rm -rf /sdcard/Android/obb/com.omixlab.MosisUnreal"
adb -s $QUEST shell "mv /data/local/tmp/obb/com.omixlab.MosisUnreal /sdcard/Android/obb/"
# Start MosisService first, then MosisUnreal
adb -s $QUEST shell am start -n com.omixlab.mosis/.MainActivity
sleep 3
adb -s $QUEST shell am start -n com.omixlab.MosisUnreal/com.epicgames.unreal.GameActivity
# Monitor logs
adb -s $QUEST logcat -s MosisSDK MosisOS MosisTest
```
**Full rebuild and deploy sequence**:
```bash
# 1. Build Android APK+OBB
"D:\Epic\UE_5.5\Engine\Build\BatchFiles\RunUAT.bat" BuildCookRun \
-project="D:\Dev\Mosis\MosisUnreal\MosisUnreal.uproject" \
-platform=Android -clientconfig=Development \
-build -cook -stage -pak -package -noP4
# 2. Stop app, push APK and OBB, restart
QUEST=2G0YC5ZF7W01T0
adb -s $QUEST shell am force-stop com.omixlab.MosisUnreal
adb -s $QUEST install -r D:/Dev/Mosis/MosisUnreal/Binaries/Android/MosisUnreal-arm64.apk
adb -s $QUEST shell "mkdir -p /data/local/tmp/obb/com.omixlab.MosisUnreal"
MSYS_NO_PATHCONV=1 adb -s $QUEST push D:/Dev/Mosis/MosisUnreal/Binaries/Android/main.1.com.omixlab.MosisUnreal.obb /data/local/tmp/obb/com.omixlab.MosisUnreal/
adb -s $QUEST shell "rm -rf /sdcard/Android/obb/com.omixlab.MosisUnreal && mv /data/local/tmp/obb/com.omixlab.MosisUnreal /sdcard/Android/obb/"
adb -s $QUEST shell am start -n com.omixlab.MosisUnreal/com.epicgames.unreal.GameActivity
```
**Common issues**:
- App stuck on loading screen: OBB not pushed or wrong version
- `adb push` path errors: Missing `MSYS_NO_PATHCONV=1` prefix
- Service not connecting: Start `com.omixlab.mosis` before the game
### VR Pointer Interaction
The MosisSDK includes `UMosisPointerComponent` for VR ray-based touch interaction.
**Key Files**:
- `Public/MosisPointerComponent.h` - Component header
- `Private/MosisPointerComponent.cpp` - Implementation
**Features**:
- Raycast from component transform (attach as child of motion controller)
- Automatic detection of `AMosisPhoneActor` hits
- Touch state machine: Down/Move/Up events
- Optional Enhanced Input binding via `TriggerAction` property
- Manual control via `SetTriggerPressed(bool)`
- Debug ray visualization
**Blueprint Setup**:
1. Add `MosisPointerComponent` as child of `MotionControllerComponent`
2. Set `TriggerAction` to your trigger input action (optional)
3. Touch events are sent automatically when pointing at phone and triggering
**C++ Setup**:
```cpp
// In VR Pawn header
UPROPERTY(VisibleAnywhere)
UMosisPointerComponent* RightPointer;
// In constructor
RightPointer = CreateDefaultSubobject<UMosisPointerComponent>(TEXT("RightPointer"));
RightPointer->SetupAttachment(RightMotionController);
// In BeginPlay (if using Enhanced Input)
RightPointer->TriggerAction = TriggerInputAction;
```
**Manual Control (without Enhanced Input)**:
```cpp
// In your input handling code
void AMyVRPawn::OnTriggerPressed()
{
RightPointer->SetTriggerPressed(true);
}
void AMyVRPawn::OnTriggerReleased()
{
RightPointer->SetTriggerPressed(false);
}
```
**Component Properties**:
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `RayLength` | float | 500.0 | Maximum ray length in cm |
| `bShowDebugRay` | bool | false | Draw debug visualization |
| `DebugRayColor` | FLinearColor | Red | Ray color when not hitting |
| `DebugRayHitColor` | FLinearColor | Green | Ray color when hitting phone |
| `TraceChannel` | ECollisionChannel | Visibility | Collision channel for raycast |
| `TriggerAction` | UInputAction* | nullptr | Enhanced Input action for trigger |
**Blueprint Functions**:
| Function | Returns | Description |
|----------|---------|-------------|
| `IsPointingAtPhone()` | bool | True if ray hits a phone actor |
| `GetTargetPhone()` | AMosisPhoneActor* | Currently targeted phone (or nullptr) |
| `GetHitLocation()` | FVector | World location of ray hit |
| `GetRayOrigin()` | FVector | Ray start position |
| `GetRayDirection()` | FVector | Ray direction vector |
| `IsTriggerPressed()` | bool | Current trigger state |
| `SetTriggerPressed(bool)` | void | Manual trigger control |
**Touch State Machine**:
```
Idle ──[raycast hit]──► Hover ──[trigger]──► Pressed
▲ │ │
│ │ raycast miss │ trigger release
│ ▼ ▼
└──────────────────── Idle ◄────────────────────
Events:
- Idle → Pressed: SendTouch(Down)
- Pressed + moving: SendTouch(Move)
- Pressed → Idle: SendTouch(Up)
```
## MosisVR (Unity 6000.3.2f1)
**Location**: `D:\Dev\Mosis\MosisVR\Packages\com.omixlab.mosis_sdk\`
### Direct APK Build (Recommended)
```batch
"C:\Program Files\Unity\Hub\Editor\6000.3.2f1\Editor\Unity.exe" ^
-batchmode -quit -nographics ^
-projectPath "D:\Dev\Mosis\MosisVR" ^
-executeMethod BuildScript.BuildAndroidDirectCI ^
-outputPath "D:\Dev\Mosis\Builds\Unity\Android\MosisVR.apk"
```
### Export + Gradle Build
For more control, export a Gradle project then build separately:
```batch
:: Step 1: Export from Unity
"C:\Program Files\Unity\Hub\Editor\6000.3.2f1\Editor\Unity.exe" ^
-batchmode -quit -nographics ^
-projectPath "D:\Dev\Mosis\MosisVR" ^
-executeMethod BuildScript.BuildAndroidCI ^
-export true ^
-outputPath "D:\Dev\Mosis\Builds\Unity\Android\MosisVR"
:: Step 2: Build with Gradle
cd D:\Dev\Mosis\Builds\Unity\Android\MosisVR
gradle assembleRelease
:: APK at: launcher\build\outputs\apk\release\launcher-release.apk
```
### Unity Editor Manual Build
1. File > Build Settings > Android
2. Player Settings: IL2CPP, ARM64, Vulkan + OpenGLES3
3. For direct APK: Uncheck "Export Project", click Build
4. For export: Check "Export Project", click Export
### Native Plugin Build (Manual)
The native plugin builds automatically via CMake during Unity's build. To rebuild manually:
```batch
cd Packages/com.omixlab.mosis_sdk/Plugins/Android/cpp
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%ANDROID_NDK_HOME%/build/cmake/android.toolchain.cmake ^
-DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-29
cmake --build build
```
## Device Testing (Both Engines)
```bash
# Install MosisService first
adb install -r MosisService-debug.apk
# Install game client
adb install -r MosisUnreal-arm64.apk # or MosisVR.apk
# Launch service
adb shell am start -n com.omixlab.mosis/.MainActivity
# Launch client
adb shell am start -n com.omixlab.MosisUnreal/com.epicgames.unreal.GameActivity
# or for Unity:
adb shell am start -n com.omixlab.mosisvr/com.unity3d.player.UnityPlayerActivity
# Monitor all Mosis logs
adb logcat -s MosisSDK MosisTest RMLUI Vulkan
```
## Vulkan HardwareBuffer Import
Both game engines use Vulkan to import AHardwareBuffer from MosisService.
### Required Vulkan Extensions
```
VK_ANDROID_external_memory_android_hardware_buffer
VK_KHR_external_memory
VK_KHR_dedicated_allocation
```
### Import Pattern
```cpp
// 1. Query buffer properties
VkAndroidHardwareBufferPropertiesANDROID props = {
.sType = VK_STRUCTURE_TYPE_ANDROID_HARDWARE_BUFFER_PROPERTIES_ANDROID
};
VkAndroidHardwareBufferFormatPropertiesANDROID formatProps = {
.sType = VK_STRUCTURE_TYPE_ANDROID_HARDWARE_BUFFER_FORMAT_PROPERTIES_ANDROID
};
props.pNext = &formatProps;
vkGetAndroidHardwareBufferPropertiesANDROID(device, buffer, &props);
// 2. Create image with external memory
VkExternalMemoryImageCreateInfo extInfo = {
.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO,
.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_ANDROID_HARDWARE_BUFFER_BIT_ANDROID
};
AHardwareBuffer_Desc desc;
AHardwareBuffer_describe(buffer, &desc);
VkImageCreateInfo imageInfo = {
.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO,
.pNext = &extInfo,
.imageType = VK_IMAGE_TYPE_2D,
.format = formatProps.format,
.extent = {desc.width, desc.height, 1},
.mipLevels = 1,
.arrayLayers = 1,
.samples = VK_SAMPLE_COUNT_1_BIT,
.tiling = VK_IMAGE_TILING_OPTIMAL,
.usage = VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT,
.sharingMode = VK_SHARING_MODE_EXCLUSIVE
};
vkCreateImage(device, &imageInfo, nullptr, &image);
// 3. Import memory from HardwareBuffer
VkImportAndroidHardwareBufferInfoANDROID importInfo = {
.sType = VK_STRUCTURE_TYPE_IMPORT_ANDROID_HARDWARE_BUFFER_INFO_ANDROID,
.buffer = buffer
};
VkMemoryDedicatedAllocateInfo dedicatedInfo = {
.sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO,
.pNext = &importInfo,
.image = image
};
VkMemoryAllocateInfo allocInfo = {
.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO,
.pNext = &dedicatedInfo,
.allocationSize = props.allocationSize,
.memoryTypeIndex = FindMemoryType(props.memoryTypeBits, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)
};
vkAllocateMemory(device, &allocInfo, nullptr, &memory);
vkBindImageMemory(device, image, memory, 0);
```
### Synchronization
Currently using CPU synchronization (`vkQueueWaitIdle` / `glFinish`). Future improvement: use Vulkan semaphores for GPU-GPU sync.
### Double Buffering
The imported image is copied to a local texture each frame to prevent data races with MosisService rendering.

126
docs/LUA-SANDBOX.md Normal file
View File

@@ -0,0 +1,126 @@
# Lua Sandbox System
The sandbox provides secure, isolated Lua environments for third-party apps.
## Security Features
| Feature | Implementation |
|---------|----------------|
| Dangerous globals removed | `os`, `io`, `loadfile`, `dofile`, `debug` |
| Memory limits | Configurable per-app (default 10MB) |
| CPU limits | Instruction counting with timeout |
| Bytecode rejected | Only source code allowed |
| Metatables protected | Cannot modify string/table metatables |
| Path traversal blocked | `../` and absolute paths rejected |
## Permission Categories
| Category | Auto-Grant | Examples |
|----------|------------|----------|
| Normal | Yes | `storage`, `network` |
| Dangerous | User consent | `camera`, `microphone`, `location`, `contacts` |
| Signature | System apps only | `system_settings`, `install_packages` |
## Available APIs
**Core APIs** (always available):
```lua
-- Timers
local id = setTimeout(function() end, 1000)
clearTimeout(id)
local id = setInterval(function() end, 500)
clearInterval(id)
-- JSON
local obj = json.decode('{"key": "value"}')
local str = json.encode({key = "value"})
-- Crypto
local bytes = crypto.randomBytes(16)
local hash = crypto.sha256("data")
local hmac = crypto.hmac("sha256", "key", "data")
```
**Storage APIs** (requires `storage` permission):
```lua
-- Virtual filesystem (sandboxed to app directory)
fs.write("data.txt", "content")
local content = fs.read("data.txt")
local files = fs.list("/")
local stat = fs.stat("data.txt")
fs.delete("data.txt")
-- SQLite database
local db = database.open("mydb")
db:execute("CREATE TABLE IF NOT EXISTS items (id INTEGER PRIMARY KEY, name TEXT)")
db:execute("INSERT INTO items (name) VALUES (?)", {"item1"})
local rows = db:query("SELECT * FROM items WHERE id = ?", {1})
```
**Network APIs** (requires `network` permission):
```lua
-- HTTP (HTTPS only, private IPs blocked)
local response = http.get("https://api.example.com/data")
local response = http.post("https://api.example.com/data", {
headers = {["Content-Type"] = "application/json"},
body = json.encode({key = "value"})
})
-- WebSocket
local ws = websocket.connect("wss://example.com/ws")
ws:send("message")
ws:onMessage(function(data) end)
ws:close()
```
**Hardware APIs** (requires dangerous permissions + user gesture):
```lua
-- Camera (requires camera permission)
camera.start(function(frame) end)
camera.stop()
-- Microphone (requires microphone permission)
microphone.start(function(samples) end)
microphone.stop()
-- Location (requires location permission)
location.getCurrentPosition(function(pos)
print(pos.latitude, pos.longitude)
end)
-- Sensors (requires sensors permission)
sensors.subscribe("accelerometer", function(data)
print(data.x, data.y, data.z)
end)
```
## Running Sandbox Tests
```bash
cd sandbox-test
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake
cmake --build build --config Debug
./build/Debug/sandbox-test.exe
# Output: 149 tests, all passing
```
## Test Categories
| Category | Tests | Description |
|----------|-------|-------------|
| Security | 11 | Globals removal, bytecode, metatables |
| Resources | 8 | Memory, CPU limits, instruction counting |
| Permissions | 7 | Normal/dangerous/signature grants |
| Rate Limiting | 6 | API call throttling |
| Timers | 7 | setTimeout/setInterval behavior |
| JSON | 5 | Encode/decode, depth limits |
| Crypto | 4 | Random, SHA256, HMAC |
| VirtualFS | 8 | Read/write, quotas, traversal |
| Database | 8 | SQLite operations, injection prevention |
| Network | 8 | URL validation, private IP blocking |
| WebSocket | 7 | Connection limits, message size |
| Hardware | 42 | Camera, mic, location, sensors, bluetooth |
| IPC | 7 | Message bus between apps |
| Integration | 9 | Full app lifecycle |
| Fuzzing | 3 | Random input crash testing |

79
docs/MATERIAL-DESIGN.md Normal file
View File

@@ -0,0 +1,79 @@
# Material Design Resources
Material Design icons and components are available in the MosisDesigner repository.
## Material Design Icons
**Location**: `D:\Dev\Mosis\MosisDesigner\material-design-icons`
A comprehensive icon library from Google with 2000+ icons across 20 categories:
| Category | Examples |
|----------|----------|
| action | home, search, settings, delete, info |
| alert | error, warning, notification |
| av | play, pause, volume, mic |
| communication | phone, message, email, contacts |
| content | add, remove, copy, paste |
| device | battery, wifi, bluetooth, gps |
| editor | format, text, color, brush |
| file | folder, attachment, download, upload |
| hardware | keyboard, mouse, phone, tablet |
| home | lightbulb, thermostat, security |
| image | camera, photo, filter, tune |
| maps | location, directions, navigation |
| navigation | arrow, chevron, menu, close |
| notification | sync, update, event |
| places | hotel, restaurant, airport |
| search | search variants |
| social | share, person, group, notifications |
| toggle | star, checkbox, radio |
**Available Formats**:
- `src/` - SVG source files organized by category
- `png/` - PNG files at multiple DPIs (24dp, 36dp, 48dp)
- `font/` - Icon fonts (WOFF, TTF)
- `symbols/` - Material Symbols variable font (newer)
- `variablefont/` - Variable font files
**Icon Styles**:
- Outlined (default)
- Filled
- Rounded
- Sharp
- Two-tone (Material Icons only)
## Material Design Lite
**Location**: `D:\Dev\Mosis\MosisDesigner\material-design-lite`
CSS/JS component library implementing Material Design (reference implementation):
| Directory | Contents |
|-----------|----------|
| `src/` | SASS source for components |
| `docs/` | Component documentation |
| `templates/` | Page templates |
**Key Components** (for design reference):
- Buttons (raised, flat, FAB)
- Cards
- Dialogs
- Lists
- Menus
- Navigation drawers
- Progress indicators
- Sliders
- Snackbars
- Tables
- Tabs
- Text fields
- Tooltips
## Using Icons in Mosis
1. **Find icon** at https://fonts.google.com/icons
2. **Export SVG** from `material-design-icons/src/<category>/<name>/`
3. **Convert to TGA** using image tool (24x24 or 32x32, RGBA)
4. **Place in** `src/main/assets/icons/`
5. **Reference in RML**: `<img src="../../icons/<name>.tga"/>`

177
docs/MILESTONE-2.md Normal file
View File

@@ -0,0 +1,177 @@
# Milestone 2: Testing Framework
**Status**: 100% Complete
**Goal**: Automated UI testing for rapid iteration and AI agent verification.
---
## Overview
The testing framework enables automated validation of UI behavior through:
- UI hierarchy inspection (JSON dump)
- Input simulation and recording
- Log verification for navigation events
- Visual regression testing via screenshot diff
- JSON test results compatible with CI/CD
---
## Current State
### Completed Components
| Component | File | Status |
|-----------|------|--------|
| Action Types | `designer/src/testing/action_types.h` | Complete |
| Action Recorder | `designer/src/testing/action_recorder.h/.cpp` | Complete |
| Action Player | `designer/src/testing/action_player.h/.cpp` | Complete |
| UI Inspector | `designer/src/testing/ui_inspector.cpp` | Complete |
| Visual Capture | `designer/src/testing/visual_capture.cpp` | Complete |
| Test Runner | `designer-test/src/test_runner.cpp` | Complete |
| CLI Integration | `designer/src/main.cpp` | Complete |
### Already Working
1. **Action Types** (`action_types.h`)
- TapAction, SwipeAction, LongPressAction
- ButtonAction, WaitAction, KeyAction
- ActionSequence container with metadata
2. **Action Recorder** (`action_recorder.h/.cpp`)
- Mouse down/up/move tracking
- Automatic gesture classification (tap vs swipe vs long press)
- JSON serialization/deserialization
- Configurable thresholds
3. **Action Player** (`action_player.h/.cpp`)
- Timestamp-based playback
- RmlUi event injection
- Pause/resume/stop controls
- Progress tracking
4. **UI Inspector** (`ui_inspector.cpp`)
- Full element tree traversal
- JSON export with bounds, IDs, classes
- Continuous hierarchy dumping mode
5. **Test Runner** (`designer-test/`)
- WindowController (Windows SendInput)
- HierarchyReader (element lookup)
- LogParser (navigation verification)
- All 5 navigation tests passing
---
## Recently Completed
### Screenshot Diff Implementation (Complete)
**File**: `designer/src/testing/visual_capture.cpp`
Implemented real PNG pixel-by-pixel comparison:
- `LoadPNG()` helper function loads PNG files using libpng
- `PixelsMatch()` compares pixels with configurable tolerance (default: 2)
- `CompareImages()` returns difference ratio (0.0 = identical, 1.0 = completely different)
### Recording/Playback CLI (Complete)
**File**: `designer/src/main.cpp`
Added CLI options and keyboard controls:
- `--record <file>` - Enable recording mode
- `--playback <file>` - Play back recorded actions
- F5 - Start/stop recording (saves to specified file)
- F6 - Pause/resume playback
**Usage**:
```bash
# Record interactions
mosis-designer.exe home.rml --record my-test.json
# Press F5 to start recording, interact, press F5 again to save
# Play back
mosis-designer.exe home.rml --playback my-test.json
```
---
## Completed: GLFW Input Hooks for Recording
### Task 2.3: GLFW Input Hooks
**Status**: Complete
**Solution Implemented**:
Forked the RmlUi backend files into `designer/src/backend/` with input recording hooks:
1. **RmlUi_Backend.h** - Added callback type definitions:
- `MouseButtonCallback` - Called on mouse button press/release
- `MouseMoveCallback` - Called on mouse movement
- `KeyCallback` - Called on key press/release
2. **RmlUi_Backend_GLFW_GL3.cpp** - Modified GLFW callbacks to:
- Track mouse position in framebuffer coordinates
- Call recording callbacks before forwarding to RmlUi
- Support all three callback types
3. **main.cpp** - Connected callbacks to ActionRecorder:
- Mouse button events trigger `RecordMouseDown`/`RecordMouseUp`
- Mouse move events trigger `RecordMouseMove`
- Key events trigger `RecordKey`
**Files Added**:
- `designer/src/backend/RmlUi_Backend.h`
- `designer/src/backend/RmlUi_Backend_GLFW_GL3.cpp`
- `designer/src/backend/RmlUi_Platform_GLFW.h`
- `designer/src/backend/RmlUi_Platform_GLFW.cpp`
---
## File Changes Summary
| File | Changes | Status |
|------|---------|--------|
| `designer/src/testing/visual_capture.cpp` | Real PNG pixel comparison | Done |
| `designer/src/main.cpp` | --record/--playback CLI, F5/F6 keys | Done |
| `designer/src/RmlUi_Backend.h` | Expose GLFW window for recording | Future |
---
## Test Recording Format
```json
{
"name": "Navigate to contacts and back",
"description": "Test navigation flow",
"screen_width": 540,
"screen_height": 960,
"initial_screen": "apps/home/home.rml",
"actions": [
{"type": "tap", "x": 413, "y": 1174, "timestamp": 0},
{"type": "wait", "duration": 1000, "timestamp": 100},
{"type": "tap", "x": 40, "y": 28, "timestamp": 1100},
{"type": "swipe", "x1": 100, "y1": 500, "x2": 100, "y2": 200, "duration": 300, "timestamp": 2000}
]
}
```
---
## Acceptance Criteria
- [x] `CompareImages()` returns accurate pixel difference ratio
- [ ] `--record <file>` captures all mouse/key events to JSON (needs GLFW hooks)
- [x] `--playback <file>` replays recorded actions with correct timing
- [x] Recording stops gracefully on F5 or window close
- [x] Playback shows progress in console
- [x] Screenshot diff with 2-pixel tolerance per channel
---
## Future Enhancements (Not This Milestone)
- Visual diff output (highlight changed pixels)
- Parallel test execution
- Android action recording/playback
- Cross-platform test runner
- Coverage reporting

333
docs/MILESTONE-3.md Normal file
View File

@@ -0,0 +1,333 @@
# Milestone 3: Virtual Hardware
**Status**: Not Started
**Goal**: Hardware-like APIs backed by game engine or real devices.
---
## Overview
Mosis needs to expose smartphone-like hardware interfaces that can be:
- Provided by game engines (Unity/Unreal) in VR mode
- Mocked for desktop testing
- Connected to real device hardware on Android
---
## Components
### 3.1 Camera Interface
**Header**: `src/main/kernel/include/camera.h`
```cpp
namespace mosis {
struct CameraFrame {
int width;
int height;
std::vector<uint8_t> rgba_data; // RGBA8 format
int64_t timestamp_ms;
};
using CameraCallback = std::function<void(const CameraFrame& frame)>;
class ICamera {
public:
virtual ~ICamera() = default;
// Start/stop frame capture
virtual void StartCapture(CameraCallback callback) = 0;
virtual void StopCapture() = 0;
// Configuration
virtual void SetResolution(int width, int height) = 0;
virtual bool IsAvailable() const = 0;
// Get current state
virtual int GetWidth() const = 0;
virtual int GetHeight() const = 0;
};
} // namespace mosis
```
**Implementations**:
| Implementation | Description | File |
|----------------|-------------|------|
| `GameCamera` | Receives texture from Unity/Unreal | `game_camera.cpp` |
| `DesktopCamera` | System webcam via OpenCV (optional) | `desktop_camera.cpp` |
| `AndroidCamera` | Camera2 API integration | `android_camera.cpp` |
| `MockCamera` | Test patterns (checkerboard, gradient) | `mock_camera.cpp` |
### 3.2 Microphone Interface
**Header**: `src/main/kernel/include/microphone.h`
```cpp
namespace mosis {
struct AudioBuffer {
std::vector<int16_t> samples; // PCM 16-bit
int sample_rate; // Typically 44100 or 48000
int channels; // 1 = mono, 2 = stereo
int64_t timestamp_ms;
};
using AudioCallback = std::function<void(const AudioBuffer& buffer)>;
class IMicrophone {
public:
virtual ~IMicrophone() = default;
virtual void StartCapture(AudioCallback callback) = 0;
virtual void StopCapture() = 0;
virtual void SetSampleRate(int rate) = 0;
virtual bool IsAvailable() const = 0;
virtual bool IsCapturing() const = 0;
};
} // namespace mosis
```
**Implementations**:
| Implementation | Description |
|----------------|-------------|
| `GameMicrophone` | Audio from Unity/Unreal AudioSource |
| `DesktopMicrophone` | System mic via PortAudio |
| `AndroidMicrophone` | AudioRecord API |
| `MockMicrophone` | Silence or test tones |
### 3.3 Speaker Interface
**Header**: `src/main/kernel/include/speaker.h`
```cpp
namespace mosis {
class ISpeaker {
public:
virtual ~ISpeaker() = default;
virtual void PlayAudio(const AudioBuffer& buffer) = 0;
virtual void SetVolume(float volume) = 0; // 0.0 - 1.0
virtual float GetVolume() const = 0;
virtual bool IsAvailable() const = 0;
};
} // namespace mosis
```
### 3.4 Filesystem Interface
**Header**: `src/main/kernel/include/filesystem.h`
```cpp
namespace mosis {
enum class FileMode { Read, Write, Append };
struct FileInfo {
std::string name;
bool is_directory;
size_t size;
int64_t modified_time;
};
class IFileSystem {
public:
virtual ~IFileSystem() = default;
// File operations
virtual std::vector<uint8_t> ReadFile(const std::string& path) = 0;
virtual bool WriteFile(const std::string& path, const std::vector<uint8_t>& data) = 0;
virtual bool DeleteFile(const std::string& path) = 0;
virtual bool FileExists(const std::string& path) = 0;
// Directory operations
virtual std::vector<FileInfo> ListDirectory(const std::string& path) = 0;
virtual bool CreateDirectory(const std::string& path) = 0;
virtual bool DeleteDirectory(const std::string& path) = 0;
// Sandboxed paths
virtual std::string GetAppDataPath(const std::string& app_id) = 0;
virtual std::string GetSharedMediaPath() = 0;
};
} // namespace mosis
```
**App Storage Structure**:
```
/data/
├── apps/
│ ├── com.example.app1/
│ │ ├── files/
│ │ ├── cache/
│ │ └── databases/
│ └── com.example.app2/
├── shared/
│ ├── photos/
│ ├── downloads/
│ └── music/
```
### 3.5 Network Interface
**Header**: `src/main/kernel/include/network.h`
```cpp
namespace mosis {
struct HttpRequest {
std::string method; // GET, POST, PUT, DELETE
std::string url;
std::map<std::string, std::string> headers;
std::vector<uint8_t> body;
};
struct HttpResponse {
int status_code;
std::map<std::string, std::string> headers;
std::vector<uint8_t> body;
};
class INetwork {
public:
virtual ~INetwork() = default;
// HTTP
using HttpCallback = std::function<void(const HttpResponse&)>;
virtual void Fetch(const HttpRequest& request, HttpCallback callback) = 0;
// Connectivity
virtual bool IsOnline() const = 0;
virtual std::string GetConnectionType() const = 0; // "wifi", "cellular", "none"
};
} // namespace mosis
```
---
## Platform Integration
### Platform Interface Extension
**File**: `src/main/kernel/include/platform.h`
```cpp
class IPlatform {
public:
// ... existing methods ...
// Hardware providers
virtual ICamera* GetCamera() = 0;
virtual IMicrophone* GetMicrophone() = 0;
virtual ISpeaker* GetSpeaker() = 0;
virtual IFileSystem* GetFileSystem() = 0;
virtual INetwork* GetNetwork() = 0;
};
```
### Desktop Implementation
**File**: `designer/src/desktop_platform.cpp`
```cpp
class DesktopPlatform : public IPlatform {
std::unique_ptr<MockCamera> m_camera;
std::unique_ptr<MockMicrophone> m_microphone;
std::unique_ptr<DesktopSpeaker> m_speaker;
std::unique_ptr<DesktopFileSystem> m_filesystem;
std::unique_ptr<DesktopNetwork> m_network;
public:
ICamera* GetCamera() override { return m_camera.get(); }
// ...
};
```
### Android Implementation
**File**: `src/main/cpp/android_platform.cpp`
```cpp
class AndroidPlatform : public IPlatform {
// Use JNI to access Android APIs
std::unique_ptr<AndroidCamera> m_camera;
// ...
};
```
### Game Engine Implementation
**File**: Unity plugin or Unreal plugin
```cpp
class GamePlatform : public IPlatform {
// Receives textures/audio from game engine
std::unique_ptr<GameCamera> m_camera;
// ...
};
```
---
## Implementation Plan
### Phase 1: Interfaces Only
- [ ] Define all interface headers
- [ ] Add to platform abstraction
- [ ] Create mock implementations for testing
### Phase 2: Desktop Implementation
- [ ] MockCamera (test patterns)
- [ ] PortAudio for speaker output
- [ ] Standard filesystem access
- [ ] libcurl for HTTP
### Phase 3: Android Implementation
- [ ] Camera2 API wrapper (JNI)
- [ ] AudioRecord/AudioTrack wrappers
- [ ] Android filesystem with proper sandboxing
- [ ] OkHttp or native networking
### Phase 4: Game Engine Integration
- [ ] Unity RenderTexture → ICamera
- [ ] Unity AudioSource → IMicrophone
- [ ] Unity AudioListener → ISpeaker
- [ ] Unreal equivalents
---
## Dependencies
| Dependency | Purpose | vcpkg Package |
|------------|---------|---------------|
| PortAudio | Desktop audio I/O | `portaudio` |
| OpenCV | Desktop webcam (optional) | `opencv4` |
| libcurl | HTTP client | `curl` |
---
## Test Cases
1. **MockCamera**: Renders test pattern, verify frame callback
2. **FileSystem**: Create, read, write, delete operations
3. **Network**: Mock HTTP responses, verify request/response
4. **Audio**: Verify sample rates, buffer formats
---
## Acceptance Criteria
- [ ] All interfaces defined in kernel/include/
- [ ] Mock implementations work on desktop
- [ ] Camera app can display camera frames
- [ ] Browser app can make HTTP requests
- [ ] Apps can persist data to filesystem

403
docs/MILESTONE-4.md Normal file
View File

@@ -0,0 +1,403 @@
# Milestone 4: App Sandboxing
**Status**: Not Started
**Goal**: Secure app runtime with defined package format and permission system.
---
## Overview
Apps in Mosis need:
- Isolation from each other and the system
- Defined package format for distribution
- Permission model for hardware/data access
- Lifecycle management
---
## Runtime Architecture
### Recommendation: Hybrid Approach
| Component | Runtime | Reason |
|-----------|---------|--------|
| UI Scripts | Lua | Native RmlUi integration, simple |
| App Logic | Lua (now), WASM (future) | Start simple, add WASM for isolation |
| System Services | C++ | Performance, direct hardware access |
### Lua Sandbox
RmlUi already uses Lua for UI scripting. We enhance it with:
```lua
-- Sandboxed globals per app
app = {
id = "com.example.myapp",
storage = AppStorage("com.example.myapp"),
permissions = {"camera", "storage"},
}
-- Restricted stdlib
-- Remove: os.execute, io.popen, loadfile, dofile
-- Keep: string, table, math, coroutine
```
**Sandbox Implementation** (`src/main/kernel/src/lua_sandbox.cpp`):
```cpp
class LuaSandbox {
public:
lua_State* CreateAppState(const std::string& app_id,
const std::vector<std::string>& permissions);
void RestrictGlobals(lua_State* L);
void InjectAppAPIs(lua_State* L, const AppManifest& manifest);
private:
void RemoveDangerousFunctions(lua_State* L);
void SetupPermissionChecks(lua_State* L);
};
```
---
## Package Format (.mpkg)
### Directory Structure
```
myapp.mpkg/
├── manifest.json # Required: metadata, permissions
├── ui/
│ ├── main.rml # Entry point
│ ├── screens/ # Additional screens
│ │ └── settings.rml
│ ├── styles/
│ │ └── app.rcss
│ └── scripts/
│ └── app.lua
├── assets/
│ ├── icon.png # 48x48 app icon
│ ├── icon_large.png # 192x192 for store
│ └── images/
└── locales/ # Optional: i18n
├── en.json
└── es.json
```
### Manifest Schema
**File**: `manifest.json`
```json
{
"$schema": "https://mosis.dev/schemas/manifest-v1.json",
"id": "com.example.myapp",
"name": "My App",
"version": "1.0.0",
"version_code": 1,
"description": "A sample Mosis app",
"author": "Developer Name",
"website": "https://example.com",
"entry": "ui/main.rml",
"icon": "assets/icon.png",
"permissions": [
"camera",
"microphone",
"storage",
"network",
"contacts.read",
"contacts.write"
],
"min_mosis_version": "1.0.0",
"intents": {
"share": {
"types": ["image/*", "text/plain"],
"action": "ui/share.rml"
}
}
}
```
### Manifest TypeScript Interface (for validation)
```typescript
interface MosisManifest {
id: string; // Reverse domain notation
name: string; // Display name
version: string; // SemVer
version_code: number; // Incremental integer
description?: string;
author?: string;
website?: string;
entry: string; // Path to main RML
icon: string; // Path to icon
permissions: Permission[];
min_mosis_version?: string;
intents?: Record<string, Intent>;
}
type Permission =
| "camera"
| "microphone"
| "speaker"
| "storage"
| "network"
| "contacts.read"
| "contacts.write"
| "messages.read"
| "messages.write"
| "location"
| "phone.call"
| "notifications";
```
---
## Permission System
### Permission Levels
| Level | Description | Example |
|-------|-------------|---------|
| Normal | Auto-granted | `storage` (app's own data) |
| Dangerous | User prompt required | `camera`, `contacts.read` |
| Signature | System apps only | `phone.call`, `system.settings` |
### Permission Request Flow
```
App declares permission in manifest
↓
User installs app
↓
On first use of protected API:
↓
┌─────────────────────────────────┐
│ "My App" wants to access your │
│ camera. Allow? │
│ │
│ [Deny] [Allow Once] [Allow] │
└─────────────────────────────────┘
↓
Decision stored in PermissionManager
↓
API call proceeds or fails
```
### Permission Manager
**File**: `src/main/kernel/include/permission_manager.h`
```cpp
namespace mosis {
enum class PermissionStatus {
NotRequested,
Granted,
Denied,
AllowedOnce
};
class IPermissionManager {
public:
virtual ~IPermissionManager() = default;
// Check if app has permission
virtual PermissionStatus Check(const std::string& app_id,
const std::string& permission) = 0;
// Request permission (may show UI)
using PermissionCallback = std::function<void(PermissionStatus)>;
virtual void Request(const std::string& app_id,
const std::string& permission,
PermissionCallback callback) = 0;
// Revoke permission
virtual void Revoke(const std::string& app_id,
const std::string& permission) = 0;
// Get all permissions for app
virtual std::map<std::string, PermissionStatus>
GetAppPermissions(const std::string& app_id) = 0;
};
} // namespace mosis
```
---
## App Lifecycle
### States
```
INSTALLED → LAUNCHING → RUNNING → PAUSED → STOPPED → UNINSTALLED
↑ ↓
└──────────────────────┘
(resume)
```
### Lifecycle Events
```lua
-- In app.lua
function onAppCreate()
-- Initialize app state
end
function onAppResume()
-- Returning from background
end
function onAppPause()
-- Going to background, save state
end
function onAppDestroy()
-- Cleanup
end
```
### App Manager
**File**: `src/main/kernel/include/app_manager.h`
```cpp
namespace mosis {
struct InstalledApp {
std::string id;
std::string name;
std::string version;
std::string icon_path;
std::string install_path;
int64_t installed_time;
};
class IAppManager {
public:
virtual ~IAppManager() = default;
// Installation
virtual bool Install(const std::string& mpkg_path) = 0;
virtual bool Uninstall(const std::string& app_id) = 0;
virtual bool Update(const std::string& mpkg_path) = 0;
// Query
virtual std::vector<InstalledApp> GetInstalledApps() = 0;
virtual std::optional<InstalledApp> GetApp(const std::string& app_id) = 0;
// Lifecycle
virtual bool Launch(const std::string& app_id) = 0;
virtual bool Stop(const std::string& app_id) = 0;
virtual bool IsRunning(const std::string& app_id) = 0;
// Inter-app communication
virtual void SendIntent(const std::string& action,
const std::map<std::string, std::string>& data) = 0;
};
} // namespace mosis
```
---
## Storage Isolation
### Per-App Storage
```cpp
class AppStorage {
public:
AppStorage(const std::string& app_id);
// Key-value storage (like SharedPreferences)
void SetString(const std::string& key, const std::string& value);
std::string GetString(const std::string& key, const std::string& default_value = "");
void SetInt(const std::string& key, int value);
int GetInt(const std::string& key, int default_value = 0);
void SetBool(const std::string& key, bool value);
bool GetBool(const std::string& key, bool default_value = false);
// File storage (app-private)
std::string GetFilesDir();
std::string GetCacheDir();
private:
std::string m_app_id;
std::string m_base_path;
};
```
### Lua API
```lua
-- Key-value storage
app.storage:set("username", "john")
local name = app.storage:get("username", "anonymous")
-- File access (sandboxed)
local data = app.files:read("state.json")
app.files:write("state.json", json.encode(state))
```
---
## Implementation Plan
### Phase 1: Package Format
- [ ] Define manifest schema
- [ ] Create manifest parser/validator
- [ ] Implement .mpkg directory loader
### Phase 2: App Manager
- [ ] Install/uninstall apps
- [ ] App registry (installed apps database)
- [ ] Launch apps from package
### Phase 3: Lua Sandbox
- [ ] Restrict dangerous globals
- [ ] Inject app-specific APIs
- [ ] Per-app Lua state management
### Phase 4: Permission System
- [ ] Permission declaration in manifest
- [ ] Runtime permission checks
- [ ] Permission request UI
### Phase 5: Storage Isolation
- [ ] Per-app directories
- [ ] Key-value storage
- [ ] Quota management
---
## Security Considerations
1. **Lua Sandbox Escape**: Audit all exposed functions
2. **Path Traversal**: Validate all file paths
3. **Memory Limits**: Set Lua memory quotas
4. **CPU Limits**: Timeout long-running scripts
5. **Network Isolation**: Apps only access allowed domains
---
## Acceptance Criteria
- [ ] Apps installable from .mpkg directories
- [ ] Apps launch in isolated Lua environment
- [ ] Permission requests shown to user
- [ ] App data isolated per app
- [ ] Apps can be uninstalled cleanly
- [ ] Store app can browse and install packages

407
docs/MILESTONE-5.md Normal file
View File

@@ -0,0 +1,407 @@
# Milestone 5: WebRTC Bridge
**Status**: Not Started
**Goal**: Cross-device communication via WebRTC for calls, messaging, and file sharing.
---
## Overview
WebRTC enables:
- Voice/video calls between virtual phones
- Text messaging across different games
- File sharing between devices
- Screen sharing
- Connection to real smartphones (companion app)
---
## Architecture
```
┌─────────────────────────────────────────────────────────────────────┐
│ Virtual Phone A │
│ (VR Game on User's PC) │
│ ┌───────────────────────────────────────────────────────────────┐ │
│ │ WebRTCBridge │ │
│ │ ├── DataChannel (messages, files) │ │
│ │ ├── AudioTrack (voice) │ │
│ │ └── VideoTrack (camera/screen share) │ │
│ └───────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
│ WebRTC (UDP/DTLS-SRTP)
│
┌─────────┴─────────┐
│ Signaling Server │
│ (WebSocket) │
└─────────┬─────────┘
│
┌───────────────┼───────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Virtual Phone B │ │ Virtual Phone C │ │ Real Smartphone │
│ (Different Game)│ │ (Same Game) │ │ (Companion App) │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
---
## Dependencies
### libdatachannel
C++ WebRTC library for data channels, audio, and video.
**vcpkg installation**:
```bash
vcpkg install libdatachannel
```
**CMakeLists.txt**:
```cmake
find_package(LibDataChannel CONFIG REQUIRED)
target_link_libraries(mosis-kernel PRIVATE LibDataChannel::LibDataChannel)
```
---
## Core Components
### WebRTC Bridge
**File**: `src/main/kernel/include/webrtc_bridge.h`
```cpp
namespace mosis {
struct PeerInfo {
std::string peer_id;
std::string display_name;
bool is_online;
};
struct CallState {
std::string peer_id;
bool is_active;
bool is_video;
bool is_muted;
int64_t start_time;
};
class IWebRTCBridge {
public:
virtual ~IWebRTCBridge() = default;
// Connection
virtual void Connect(const std::string& signaling_server_url) = 0;
virtual void Disconnect() = 0;
virtual bool IsConnected() const = 0;
// Identity
virtual void SetIdentity(const std::string& phone_id,
const std::string& display_name) = 0;
virtual std::string GetPhoneId() const = 0;
// Peer discovery
virtual std::vector<PeerInfo> GetOnlinePeers() = 0;
// Messaging
using MessageCallback = std::function<void(const std::string& from,
const std::string& message)>;
virtual void SetMessageCallback(MessageCallback callback) = 0;
virtual void SendMessage(const std::string& to, const std::string& message) = 0;
// Voice calls
using CallCallback = std::function<void(const CallState& state)>;
virtual void SetCallCallback(CallCallback callback) = 0;
virtual void StartCall(const std::string& peer_id, bool with_video) = 0;
virtual void AnswerCall(const std::string& peer_id) = 0;
virtual void EndCall(const std::string& peer_id) = 0;
virtual void MuteCall(bool muted) = 0;
// File transfer
using FileTransferCallback = std::function<void(const std::string& from,
const std::string& filename,
const std::vector<uint8_t>& data)>;
virtual void SetFileTransferCallback(FileTransferCallback callback) = 0;
virtual void SendFile(const std::string& to,
const std::string& filename,
const std::vector<uint8_t>& data) = 0;
};
} // namespace mosis
```
### Implementation
**File**: `src/main/kernel/src/webrtc_bridge.cpp`
```cpp
#include <rtc/rtc.hpp>
#include "webrtc_bridge.h"
namespace mosis {
class WebRTCBridgeImpl : public IWebRTCBridge {
public:
WebRTCBridgeImpl() {
rtc::InitLogger(rtc::LogLevel::Warning);
}
void Connect(const std::string& signaling_url) override {
m_ws = std::make_shared<rtc::WebSocket>();
m_ws->onOpen([this]() {
SendSignaling({{"type", "register"}, {"id", m_phone_id}});
});
m_ws->onMessage([this](std::variant<rtc::binary, rtc::string> msg) {
if (auto* str = std::get_if<rtc::string>(&msg)) {
HandleSignaling(nlohmann::json::parse(*str));
}
});
m_ws->open(signaling_url);
}
void SendMessage(const std::string& to, const std::string& message) override {
if (auto it = m_peers.find(to); it != m_peers.end()) {
auto& peer = it->second;
if (peer.data_channel && peer.data_channel->isOpen()) {
nlohmann::json msg = {
{"type", "message"},
{"text", message}
};
peer.data_channel->send(msg.dump());
}
}
}
private:
struct PeerConnection {
std::shared_ptr<rtc::PeerConnection> pc;
std::shared_ptr<rtc::DataChannel> data_channel;
std::shared_ptr<rtc::Track> audio_track;
std::shared_ptr<rtc::Track> video_track;
};
void CreatePeerConnection(const std::string& peer_id) {
rtc::Configuration config;
config.iceServers.emplace_back("stun:stun.l.google.com:19302");
auto pc = std::make_shared<rtc::PeerConnection>(config);
pc->onStateChange([this, peer_id](rtc::PeerConnection::State state) {
// Handle connection state changes
});
pc->onLocalDescription([this, peer_id](rtc::Description desc) {
SendSignaling({
{"type", desc.typeString()},
{"to", peer_id},
{"sdp", std::string(desc)}
});
});
pc->onLocalCandidate([this, peer_id](rtc::Candidate cand) {
SendSignaling({
{"type", "candidate"},
{"to", peer_id},
{"candidate", std::string(cand)}
});
});
m_peers[peer_id] = {pc, nullptr, nullptr, nullptr};
}
void HandleSignaling(const nlohmann::json& msg);
void SendSignaling(const nlohmann::json& msg);
std::string m_phone_id;
std::shared_ptr<rtc::WebSocket> m_ws;
std::map<std::string, PeerConnection> m_peers;
MessageCallback m_message_cb;
CallCallback m_call_cb;
};
} // namespace mosis
```
---
## Signaling Protocol
### Message Types
```json
// Register with server
{"type": "register", "id": "phone_123", "name": "John's Phone"}
// Peer discovery
{"type": "get_peers"}
{"type": "peers", "list": [{"id": "phone_456", "name": "Jane's Phone", "online": true}]}
// WebRTC signaling
{"type": "offer", "to": "phone_456", "sdp": "v=0\r\n..."}
{"type": "answer", "to": "phone_123", "sdp": "v=0\r\n..."}
{"type": "candidate", "to": "phone_456", "candidate": "candidate:..."}
// Call signaling
{"type": "call_request", "to": "phone_456", "video": false}
{"type": "call_accept", "to": "phone_123"}
{"type": "call_reject", "to": "phone_123", "reason": "busy"}
{"type": "call_end", "to": "phone_456"}
```
### Signaling Server
**Simple Node.js reference implementation**:
```javascript
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });
const peers = new Map();
wss.on('connection', (ws) => {
ws.on('message', (data) => {
const msg = JSON.parse(data);
if (msg.type === 'register') {
peers.set(msg.id, { ws, name: msg.name });
broadcast({ type: 'peer_joined', id: msg.id, name: msg.name });
}
else if (msg.to) {
// Forward to recipient
const peer = peers.get(msg.to);
if (peer) {
msg.from = [...peers.entries()].find(([id, p]) => p.ws === ws)?.[0];
peer.ws.send(JSON.stringify(msg));
}
}
});
ws.on('close', () => {
const id = [...peers.entries()].find(([_, p]) => p.ws === ws)?.[0];
if (id) {
peers.delete(id);
broadcast({ type: 'peer_left', id });
}
});
});
function broadcast(msg) {
peers.forEach((peer) => peer.ws.send(JSON.stringify(msg)));
}
```
---
## Lua API
```lua
-- Connect to signaling server
mosis.webrtc.connect("wss://signal.mosis.dev")
-- Set identity
mosis.webrtc.setIdentity("user_12345", "John's Phone")
-- Get online peers
local peers = mosis.webrtc.getPeers()
for _, peer in ipairs(peers) do
print(peer.id, peer.name, peer.online)
end
-- Send message
mosis.webrtc.sendMessage("peer_id", "Hello!")
-- Receive messages
mosis.webrtc.onMessage(function(from, message)
print("Message from " .. from .. ": " .. message)
end)
-- Make a call
mosis.webrtc.call("peer_id", { video = false })
-- Handle incoming call
mosis.webrtc.onIncomingCall(function(from, hasVideo)
-- Show call UI
-- Accept: mosis.webrtc.answerCall(from)
-- Reject: mosis.webrtc.rejectCall(from)
end)
```
---
## Real Smartphone Bridge
### Companion App
A mobile app (Android/iOS) that:
1. Connects to same signaling server
2. Bridges to real phone's contacts, messages
3. Allows receiving calls from virtual phone
4. Sends notifications to virtual phone
**Use Cases**:
- Receive real SMS in virtual phone
- Make real calls from VR
- Sync contacts between real and virtual phone
- Get push notifications in VR
---
## Implementation Plan
### Phase 1: Core WebRTC
- [ ] Add libdatachannel to vcpkg.json
- [ ] Implement WebRTCBridgeImpl
- [ ] Basic signaling protocol
### Phase 2: Messaging
- [ ] Data channel messaging
- [ ] Message history storage
- [ ] Messages app integration
### Phase 3: Voice Calls
- [ ] Audio track setup
- [ ] Microphone/speaker integration
- [ ] Dialer app integration
### Phase 4: Signaling Server
- [ ] Production signaling server
- [ ] User authentication
- [ ] Peer discovery service
### Phase 5: Companion App
- [ ] Android companion app
- [ ] iOS companion app (future)
- [ ] Contact/message sync
---
## Network Requirements
- UDP ports for WebRTC media
- WebSocket for signaling
- TURN server for NAT traversal (optional)
**TURN/STUN Configuration**:
```cpp
rtc::Configuration config;
config.iceServers.emplace_back("stun:stun.l.google.com:19302");
config.iceServers.emplace_back("turn:turn.mosis.dev:3478", "user", "pass");
```
---
## Acceptance Criteria
- [ ] Two virtual phones can exchange messages
- [ ] Voice calls work between virtual phones
- [ ] Files can be transferred between devices
- [ ] Connection survives game restarts (rejoin)
- [ ] Messages app shows real-time chat
- [ ] Dialer app can place/receive calls

500
docs/MILESTONE-6.md Normal file
View File

@@ -0,0 +1,500 @@
# Milestone 6: System Apps
**Status**: 75% Complete
**Goal**: Core smartphone apps with full functionality.
---
## Overview
System apps provide the essential smartphone experience:
- Home launcher
- Phone/Dialer
- Messages
- Contacts
- Settings
- Browser
- Store (TODO)
- Camera (TODO)
- Music (TODO)
---
## App Status
### Completed Apps
| App | Location | Features | Status |
|-----|----------|----------|--------|
| Home | `apps/home/` | App grid, dock, navigation | Complete |
| Dialer | `apps/dialer/` | Keypad, call UI (mock) | Complete |
| Messages | `apps/messages/` | Conversation list, chat | Complete |
| Contacts | `apps/contacts/` | List, search, detail | Complete |
| Settings | `apps/settings/` | Display, sound, about | Complete |
| Browser | `apps/browser/` | URL bar, placeholder | Complete |
### Remaining Apps
| App | Priority | Description |
|-----|----------|-------------|
| Store | High | Browse and install apps |
| Camera | Medium | Viewfinder, capture photos |
| Music | Low | Audio playback |
---
## App: Store
**Location**: `src/main/assets/apps/store/`
### Features
1. **Browse Apps**
- Featured apps carousel
- Categories (Games, Utilities, Social)
- Search functionality
2. **App Details**
- Name, icon, description
- Screenshots
- Permissions list
- Install/Update button
3. **My Apps**
- Installed apps list
- Update available indicator
- Uninstall option
### UI Screens
```
store/
├── store.rml # Main store screen
├── store.rcss # Store styles
├── category.rml # Category listing
├── detail.rml # App detail page
└── scripts/
└── store.lua # Store logic
```
### Main Screen (`store.rml`)
```html
<rml>
<head>
<link type="text/rcss" href="store.rcss"/>
<link type="text/rcss" href="../../ui/theme.rcss"/>
</head>
<body class="store-screen">
<div class="app-bar">
<div class="app-bar-nav btn-icon" onclick="goBack()">
<img src="../../icons/back.tga"/>
</div>
<span class="app-bar-title">Store</span>
<div class="app-bar-action btn-icon" onclick="openSearch()">
<img src="../../icons/search.tga"/>
</div>
</div>
<div class="store-content">
<!-- Featured carousel -->
<div class="featured-section">
<h2>Featured</h2>
<div class="featured-carousel" data-for="app : featured_apps">
<div class="featured-card" data-event-click="showDetail(app.id)">
<img class="featured-banner" data-attr-src="app.banner"/>
<span class="featured-name" data-text="app.name"/>
</div>
</div>
</div>
<!-- Categories -->
<div class="category-section">
<h2>Categories</h2>
<div class="category-grid">
<div class="category-item" onclick="showCategory('games')">
<img src="../../icons/games.tga"/>
<span>Games</span>
</div>
<div class="category-item" onclick="showCategory('utilities')">
<img src="../../icons/tools.tga"/>
<span>Utilities</span>
</div>
</div>
</div>
<!-- Top Apps -->
<div class="top-apps-section">
<h2>Top Apps</h2>
<div class="app-list" data-for="app : top_apps">
<div class="app-list-item" data-event-click="showDetail(app.id)">
<img class="app-icon" data-attr-src="app.icon"/>
<div class="app-info">
<span class="app-name" data-text="app.name"/>
<span class="app-category" data-text="app.category"/>
</div>
<button class="install-btn" data-event-click="install(app.id)">
Install
</button>
</div>
</div>
</div>
</div>
</body>
</rml>
```
### Data Model
```cpp
// In data_models.cpp
struct StoreApp {
std::string id;
std::string name;
std::string icon;
std::string banner;
std::string category;
std::string description;
std::string version;
std::vector<std::string> screenshots;
std::vector<std::string> permissions;
bool installed;
};
void setupStoreDataModel(Rml::Context* context) {
auto model = context->CreateDataModel("store");
model.Bind("featured_apps", &g_featured_apps);
model.Bind("top_apps", &g_top_apps);
model.Bind("categories", &g_categories);
model.Bind("current_app", &g_current_app);
model.BindEventCallback("install", [](auto& event, auto& args) {
// Install app
});
}
```
---
## App: Camera
**Location**: `src/main/assets/apps/camera/`
### Features
1. **Viewfinder**
- Live camera preview (from ICamera)
- Capture button
- Switch camera (front/back)
- Flash toggle
2. **Capture**
- Take photo
- Save to gallery
- Share option
3. **Gallery**
- View captured photos
- Delete photos
- Share photos
### UI Screens
```
camera/
├── camera.rml # Viewfinder
├── camera.rcss # Camera styles
├── gallery.rml # Photo gallery
└── scripts/
└── camera.lua # Camera logic
```
### Viewfinder (`camera.rml`)
```html
<rml>
<head>
<link type="text/rcss" href="camera.rcss"/>
</head>
<body class="camera-screen">
<!-- Camera preview (texture from ICamera) -->
<div id="camera-preview">
<img id="preview-frame" data-attr-src="camera_frame"/>
</div>
<!-- Top controls -->
<div class="camera-top-bar">
<div class="btn-icon" onclick="goBack()">
<img src="../../icons/close.tga"/>
</div>
<div class="btn-icon" onclick="toggleFlash()">
<img id="flash-icon" src="../../icons/flash_off.tga"/>
</div>
</div>
<!-- Bottom controls -->
<div class="camera-bottom-bar">
<div class="btn-icon" onclick="openGallery()">
<img id="last-photo" data-attr-src="last_photo_thumb"/>
</div>
<div id="capture-btn" onclick="capture()">
<div class="capture-ring"/>
</div>
<div class="btn-icon" onclick="switchCamera()">
<img src="../../icons/flip_camera.tga"/>
</div>
</div>
</body>
</rml>
```
### Camera Lua Script
```lua
-- camera.lua
local camera = mosis.platform.getCamera()
local capture = mosis.testing.VisualCapture(540, 960)
local is_front_camera = false
local flash_on = false
function onAppCreate()
if camera:isAvailable() then
camera:startCapture(function(frame)
-- Update preview texture
document:GetElementById("preview-frame"):SetAttribute("src", frame.texture_url)
end)
else
-- Show "no camera" message
end
end
function capture()
local path = mosis.filesystem:getSharedMediaPath() .. "/photos/" .. os.time() .. ".png"
capture:CaptureScreenshot(path)
-- Show capture animation
playSound("shutter")
flashScreen()
-- Update last photo thumbnail
document:GetElementById("last-photo"):SetAttribute("src", path)
end
function switchCamera()
is_front_camera = not is_front_camera
-- camera:setFacing(is_front_camera and "front" or "back")
end
function toggleFlash()
flash_on = not flash_on
local icon = flash_on and "flash_on" or "flash_off"
document:GetElementById("flash-icon"):SetAttribute("src", "../../icons/" .. icon .. ".tga")
end
function openGallery()
navigateTo("camera/gallery")
end
```
---
## App: Music
**Location**: `src/main/assets/apps/music/`
### Features
1. **Library**
- Songs list
- Albums
- Artists
- Playlists
2. **Player**
- Play/pause
- Next/previous
- Seek bar
- Volume control
- Shuffle/repeat
3. **Now Playing**
- Album art
- Song info
- Progress bar
### UI Screens
```
music/
├── music.rml # Library view
├── music.rcss # Music styles
├── player.rml # Now playing
├── playlist.rml # Playlist view
└── scripts/
└── music.lua # Player logic
```
---
## Data Persistence
### Storage Layer
Apps need persistent storage for:
- Contacts
- Messages
- Settings
- Photos
**Implementation Options**:
1. **JSON Files** (Simple)
```
/data/contacts.json
/data/messages.json
/data/settings.json
```
2. **SQLite** (Robust)
```
/data/mosis.db
- contacts table
- messages table
- settings table
```
### Contact Storage
```cpp
// contact_storage.h
struct Contact {
std::string id;
std::string name;
std::string phone;
std::string email;
std::string avatar;
};
class ContactStorage {
public:
std::vector<Contact> GetAll();
std::optional<Contact> GetById(const std::string& id);
bool Save(const Contact& contact);
bool Delete(const std::string& id);
std::vector<Contact> Search(const std::string& query);
};
```
### Message Storage
```cpp
// message_storage.h
struct Message {
std::string id;
std::string conversation_id;
std::string sender;
std::string text;
int64_t timestamp;
bool read;
};
struct Conversation {
std::string id;
std::string contact_id;
std::string last_message;
int64_t last_timestamp;
int unread_count;
};
class MessageStorage {
public:
std::vector<Conversation> GetConversations();
std::vector<Message> GetMessages(const std::string& conversation_id);
bool SaveMessage(const Message& message);
bool MarkAsRead(const std::string& conversation_id);
};
```
---
## Implementation Plan
### Phase 1: Store App UI
- [ ] Main store screen layout
- [ ] Category browsing
- [ ] App detail page
- [ ] Mock data for testing
### Phase 2: Camera App
- [ ] Viewfinder UI
- [ ] Capture to file
- [ ] Gallery view
- [ ] Integration with ICamera
### Phase 3: Music App
- [ ] Library UI
- [ ] Player UI
- [ ] Audio playback (stub)
### Phase 4: Data Persistence
- [ ] JSON storage layer
- [ ] Contact CRUD
- [ ] Message storage
- [ ] Settings persistence
### Phase 5: Real Functionality
- [ ] Store: Install real .mpkg files
- [ ] Camera: Real camera frames
- [ ] Music: Audio playback
---
## Testing
### Test IDs for Store
| ID | Element |
|----|---------|
| `store-search` | Search button |
| `store-featured` | Featured carousel |
| `store-categories` | Category grid |
| `app-install-btn` | Install button on detail |
### Test IDs for Camera
| ID | Element |
|----|---------|
| `camera-preview` | Preview area |
| `capture-btn` | Capture button |
| `gallery-btn` | Gallery button |
| `switch-camera-btn` | Switch camera |
---
## Acceptance Criteria
### Store
- [ ] Browse featured and top apps
- [ ] View app details
- [ ] See permission requirements
- [ ] Install apps (mock or real)
### Camera
- [ ] Display camera preview
- [ ] Capture photos
- [ ] View in gallery
- [ ] Share photos
### Music
- [ ] Display music library
- [ ] Play/pause audio
- [ ] Show now playing
### Persistence
- [ ] Contacts persist across sessions
- [ ] Messages persist across sessions
- [ ] Settings persist across sessions

496
docs/MILESTONE-7.md Normal file
View File

@@ -0,0 +1,496 @@
# Milestone 7: Game Integration
**Status**: Not Started
**Goal**: Production-ready Unity and Unreal plugins for seamless VR phone integration.
---
## Overview
Game engine plugins enable:
- Rendering the phone in VR scenes
- Touch/raycast interaction
- Virtual hardware provision (camera, mic, speaker)
- Event callbacks to game code
---
## Current State
| Platform | Location | Status |
|----------|----------|--------|
| Unity | `D:\Dev\Mosis\Mosis Unity` | Basic Binder client |
| Unreal | `D:\Dev\Mosis\Mosis Unreal` | WIP |
---
## Unity Plugin
### Package Structure
```
com.mosis.phone/
├── package.json
├── Runtime/
│ ├── MosisPhone.cs # Main component
│ ├── MosisService.cs # Binder client
│ ├── MosisInputHandler.cs # Touch/raycast
│ ├── MosisHardwareProvider.cs # Virtual hardware
│ └── Native/
│ ├── MosisNative.cs # P/Invoke declarations
│ └── Plugins/
│ └── Android/
│ └── libmosis-client.so
├── Prefabs/
│ ├── MosisPhone.prefab # Ready-to-use phone
│ └── MosisPhoneVR.prefab # VR-optimized version
├── Samples~/
│ └── BasicIntegration/
│ └── PhoneDemo.unity
└── Documentation~/
└── integration-guide.md
```
### Main Component
**File**: `MosisPhone.cs`
```csharp
using UnityEngine;
using UnityEngine.Events;
namespace Mosis {
public class MosisPhone : MonoBehaviour {
[Header("Display")]
[SerializeField] private MeshRenderer phoneScreen;
[SerializeField] private Vector2Int resolution = new Vector2Int(540, 960);
[Header("Input")]
[SerializeField] private bool enableRaycast = true;
[SerializeField] private LayerMask raycastLayers;
[Header("Virtual Hardware")]
[SerializeField] private Camera virtualCamera;
[SerializeField] private AudioSource virtualMicrophone;
[SerializeField] private AudioSource virtualSpeaker;
[Header("Events")]
public UnityEvent onPhoneReady;
public UnityEvent<string> onNavigate;
public UnityEvent<string, string> onMessageReceived;
public UnityEvent<string> onCallStarted;
private MosisService service;
private RenderTexture screenTexture;
private Material screenMaterial;
void Awake() {
// Create render texture for phone screen
screenTexture = new RenderTexture(resolution.x, resolution.y, 0);
screenMaterial = new Material(Shader.Find("Unlit/Texture"));
screenMaterial.mainTexture = screenTexture;
phoneScreen.material = screenMaterial;
}
void Start() {
service = new MosisService();
service.OnFrameAvailable += OnFrameAvailable;
service.OnServiceInitialized += () => onPhoneReady?.Invoke();
service.Connect();
}
void Update() {
if (enableRaycast) {
HandleRaycastInput();
}
}
private void HandleRaycastInput() {
// VR controller raycast
if (OVRInput.GetDown(OVRInput.Button.PrimaryIndexTrigger)) {
var ray = new Ray(
OVRInput.GetLocalControllerPosition(OVRInput.Controller.RTouch),
OVRInput.GetLocalControllerRotation(OVRInput.Controller.RTouch) * Vector3.forward
);
if (Physics.Raycast(ray, out var hit, 10f, raycastLayers)) {
if (hit.collider.gameObject == phoneScreen.gameObject) {
var uv = hit.textureCoord;
var x = uv.x * resolution.x;
var y = (1 - uv.y) * resolution.y;
service.SendTouchDown(x, y);
}
}
}
}
private void OnFrameAvailable(byte[] pixels) {
// Update screen texture
screenTexture.LoadRawTextureData(pixels);
screenTexture.Apply();
}
// Public API
public void NavigateTo(string screen) {
service.Navigate(screen);
}
public void SendMessage(string to, string text) {
service.SendMessage(to, text);
}
public void MakeCall(string number) {
service.MakeCall(number);
}
void OnDestroy() {
service?.Disconnect();
Destroy(screenTexture);
Destroy(screenMaterial);
}
}
}
```
### Hardware Provider
**File**: `MosisHardwareProvider.cs`
```csharp
namespace Mosis {
public class MosisHardwareProvider : MonoBehaviour {
[SerializeField] private MosisPhone phone;
[SerializeField] private Camera gameCamera;
[SerializeField] private AudioListener audioListener;
private RenderTexture cameraTexture;
void Start() {
// Setup virtual camera
cameraTexture = new RenderTexture(640, 480, 0);
gameCamera.targetTexture = cameraTexture;
// Register with phone service
phone.Service.SetCameraProvider(() => {
var pixels = new byte[cameraTexture.width * cameraTexture.height * 4];
// Read pixels from RenderTexture
return pixels;
});
phone.Service.SetMicrophoneProvider(() => {
// Get audio from AudioListener
return audioSamples;
});
}
public void PlayAudio(float[] samples) {
// Play on virtual speaker
var audioSource = GetComponent<AudioSource>();
audioSource.clip = AudioClip.Create("phone", samples.Length, 1, 44100, false);
audioSource.clip.SetData(samples, 0);
audioSource.Play();
}
}
}
```
### VR Interaction
**File**: `MosisInputHandler.cs`
```csharp
namespace Mosis {
public class MosisInputHandler : MonoBehaviour {
[SerializeField] private MosisPhone phone;
[SerializeField] private Transform pointerOrigin;
[SerializeField] private LineRenderer laserPointer;
private bool isTouching;
private Vector2 lastTouchPos;
void Update() {
// Update laser pointer
var ray = new Ray(pointerOrigin.position, pointerOrigin.forward);
if (Physics.Raycast(ray, out var hit, 10f)) {
laserPointer.SetPosition(1, hit.point);
if (hit.collider.GetComponent<MosisPhone>() != null) {
var uv = hit.textureCoord;
var touchPos = new Vector2(
uv.x * phone.Resolution.x,
(1 - uv.y) * phone.Resolution.y
);
HandleTouch(touchPos);
}
}
}
private void HandleTouch(Vector2 pos) {
bool triggerDown = OVRInput.Get(OVRInput.Button.PrimaryIndexTrigger);
if (triggerDown && !isTouching) {
phone.Service.SendTouchDown(pos.x, pos.y);
isTouching = true;
}
else if (triggerDown && isTouching) {
if (Vector2.Distance(pos, lastTouchPos) > 2f) {
phone.Service.SendTouchMove(pos.x, pos.y);
}
}
else if (!triggerDown && isTouching) {
phone.Service.SendTouchUp(pos.x, pos.y);
isTouching = false;
}
lastTouchPos = pos;
}
}
}
```
---
## Unreal Plugin
### Module Structure
```
MosisPhone/
├── MosisPhone.uplugin
├── Source/
│ ├── MosisPhone/
│ │ ├── MosisPhone.Build.cs
│ │ ├── Public/
│ │ │ ├── MosisPhoneActor.h
│ │ │ ├── MosisService.h
│ │ │ └── MosisHardwareProvider.h
│ │ └── Private/
│ │ ├── MosisPhoneActor.cpp
│ │ ├── MosisService.cpp
│ │ └── MosisHardwareProvider.cpp
│ └── ThirdParty/
│ └── MosisClient/
│ └── libmosis-client.so
├── Content/
│ ├── BP_MosisPhone.uasset
│ └── M_PhoneScreen.uasset
└── Documentation/
└── integration-guide.md
```
### Main Actor
**File**: `MosisPhoneActor.h`
```cpp
#pragma once
#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "MosisPhoneActor.generated.h"
DECLARE_DYNAMIC_MULTICAST_DELEGATE(FOnPhoneReady);
DECLARE_DYNAMIC_MULTICAST_DELEGATE_OneParam(FOnNavigate, FString, Screen);
UCLASS()
class MOSISPHONE_API AMosisPhoneActor : public AActor {
GENERATED_BODY()
public:
AMosisPhoneActor();
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Display")
FIntPoint Resolution = FIntPoint(540, 960);
UPROPERTY(BlueprintAssignable, Category = "Events")
FOnPhoneReady OnPhoneReady;
UPROPERTY(BlueprintAssignable, Category = "Events")
FOnNavigate OnNavigate;
UFUNCTION(BlueprintCallable, Category = "Mosis")
void NavigateTo(const FString& Screen);
UFUNCTION(BlueprintCallable, Category = "Mosis")
void SendMessage(const FString& To, const FString& Text);
UFUNCTION(BlueprintCallable, Category = "Mosis")
void MakeCall(const FString& Number);
protected:
virtual void BeginPlay() override;
virtual void Tick(float DeltaTime) override;
virtual void EndPlay(const EEndPlayReason::Type EndPlayReason) override;
private:
UPROPERTY()
UTextureRenderTarget2D* ScreenTexture;
UPROPERTY()
UMaterialInstanceDynamic* ScreenMaterial;
TSharedPtr<FMosisService> Service;
void HandleRaycastInput();
void OnFrameAvailable(const TArray<uint8>& Pixels);
};
```
### Blueprint Integration
```cpp
// Blueprint callable functions for easy integration
UFUNCTION(BlueprintCallable, Category = "Mosis", meta = (WorldContext = "WorldContextObject"))
static AMosisPhoneActor* SpawnMosisPhone(
UObject* WorldContextObject,
FTransform SpawnTransform,
FIntPoint Resolution = FIntPoint(540, 960)
);
```
---
## Hardware Button Simulation
### Physical Buttons
| Button | Function |
|--------|----------|
| Power | Lock/wake screen |
| Volume Up | Increase volume |
| Volume Down | Decrease volume |
| Home (soft) | Go to home screen |
| Back (soft) | Navigation back |
### Unity Implementation
```csharp
public class MosisPhoneButtons : MonoBehaviour {
[SerializeField] private MosisPhone phone;
[SerializeField] private Collider powerButton;
[SerializeField] private Collider volumeUp;
[SerializeField] private Collider volumeDown;
void Update() {
// Check for button presses via physics overlap
if (IsButtonPressed(powerButton)) {
phone.Service.SendButton("power");
}
// ...
}
}
```
### Unreal Implementation
```cpp
// In Blueprint or C++
UFUNCTION(BlueprintCallable)
void PressButton(EMosisButton Button) {
switch (Button) {
case EMosisButton::Power:
Service->SendButton("power");
break;
case EMosisButton::VolumeUp:
Service->SendButton("volume_up");
break;
case EMosisButton::VolumeDown:
Service->SendButton("volume_down");
break;
}
}
```
---
## Implementation Plan
### Phase 1: Unity Core
- [ ] Package structure
- [ ] MosisPhone component
- [ ] Basic touch input
- [ ] Frame rendering
### Phase 2: Unity VR
- [ ] VR raycast interaction
- [ ] Laser pointer visualization
- [ ] Controller haptics
- [ ] Two-handed support
### Phase 3: Unity Hardware
- [ ] Camera provider (RenderTexture)
- [ ] Microphone provider
- [ ] Speaker output
- [ ] Vibration
### Phase 4: Unreal Core
- [ ] Plugin structure
- [ ] MosisPhoneActor
- [ ] Blueprint integration
- [ ] Touch input
### Phase 5: Unreal VR
- [ ] Motion controller input
- [ ] Widget interaction
- [ ] Hardware providers
### Phase 6: Documentation
- [ ] Quick start guide
- [ ] API reference
- [ ] Sample scenes
- [ ] Troubleshooting
---
## Testing
### Unity Test Scene
1. Phone mounted in VR scene
2. Touch interaction works
3. Virtual camera shows game view
4. Audio plays through virtual speaker
### Unreal Test Level
1. Blueprint phone actor
2. Motion controller interaction
3. Event callbacks work
4. Performance acceptable
---
## Performance Targets
| Metric | Target |
|--------|--------|
| Frame latency | < 16ms |
| Touch latency | < 50ms |
| Memory usage | < 100MB |
| Draw calls | < 5 per phone |
---
## Acceptance Criteria
### Unity
- [ ] One-click prefab placement
- [ ] VR raycast touch works
- [ ] Events fire correctly
- [ ] Camera feed from game
- [ ] Audio bidirectional
### Unreal
- [ ] Blueprint spawnable
- [ ] Motion controller support
- [ ] Blueprint events
- [ ] Hardware providers
### Documentation
- [ ] Integration guide
- [ ] API docs
- [ ] Example projects
- [ ] Video tutorial

165
docs/PLAN.md Normal file
View File

@@ -0,0 +1,165 @@
# MosisService Implementation Plan
## Overview
This document outlines the implementation plan for adding Lua scripting support to the MosisService project, enabling interactive UI elements that can execute Lua code.
## Project Analysis
### Current Architecture
The project consists of:
1. Kotlin-based Android UI components
2. Two native C++ libraries (`mosis-service` and `mosis-test`)
3. RmlUi-based UI rendering engine
4. Android Binder service integration
5. OpenGL ES 2.0 rendering pipeline
### Build System
- Android Studio with Gradle
- CMake build system
- Android NDK (version 29.0.14206865)
- vcpkg for dependency management
## Implementation Steps
### Phase 1: Enable Lua Support in Build System
1. **Update vcpkg.json**
- Verify RmlUi package includes Lua support
- Add necessary Lua dependency if not included
2. **Modify CMakeLists.txt**
- Add RmlUi Lua binding options
- Configure proper linking with Lua libraries
- Set compiler flags for maximum feedback
### Phase 2: Integrate Lua in Core Engine
1. **Update kernel.cpp**
- Include RmlUi/Lua.h header
- Add Lua initialization call after main RmlUi initialization
- Register C++ functions accessible from Lua scripts
2. **Configure Build with Debug Flags**
- Enable all compiler warnings
- Add debugging symbols
- Enable sanitizers for memory and undefined behavior checks
### Phase 3: UI Integration
1. **Modify demo.rml**
- Add center button element
- Position button using CSS transforms
- Implement onclick handler that calls registered Lua function
2. **Add CSS Styles**
- Create centered positioning for UI elements
- Style the button appropriately
- Ensure proper z-index for overlay elements
### Phase 4: Script Execution
1. **Create Lua Function Registration**
- Register C++ functions that interact with Android services
- Implement callback mechanisms for UI events
- Handle error cases in script execution
### Phase 5: Testing and Validation
1. **Verify Build with Debug Feedback**
- Compile with all warnings enabled
- Test compilation with sanitizers
- Validate debug symbol generation
2. **Functional Testing**
- Test button click functionality
- Verify Lua script execution
- Validate UI updates from script execution
## Specific Technical Requirements
### CMake Configuration Changes
In `CMakeLists.txt`:
```
# Enable Lua bindings for RmlUi
# Add after find_package(RmlUi CONFIG REQUIRED)
set(RMLUI_LUA_BINDINGS ON)
set(RML_LUA_BINDINGS_LIBRARY lua_as_cxx)
# Link with Lua libraries if needed
# target_link_libraries(mosis-service RmlUi::RmlUi RmlUi::Lua)
```
### Kernel Integration
In `kernel.cpp`:
```cpp
#include <RmlUi/Lua.h>
// In main_loop() after Rml::Initialise():
Rml::Lua::Initialise();
// Register functions that can be called from RML
// Example of registering a basic callback
Rml::Lua::RegisterGlobalFunction("handleButtonClick", []() {
Logger::Log("Button clicked via Lua!");
// Add actual implementation
});
```
### RML Button Implementation
In `demo.rml`:
```xml
<div id="center-button-container">
<button id="trigger-button" onclick="handleButtonClick()">Execute Script</button>
</div>
```
### CSS Styling
Add to CSS:
```css
#center-button-container {
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
z-index: 1000;
}
#trigger-button {
font-size: 2em;
padding: 20px 40px;
background-color: #3498db;
color: white;
border: none;
border-radius: 10px;
cursor: pointer;
text-align: center;
}
```
## Build Commands for Maximum Feedback
### Debug Build with All Warnings
```
./gradlew clean && ./gradlew assembleDebug
--warning-mode all
--info
--stacktrace
-Pandroid.jetifier.enable=true
```
### Compilation with Sanitizers
```
cmake -DCMAKE_BUILD_TYPE=Debug
-DCMAKE_CXX_FLAGS_DEBUG="-g -O0 -Wall -Wextra -Wpedantic -Werror -fsanitize=address,undefined"
.
```
## Expected Outcome
After implementation, the UI will:
1. Display a center button in the demo interface
2. Allow button clicks to trigger Lua scripts
3. Execute registered C++ functions through Lua
4. Provide full build feedback with debug symbols
5. Support all compiler warnings and sanitization
## Validation Steps
1. Successful compilation with no errors or warnings
2. UI renders with centered button
3. Button click executes Lua script successfully
4. Debug symbols present in native libraries
5. No memory leaks detected by sanitizers

463
docs/ROADMAP.md Normal file
View File

@@ -0,0 +1,463 @@
# Mosis Virtual Smartphone Platform - Roadmap
## Vision
Mosis is a **virtual smartphone OS** for VR games and applications. It provides a phone-like device that users interact with inside VR environments, running virtual apps with real smartphone functionality.
### Key Use Cases
1. **In-Game Phone**: Players use a virtual phone in VR games for communication, utilities, entertainment
2. **Cross-Game Communication**: Phone identity persists across different games (unified contacts, messages)
3. **Real-World Bridge**: Connect to real smartphones via WebRTC for mixed-reality communication
4. **Virtual Hardware**: Game engine provides camera textures, audio, and other "hardware" to the phone OS
5. **Store Ecosystem**: Downloadable apps from a store, self-contained packages with isolation
### Integration Points
| Platform | Location | Status |
|----------|----------|--------|
| Unity Plugin | `D:\Dev\Mosis\Mosis Unity` | Binder client, raycast, viewport |
| Unreal Plugin | `D:\Dev\Mosis\Mosis Unreal` | WIP |
| Desktop Designer | `MosisService/designer/` | Complete |
| Designer Tests | `MosisService/designer-test/` | Mostly complete |
---
## Milestones Overview
| # | Milestone | Status | Description |
|---|-----------|--------|-------------|
| 1 | Cross-Platform Kernel | ✅ Complete | Desktop designer with shared kernel code |
| 2 | Testing Framework | ✅ Complete | Automated UI testing and inspection |
| 3 | Virtual Hardware | ❌ Not started | Camera, mic, speaker, filesystem APIs |
| 4 | App Sandboxing | ❌ Not started | Lua/WASM runtime, package format |
| 5 | WebRTC Bridge | ❌ Not started | Phone-to-phone communication |
| 6 | System Apps | 🔶 75% | Core phone apps |
| 7 | Game Integration | ❌ Not started | Unity/Unreal plugin polish |
---
## Milestone 1: Cross-Platform Kernel ✅ COMPLETE
**Goal**: Desktop designer with shared kernel code for rapid UI iteration.
### Completed Tasks
- [x] Platform abstraction layer (`src/main/kernel/include/`)
- `platform.h` - IPlatform, IGraphicsContext, IRenderTarget
- `service_interface.h` - IKernel, IServiceListener
- `file_interface.h` - IFileInterface extending Rml::FileInterface
- [x] Desktop designer project (`designer/`)
- GLFW + OpenGL 3.3 context
- Hot-reload on RML/RCSS/Lua changes
- Mouse input maps to touch events
- [x] UI designs imported from MosisDesigner
- Theme and component stylesheets
- Navigation system (Lua)
- Fonts and icons
- [x] Android build unchanged and working
---
## Milestone 2: Testing Automation ✅ COMPLETE
**Goal**: Automated UI testing for rapid iteration and AI agent verification.
### Completed Tasks
- [x] UI hierarchy dumping to JSON
- [x] PNG screenshot capture (F12 key)
- [x] Agent-compatible JSON test results
- [x] `designer-test` executable
- WindowController (Windows SendInput API)
- HierarchyReader (element lookup by ID/class)
- LogParser (navigation event verification)
- [x] All 5 navigation tests passing
- [x] Android event injection via ADB broadcast
- [x] **Action Recording** - Capture tap, swipe, long_press to JSON with timestamps
- [x] **Action Playback** - Load recorded JSON and replay with timing
- [x] **Screenshot Diff** - Pixel-level PNG comparison with configurable tolerance
- [x] **GLFW Input Hooks** - Automatic mouse/key recording via forked backend
### Test Recording Format
```json
{
"name": "Navigate to contacts and back",
"recorded_at": "2024-01-16T12:00:00Z",
"resolution": {"width": 677, "height": 1202},
"actions": [
{"type": "tap", "x": 413, "y": 1174, "timestamp": 0},
{"type": "wait", "duration": 1000, "timestamp": 100},
{"type": "tap", "x": 40, "y": 28, "timestamp": 1100},
{"type": "wait", "duration": 500, "timestamp": 1200}
],
"assertions": [
{"after_action": 0, "type": "screen_is", "expected": "contacts"},
{"after_action": 2, "type": "screen_is", "expected": "home"}
]
}
```
---
## Milestone 3: Virtual Hardware ❌ NOT STARTED
**Goal**: Hardware-like APIs backed by game engine or real devices.
### 3.1 Camera Interface
```cpp
class ICamera {
virtual void RequestFrame(FrameCallback callback) = 0;
virtual void SetResolution(int width, int height) = 0;
virtual bool IsAvailable() const = 0;
};
```
**Implementations**:
- **Game Mode**: Receives texture from Unity/Unreal
- **Desktop Mode**: System webcam (optional)
- **Android Test Mode**: SharedTexture from MainActivity
- **Mock Mode**: Test patterns
### 3.2 Microphone Interface
```cpp
class IMicrophone {
virtual void StartCapture(AudioCallback callback) = 0;
virtual void StopCapture() = 0;
virtual bool IsAvailable() const = 0;
};
```
### 3.3 Speaker Interface
```cpp
class ISpeaker {
virtual void PlayAudio(AudioBuffer buffer) = 0;
virtual void SetVolume(float volume) = 0;
};
```
### 3.4 Filesystem Interface
```cpp
class IFileSystem {
virtual FileHandle Open(const std::string& path, Mode mode) = 0;
virtual std::vector<std::string> List(const std::string& dir) = 0;
virtual bool CreateDirectory(const std::string& path) = 0;
virtual bool Delete(const std::string& path) = 0;
};
```
### 3.5 Network Interface
```cpp
class INetwork {
virtual HttpResponse Fetch(const HttpRequest& request) = 0;
virtual WebSocket Connect(const std::string& url) = 0;
virtual bool IsOnline() const = 0;
};
```
---
## Milestone 4: App Sandboxing ❌ NOT STARTED
**Goal**: Secure app runtime with defined package format.
### 4.1 Runtime Decision
| Aspect | Lua | WASM |
|--------|-----|------|
| Isolation | Weak | Strong |
| Performance | Good for UI | Near-native |
| RmlUi Integration | Native | Needs bridge |
| Ecosystem | Small | Large |
**Recommendation**: Hybrid approach
- Lua for UI scripting (RmlUi integration)
- WASM for app logic needing isolation (future)
### 4.2 Package Format (.mpkg)
```
myapp.mpkg/
├── manifest.json # Metadata, permissions, entry
├── ui/
│ ├── main.rml
│ ├── styles.rcss
│ └── scripts/
├── assets/
│ ├── icons/
│ └── images/
└── data/
```
### 4.3 Manifest Schema
```json
{
"id": "com.example.myapp",
"name": "My App",
"version": "1.0.0",
"permissions": ["camera", "network", "storage"],
"entry": "ui/main.rml",
"icon": "assets/icons/app.png"
}
```
### 4.4 App Lifecycle
```
INSTALLED → LAUNCHING → RUNNING → PAUSED → STOPPED → UNINSTALLED
```
---
## Milestone 5: WebRTC Bridge ❌ NOT STARTED
**Goal**: Cross-device communication via WebRTC.
### 5.1 Dependencies
- libdatachannel (add to vcpkg)
### 5.2 Communication Protocol
```json
{
"type": "call" | "message" | "file" | "screen_share",
"from": "phone_id",
"to": "phone_id",
"payload": { ... }
}
```
### 5.3 Use Cases
- Voice/video calls between virtual phones
- Text messaging across games
- File sharing
- Screen sharing
### 5.4 Components
- **WebRTCBridge** class for data channels
- **Signaling Server** (WebSocket-based)
- **Real Smartphone Bridge** (companion app)
---
## Milestone 6: System Apps 🔶 75% COMPLETE
### Completed Apps
| App | Features | Status |
|-----|----------|--------|
| Home | App grid, dock, navigation | ✅ Complete |
| Dialer | Keypad, call UI (mock) | ✅ Complete |
| Messages | Conversation list, chat view | ✅ Complete |
| Contacts | Contact list, search, detail | ✅ Complete |
| Settings | Display, sound, about | ✅ Complete |
| Browser | URL bar, placeholder | ✅ Complete |
### Remaining Apps
| App | Features | Status |
|-----|----------|--------|
| Store | Browse apps, install packages | ❌ UI only |
| Camera | Viewfinder, capture, gallery | ❌ UI only |
| Music/Audio | Playback, playlists | ❌ Outline only |
### App Data Persistence (Future)
- Contacts: JSON or SQLite storage
- Messages: Conversation history
- Settings: Preferences file
- Photos: Virtual filesystem
---
## Milestone 7: Game Integration ❌ NOT STARTED
**Goal**: Production-ready game engine plugins.
### 7.1 Unity Package
- [ ] Improved raycast interaction
- [ ] Hardware button simulation (volume, power)
- [ ] Virtual camera provider (RenderTexture → phone)
- [ ] Virtual microphone provider
- [ ] Virtual speaker provider
### 7.2 Unreal Plugin
- [ ] Full Binder client implementation
- [ ] Blueprint integration
- [ ] Same features as Unity
### 7.3 Documentation
- [ ] How to embed phone in game scene
- [ ] How to provide virtual hardware
- [ ] How to handle phone events
---
## Extended Features (Future)
| Feature | Description |
|---------|-------------|
| Notifications | System notification center, per-app channels |
| Widgets | Home screen widgets, live data |
| Gestures | Swipe up for home, swipe down for notifications |
| Themes | Dark/light mode, custom colors |
| Accessibility | Font size, high contrast, TTS |
| Multi-Window | Split-screen, picture-in-picture |
| Clipboard | Copy/paste between apps and game |
| Voice Assistant | Wake word, voice commands |
| Virtual Location | GPS from game, map integration |
---
## Architecture
```
┌─────────────────────────────────────────────────────────────────────┐
│ VR Game (Unity/Unreal) │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │
│ │ Phone Render │ │ Touch/Button │ │ Virtual Hardware Provider │ │
│ │ Viewport │ │ Input │ │ (Camera, Mic, Speakers) │ │
│ └──────┬───────┘ └──────┬───────┘ └────────────┬─────────────┘ │
└─────────┼─────────────────┼────────────────────────┼────────────────┘
│ Binder IPC │ Binder IPC │ Binder IPC
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────┐
│ Mosis Service │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Kernel │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ │
│ │ │ RmlUi │ │ App Runtime │ │ Virtual Hardware │ │ │
│ │ │ Renderer │ │ (Lua) │ │ Abstraction │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────────────┘ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ │
│ │ │ App │ │ IPC │ │ libdatachannel │ │ │
│ │ │ Manager │ │ Router │ │ (WebRTC Bridge) │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
│ WebRTC (libdatachannel)
▼
┌─────────────────────────────────────────────────────────────────────┐
│ Real Smartphone / Other Devices │
└─────────────────────────────────────────────────────────────────────┘
```
---
## Folder Structure
```
MosisService/
├── src/main/
│ ├── kernel/ # Shared cross-platform kernel
│ │ ├── include/ # Platform abstractions
│ │ └── src/ # Shared implementation
│ ├── cpp/ # Android-specific native code
│ ├── java/ # Kotlin/Java code
│ └── assets/ # Shared UI assets
│ ├── apps/ # System apps
│ │ ├── home/
│ │ ├── dialer/
│ │ ├── messages/
│ │ ├── contacts/
│ │ ├── settings/
│ │ ├── browser/
│ │ ├── store/ # TODO
│ │ └── camera/ # TODO
│ ├── ui/ # Shared stylesheets
│ ├── scripts/ # Lua scripts
│ ├── icons/ # TGA icons
│ └── fonts/ # TTF fonts
│
├── designer/ # Desktop designer
│ ├── src/
│ │ ├── testing/ # UI inspection, capture
│ │ └── ...
│ └── build/
│
├── designer-test/ # Automated UI tests
│ ├── src/
│ └── build/
│
└── docs/ # Documentation (future)
```
---
## Build Commands
```bash
# Android
./gradlew assembleDebug
./gradlew installDebug
# Desktop Designer
cd designer
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake
cmake --build build --config Debug
./build/Debug/mosis-designer.exe ../src/main/assets/apps/home/home.rml
# Designer Tests
cd designer-test
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake
cmake --build build --config Debug
./build/Debug/designer-test.exe
```
---
## Current Sprint: Complete Partial Tasks
### Priority 1: Testing Framework Completion ✅ DONE
- [x] Action recording (capture interactions to JSON)
- [x] Action playback (replay with timing)
- [x] Screenshot diff (visual regression)
- [x] GLFW input hooks for automatic recording
### Priority 2: Remaining System Apps
- [ ] Store app (UI only - browse, install)
- [ ] Camera app (UI + shared texture from MainActivity)
- [ ] Music app (UI outline only)
### Priority 3: App Data Persistence
- [ ] JSON/SQLite storage layer
- [ ] Contact CRUD operations
- [ ] Message history
- [ ] Settings persistence
---
## Resources
### Material Design
- **Icons**: `D:\Dev\Mosis\MosisDesigner\material-design-icons`
- 20 categories, SVG/PNG/Font formats
- Browse at https://fonts.google.com/icons
- **Components**: `D:\Dev\Mosis\MosisDesigner\material-design-lite`
- Reference implementation for design patterns
### Documentation
- `CLAUDE.md` - Development guide
- `TESTING.md` - Testing framework documentation
- `ROADMAP.md` - This file
---
*Last updated: 2026-01-16*

3569
docs/SANDBOX.md Normal file
View File

File diff suppressed because it is too large Load Diff

986
docs/SANDBOX_MILESTONES.md Normal file
View File

@@ -0,0 +1,986 @@
# Sandbox Implementation Milestones
Based on SANDBOX.md design document. Implementation order optimized for dependencies.
---
## Milestone 1: Core Sandbox Foundation ✅
**Status**: Complete
**Goal**: Create isolated Lua environments with resource limits.
**Estimated Files**: 3 new files
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| LuaSandbox class | `src/main/cpp/sandbox/lua_sandbox.h` | Main sandbox container |
| Custom allocator | `src/main/cpp/sandbox/lua_sandbox.cpp` | Per-app memory tracking/limits |
| Instruction hook | `src/main/cpp/sandbox/lua_sandbox.cpp` | CPU limit enforcement |
| Globals removal | `src/main/cpp/sandbox/lua_sandbox.cpp` | Remove os, io, debug, etc. |
| Bytecode prevention | `src/main/cpp/sandbox/lua_sandbox.cpp` | Text-only loading mode |
| Metatable protection | `src/main/cpp/sandbox/lua_sandbox.cpp` | Freeze _G and string metatable |
### Implementation Tasks
1. Create `LuaSandbox` class with:
- `lua_State* m_L` - isolated state
- `SandboxLimits m_limits` - memory/CPU/stack limits
- `SandboxContext m_context` - app_id, permissions, etc.
- Custom allocator that tracks `m_memory_used`
2. Implement `RemoveDangerousGlobals()`:
- Remove: `os`, `io`, `debug`, `package`, `require`, `ffi`, `jit`
- Remove: `dofile`, `loadfile`, `load`, `loadstring`
- Remove: `rawget`, `rawset`, `rawequal`, `rawlen`
- Remove: `collectgarbage`, `newproxy`
- Remove: `string.dump`
3. Implement instruction hook:
- `lua_sethook(L, InstructionHook, LUA_MASKCOUNT, 1000)`
- Throw error when limit exceeded
4. Implement `ProtectBuiltinTables()`:
- Set `__metatable` on string metatable
- Freeze `_G` with `__newindex` error
### Dependencies
- Lua library already linked (via RmlUi)
### Test Criteria
```cpp
TEST(LuaSandbox, DangerousGlobalsRemoved);
TEST(LuaSandbox, BytecodeRejected);
TEST(LuaSandbox, MemoryLimitEnforced);
TEST(LuaSandbox, CPULimitEnforced);
TEST(LuaSandbox, MetatableProtected);
```
---
## Milestone 2: Permission System ✅
**Status**: Complete
**Goal**: Gate API access based on app permissions.
**Estimated Files**: 2 new files
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| PermissionGate | `src/main/cpp/sandbox/permission_gate.h` | Permission checking |
| User gesture tracking | `src/main/cpp/sandbox/permission_gate.cpp` | Recent tap/click detection |
| Permission categories | `src/main/cpp/sandbox/permission_gate.cpp` | Normal/Dangerous/Signature |
### Implementation Tasks
1. Define permission categories:
```cpp
enum class PermissionCategory { Normal, Dangerous, Signature };
```
2. Create `PermissionGate` class:
- `bool Check(lua_State* L, const std::string& permission)`
- `int RequirePermission(lua_State* L, const char* perm)` - throws Lua error
3. Implement user gesture tracking:
- `void RecordUserGesture()` - called on touch down
- `bool HasRecentUserGesture(int ms)` - check within time window
4. Define permission manifest parsing:
- Read `manifest.json` from app package
- Extract `permissions` array
### Dependencies
- Milestone 1 (LuaSandbox)
### Test Criteria
```cpp
TEST(PermissionGate, NormalPermissionAutoGranted);
TEST(PermissionGate, DangerousPermissionRequiresGrant);
TEST(PermissionGate, SignaturePermissionSystemOnly);
TEST(PermissionGate, UserGestureRequired);
```
---
## Milestone 3: Audit Logging & Rate Limiting ✅
**Status**: Complete
**Goal**: Track security events and prevent API abuse.
**Estimated Files**: 2 new files
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| AuditLog | `src/main/cpp/sandbox/audit_log.h/cpp` | Security event logging |
| RateLimiter | `src/main/cpp/sandbox/rate_limiter.h/cpp` | Token bucket rate limiting |
### Implementation Tasks
1. Create `AuditLog` class:
```cpp
enum class AuditEvent {
AppStart, AppStop, PermissionCheck, PermissionDenied,
NetworkRequest, FileAccess, SandboxViolation, ResourceLimitHit
};
```
2. Implement thread-safe logging:
- Ring buffer with max entries
- Log critical events to Android logcat
3. Create `RateLimiter` class:
- Token bucket algorithm
- Per-operation limits: `network.request`, `storage.write`, etc.
- `bool Check(const std::string& app_id, const std::string& operation)`
### Dependencies
- Milestone 1 (LuaSandbox)
### Test Criteria
```cpp
TEST(AuditLog, LogsSecurityEvents);
TEST(AuditLog, ThreadSafe);
TEST(RateLimiter, EnforcesLimits);
TEST(RateLimiter, RefillsOverTime);
```
---
## Milestone 4: Safe Path & Require ✅
**Status**: Complete
**Goal**: Secure file access within app sandbox.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| Path validation | `src/main/cpp/sandbox/path_sandbox.h/cpp` | Canonical path checking |
| Safe require | `src/main/cpp/sandbox/path_sandbox.cpp` | Module loader from app/scripts/ |
### Implementation Tasks
1. Implement `ValidatePath()`:
- Reject `..` traversal
- Reject absolute paths
- Canonicalize and verify prefix match
2. Implement `SafeRequire()`:
- Validate module name (alphanumeric + underscore)
- Load from `app_path/scripts/name.lua`
- Cache in registry `__loaded`
- Text-only mode (`"t"`)
3. Register as global `require`:
```cpp
lua_pushcfunction(L, SafeRequire);
lua_setglobal(L, "require");
```
### Dependencies
- Milestone 1 (LuaSandbox)
### Test Criteria
```cpp
TEST(PathSandbox, RejectsTraversal);
TEST(PathSandbox, RejectsAbsolutePaths);
TEST(SafeRequire, LoadsFromScriptsDir);
TEST(SafeRequire, CachesModules);
```
---
## Milestone 5: Timer & Callback System ✅
**Status**: Complete
**Goal**: Safe timer APIs managed by kernel.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| TimerManager | `src/main/cpp/sandbox/timer_manager.h/cpp` | setTimeout/setInterval |
### Implementation Tasks
1. Create `TimerManager` class:
- Priority queue of pending timers
- Per-app timer limits (max 100)
- Minimum 10ms granularity
2. Implement Lua APIs:
```lua
setTimeout(callback, ms) -> id
clearTimeout(id)
setInterval(callback, ms) -> id
clearInterval(id)
```
3. Integrate with kernel main loop:
- Check and fire timers each frame
- Reset instruction count before callback
4. Implement cleanup:
- `ClearAppTimers(app_id)` on app stop
### Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 3 (RateLimiter)
### Test Criteria
```cpp
TEST(TimerManager, FiresTimeout);
TEST(TimerManager, FiresInterval);
TEST(TimerManager, LimitsPerApp);
TEST(TimerManager, CleansUpOnStop);
```
---
## Milestone 6: JSON & Crypto APIs ✅
**Status**: Complete
**Goal**: Safe data parsing and cryptographic primitives.
**Estimated Files**: 2 new files
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| JSON API | `src/main/cpp/sandbox/json_api.cpp` | Safe encode/decode |
| Crypto API | `src/main/cpp/sandbox/crypto_api.cpp` | Hash, random, HMAC |
### Implementation Tasks
1. Implement `json.decode()`:
- Depth limit (32)
- String length limit (1 MB)
- Array/object size limits
2. Implement `json.encode()`:
- Cycle detection
- Output size limit
3. Implement crypto APIs:
- `crypto.randomBytes(n)` - CSPRNG
- `crypto.hash("sha256", data)`
- `crypto.hmac("sha256", key, data)`
- Remove `math.randomseed`, replace `math.random` with per-app RNG
### Dependencies
- Milestone 1 (LuaSandbox)
- nlohmann-json (already in vcpkg)
### Test Criteria
```cpp
TEST(JsonApi, DecodesValid);
TEST(JsonApi, RejectsDeepNesting);
TEST(JsonApi, RejectsTooLarge);
TEST(CryptoApi, RandomBytesSecure);
TEST(CryptoApi, HashCorrect);
```
---
## Milestone 7: Virtual Filesystem ✅
**Status**: Complete
**Goal**: Per-app isolated storage with quotas.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| VirtualFS | `src/main/cpp/sandbox/virtual_fs.h/cpp` | App storage API |
### Implementation Tasks
1. Define virtual path mapping:
```
/data/ → /data/data/com.omixlab.mosis/apps/<app_id>/data/
/cache/ → /data/data/com.omixlab.mosis/apps/<app_id>/cache/
/temp/ → (session-only, cleared on stop)
/shared/ → requires storage.shared permission
```
2. Implement Lua file API:
```lua
fs.read(path) -> string
fs.write(path, data) -> bool
fs.append(path, data) -> bool
fs.delete(path) -> bool
fs.exists(path) -> bool
fs.list(dir) -> table
fs.mkdir(path) -> bool
fs.stat(path) -> {size, modified, isDir}
```
3. Enforce limits:
- Per-app quota (50 MB default)
- Max file size (10 MB)
- Max path depth (10)
4. Cleanup `/temp/` on app stop.
### Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate)
- Milestone 4 (Path validation)
### Test Criteria
```cpp
TEST(VirtualFS, ReadsWritesInAppDir);
TEST(VirtualFS, BlocksTraversal);
TEST(VirtualFS, EnforcesQuota);
TEST(VirtualFS, CleansUpTemp);
```
---
## Milestone 8: SQLite Database ✅
**Status**: Complete
**Goal**: Per-app SQLite with injection prevention.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| DatabaseManager | `src/main/cpp/sandbox/database_manager.h/cpp` | SQLite sandbox |
### Implementation Tasks
1. Create per-app database:
- Location: `/data/data/com.omixlab.mosis/apps/<app_id>/db/app.db`
- Max database size: 50 MB
2. Implement SQLite authorizer:
```cpp
sqlite3_set_authorizer(db, Authorizer, sandbox);
// Block: ATTACH, DETACH, dangerous PRAGMAs, load_extension
```
3. Implement Lua API:
```lua
local db = database.open("mydata")
db:execute("CREATE TABLE IF NOT EXISTS items (id INTEGER PRIMARY KEY, name TEXT)")
db:execute("INSERT INTO items (name) VALUES (?)", {"Item 1"})
local rows = db:query("SELECT * FROM items WHERE id > ?", {0})
db:close()
```
4. Enforce prepared statements only (no raw SQL interpolation).
### Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate)
- SQLite3 (add to vcpkg)
### Test Criteria
```cpp
TEST(Database, CreatesTables);
TEST(Database, PreparedStatements);
TEST(Database, BlocksAttach);
TEST(Database, BlocksDangerousPragma);
```
---
## Milestone 9: Network - HTTP ✅
**Status**: Complete
**Goal**: Secure HTTP requests with domain filtering.
**Estimated Files**: 2 new files
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| HttpRequestValidator | `src/main/cpp/sandbox/http_validator.h/cpp` | URL/domain validation |
| NetworkManager | `src/main/cpp/sandbox/network_manager.h/cpp` | HTTP client wrapper |
### Implementation Tasks
1. Implement URL parsing and validation:
- Require HTTPS
- Block private IPs, localhost, metadata IPs
- Domain whitelist from manifest
2. Implement Lua HTTP API:
```lua
local response = network.request({
url = "https://api.example.com/data",
method = "POST",
headers = {...},
body = "...",
timeout = 30000
})
```
3. Integrate with Android HttpURLConnection or libcurl.
4. Enforce limits:
- Request body: 10 MB
- Response body: 50 MB
- Timeout: 60s
- Concurrent requests: 6
### Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate)
- Milestone 3 (RateLimiter, AuditLog)
### Test Criteria
```cpp
TEST(HttpValidator, BlocksPrivateIP);
TEST(HttpValidator, BlocksLocalhost);
TEST(HttpValidator, RequiresHttps);
TEST(HttpValidator, EnforcesDomainWhitelist);
TEST(NetworkManager, MakesRequest);
```
---
## Milestone 10: Network - WebSocket ✅
**Status**: Complete
**Goal**: Secure WebSocket connections.
**Estimated Files**: 1 new file (extends NetworkManager)
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| WebSocketManager | `src/main/cpp/sandbox/websocket_manager.h/cpp` | WS connections |
### Implementation Tasks
1. Implement connection management:
- Max 5 connections per app
- Idle timeout: 5 minutes
- Message size limit: 1 MB
2. Implement Lua WebSocket API:
```lua
local ws = network.websocket("wss://api.example.com/ws")
ws:on("open", function() ... end)
ws:on("message", function(data) ... end)
ws:on("close", function(code, reason) ... end)
ws:send(data)
ws:close()
```
3. Use same URL validation as HTTP.
4. Cleanup connections on app stop.
### Dependencies
- Milestone 9 (HttpRequestValidator)
### Test Criteria
```cpp
TEST(WebSocketManager, Connects);
TEST(WebSocketManager, LimitsPerApp);
TEST(WebSocketManager, CleansUpOnStop);
```
---
## Milestone 11: Virtual Hardware - Camera ✅
**Status**: Complete
**Goal**: Camera access with mandatory indicators.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| CameraInterface | `src/main/cpp/sandbox/camera_interface.h/cpp` | Camera API |
### Implementation Tasks
1. Implement session management:
- `startSession(options)` - requires permission + user gesture
- `capturePhoto()` - returns frame data
- `stopSession()`
2. Mandatory recording indicator:
- Show in system UI (cannot be hidden by app)
3. Implement Lua API:
```lua
local session = camera.start({facing = "back", resolution = "720p"})
session:on("frame", function(data) ... end)
local photo = session:capture()
session:stop()
```
4. Rate limit: 1 session at a time, max 30 fps.
### Dependencies
- Milestone 2 (PermissionGate + user gesture)
- Milestone 3 (AuditLog)
- Game engine camera bridge (future)
### Test Criteria
```cpp
TEST(CameraInterface, RequiresPermission);
TEST(CameraInterface, RequiresUserGesture);
TEST(CameraInterface, ShowsIndicator);
TEST(CameraInterface, StopsOnAppStop);
```
---
## Milestone 12: Virtual Hardware - Microphone ✅
**Status**: Complete
**Goal**: Audio recording with mandatory indicators.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| MicrophoneInterface | `src/main/cpp/sandbox/microphone_interface.h/cpp` | Mic API |
### Implementation Tasks
1. Implement recording session:
- `startRecording(options)` - requires permission + user gesture
- `stopRecording()` - returns audio data
2. Mandatory recording indicator (same as camera).
3. Implement Lua API:
```lua
local recording = microphone.start({sampleRate = 44100, channels = 1})
recording:on("data", function(samples) ... end)
local audio = recording:stop()
```
4. Rate limit: 1 recording at a time.
### Dependencies
- Milestone 2 (PermissionGate + user gesture)
- Milestone 3 (AuditLog)
### Test Criteria
```cpp
TEST(MicrophoneInterface, RequiresPermission);
TEST(MicrophoneInterface, RequiresUserGesture);
TEST(MicrophoneInterface, ShowsIndicator);
```
---
## Milestone 13: Virtual Hardware - Audio Output ✅
**Status**: Complete
**Goal**: Safe audio playback.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| AudioOutputInterface | `src/main/cpp/sandbox/audio_output.h/cpp` | Speaker API |
### Implementation Tasks
1. Implement playback:
- `play(audioData, options)`
- `stop()`
- `setVolume(0.0-1.0)`
2. System volume limit (app cannot exceed system volume).
3. Implement Lua API:
```lua
local player = audio.play(soundData, {loop = false, volume = 0.8})
player:on("ended", function() ... end)
player:stop()
```
4. Concurrent sound limit: 10 per app.
### Dependencies
- Milestone 1 (LuaSandbox)
### Test Criteria
```cpp
TEST(AudioOutput, PlaysSound);
TEST(AudioOutput, RespectsVolumeLimit);
TEST(AudioOutput, LimitsConcurrent);
```
---
## Milestone 14: Virtual Hardware - Location ✅
**Status**: Complete
**Goal**: Location with privacy controls.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| LocationInterface | `src/main/cpp/sandbox/location_interface.h/cpp` | GPS API |
### Implementation Tasks
1. Two permission levels:
- `location.coarse` - city-level (1km accuracy)
- `location.fine` - precise GPS
2. Implement Lua API:
```lua
location.getCurrentPosition(function(pos)
print(pos.latitude, pos.longitude, pos.accuracy)
end)
local watch = location.watchPosition(function(pos) ... end, {
enableHighAccuracy = true,
timeout = 30000
})
watch:stop()
```
3. Rate limit: 1 request per second.
4. Reduce precision for coarse permission.
### Dependencies
- Milestone 2 (PermissionGate)
- Milestone 3 (RateLimiter)
### Test Criteria
```cpp
TEST(LocationInterface, RequiresPermission);
TEST(LocationInterface, CoarseReducesPrecision);
TEST(LocationInterface, RateLimits);
```
---
## Milestone 15: Virtual Hardware - Sensors ✅
**Status**: Complete
**Goal**: Motion sensors with fingerprinting prevention.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| SensorInterface | `src/main/cpp/sandbox/sensor_interface.h/cpp` | Accelerometer, gyro, etc. |
### Implementation Tasks
1. Supported sensors:
- Accelerometer (requires `sensors.motion`)
- Gyroscope (requires `sensors.motion`)
- Magnetometer (requires `sensors.motion`)
- Proximity (auto-granted)
- Ambient light (auto-granted)
2. Implement Lua API:
```lua
local sub = sensors.subscribe("accelerometer", function(data)
print(data.x, data.y, data.z, data.timestamp)
end, {frequency = 60})
sub:stop()
```
3. Reduce precision to prevent fingerprinting:
- Round values to 2 decimal places
- Limit update frequency to 60 Hz
### Dependencies
- Milestone 2 (PermissionGate)
### Test Criteria
```cpp
TEST(SensorInterface, RequiresPermission);
TEST(SensorInterface, ReducesPrecision);
TEST(SensorInterface, LimitsFrequency);
```
---
## Milestone 16: Virtual Hardware - Bluetooth ✅
**Status**: Complete
**Goal**: Bluetooth discovery and pairing.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| BluetoothInterface | `src/main/cpp/sandbox/bluetooth_interface.h/cpp` | BT API |
### Implementation Tasks
1. Discovery with user consent:
- `startDiscovery()` requires user confirmation
- Return device name/address only (no full UUID list)
2. Implement Lua API:
```lua
bluetooth.startDiscovery(function(devices)
for _, device in ipairs(devices) do
print(device.name, device.address)
end
end, {timeout = 30})
bluetooth.connect(address, function(connection)
connection:send(data)
end)
```
3. Limit: 5 connections per app.
### Dependencies
- Milestone 2 (PermissionGate + user gesture)
### Test Criteria
```cpp
TEST(BluetoothInterface, RequiresPermission);
TEST(BluetoothInterface, RequiresUserConsent);
```
---
## Milestone 17: Virtual Hardware - Contacts ✅
**Status**: Complete
**Goal**: Contact access with granular permissions.
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| ContactsInterface | `src/main/cpp/sandbox/contacts_interface.h/cpp` | Contacts API |
### Implementation Tasks
1. Separate read/write permissions:
- `contacts.read` - query contacts
- `contacts.write` - add/modify contacts
2. Implement Lua API:
```lua
local contacts = contacts.query({search = "John"})
for _, c in ipairs(contacts) do
print(c.name, c.phone, c.email)
end
contacts.add({name = "New Contact", phone = "555-1234"})
```
3. Limit fields returned (no raw account data).
### Dependencies
- Milestone 2 (PermissionGate)
### Test Criteria
```cpp
TEST(ContactsInterface, RequiresReadPermission);
TEST(ContactsInterface, RequiresWritePermission);
```
---
## ✅ Milestone 18: Inter-App Communication
**Goal**: Kernel-mediated message passing.
**Status**: Complete
**Estimated Files**: 1 new file
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| MessageBus | `src/main/cpp/sandbox/message_bus.h/cpp` | App-to-app messaging |
### Implementation Tasks
1. Implement intent system:
- Manifest declares received intents
- Sender must have required permissions
2. Implement Lua API:
```lua
-- Sending
intents.send({
action = "share",
type = "text/plain",
data = "Hello world"
})
-- Receiving (in manifest + handler)
intents.on("share", function(intent)
print(intent.from, intent.data)
end)
```
3. Validate sender/receiver at runtime.
### Dependencies
- Milestone 2 (PermissionGate)
- Milestone 3 (AuditLog)
### Test Criteria
```cpp
TEST(MessageBus, SendsToRegisteredReceiver);
TEST(MessageBus, BlocksUnregisteredAction);
```
---
## ✅ Milestone 19: Security Testing Suite
**Goal**: Comprehensive test coverage.
**Status**: Complete
**Estimated Files**: 3 new files
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| Unit tests | `tests/test_sandbox_security.cpp` | Core sandbox tests |
| Integration tests | `tests/test_sandbox_integration.cpp` | Full system tests |
| Fuzzer | `tests/fuzz_sandbox.cpp` | Random input testing |
### Implementation Tasks
1. Write unit tests for all milestones (see individual test criteria).
2. Write integration tests:
- Full app lifecycle
- Permission flow
- Resource cleanup
3. Implement fuzzer:
- Generate random Lua code
- Verify no crashes
- Verify sandbox integrity after each run
4. Security audit checklist (from SANDBOX.md).
### Dependencies
- All previous milestones
---
## Milestone 20: Kernel Integration ✅
**Status**: Complete
**Goal**: Multi-app sandbox orchestrator for kernel integration.
### Deliverables
| Component | File | Description |
|-----------|------|-------------|
| LuaSandboxManager | `src/main/cpp/sandbox/sandbox_manager.h` | Multi-app orchestrator |
| Implementation | `src/main/cpp/sandbox/sandbox_manager.cpp` | App lifecycle management |
| AppSandbox struct | `src/main/cpp/sandbox/sandbox_manager.h` | Per-app component container |
### Implementation Tasks
1. ✅ Create `LuaSandboxManager` class:
- Multi-app management with Start/Stop lifecycle
- Shared components (AuditLog, RateLimiter, MessageBus, TimerManager)
- Thread-safe app map access
2. ✅ Create `AppSandbox` struct:
- Per-app isolated Lua state and permissions
- Per-app VirtualFS, DatabaseManager, NetworkManager
- Per-app hardware interfaces (camera, mic, audio, location, sensors, bluetooth, contacts)
3. ✅ Wire up resource cleanup on app stop:
- Clear timers, close websockets, shutdown hardware
- Clean temp files, close databases, unregister from message bus
### Dependencies
- Milestones 1-19
---
## Summary
| Phase | Milestones | Description |
|-------|------------|-------------|
| **Foundation** | 1-4 | Core sandbox, permissions, logging, paths |
| **Core APIs** | 5-8 | Timers, JSON, crypto, filesystem, database |
| **Network** | 9-10 | HTTP and WebSocket |
| **Hardware** | 11-17 | Camera, mic, audio, location, sensors, BT, contacts |
| **System** | 18-20 | IPC, testing, integration |
### Recommended Order
1. **Milestone 1** - Must be first (foundation)
2. **Milestone 2** - Permissions (needed by everything)
3. **Milestone 3** - Audit/rate limiting (needed by APIs)
4. **Milestone 4** - Path security (needed by fs/require)
5. **Milestone 5-6** - Timers, JSON (high value)
6. **Milestone 7-8** - Storage, database (app data)
7. **Milestone 9-10** - Network (app connectivity)
8. **Milestones 11-17** - Hardware (as needed)
9. **Milestone 18** - IPC (multi-app)
10. **Milestone 19-20** - Testing & integration
### Quick Start
Begin with **Milestone 1** to establish the core sandbox. This is the foundation everything else builds on.

595
docs/SANDBOX_MILESTONE_1.md Normal file
View File

@@ -0,0 +1,595 @@
# Milestone 1: Core Sandbox Foundation
**Status**: ✅ Complete
**Goal**: Create isolated Lua environments with resource limits and dangerous global removal.
---
## Overview
This milestone establishes the foundational security layer for the Mosis app sandbox. Every subsequent milestone builds on this foundation.
### Key Deliverables
1. **LuaSandbox class** - Per-app isolated Lua state
2. **Custom allocator** - Memory tracking and limits
3. **Instruction hook** - CPU limit enforcement
4. **Globals removal** - Remove dangerous functions
5. **Bytecode prevention** - Text-only loading
6. **Metatable protection** - Freeze globals
---
## File Structure
```
src/main/cpp/sandbox/
├── lua_sandbox.h # Main header
├── lua_sandbox.cpp # Implementation
└── sandbox_tests.h # Test declarations (for test runner)
sandbox-test/
├── CMakeLists.txt # Test build config
├── README.md # Test documentation
├── src/
│ ├── main.cpp # Test runner entry
│ ├── test_harness.h # Test framework
│ └── test_harness.cpp # Test framework impl
└── scripts/
├── test_globals_removed.lua
├── test_bytecode_rejected.lua
├── test_memory_limit.lua
├── test_cpu_limit.lua
├── test_metatable_protected.lua
├── test_safe_operations.lua
└── test_string_dump_removed.lua
```
---
## Implementation Details
### 1. SandboxLimits Structure
```cpp
// lua_sandbox.h
struct SandboxLimits {
size_t memory_bytes = 16 * 1024 * 1024; // 16 MB default
size_t max_string_size = 1 * 1024 * 1024; // 1 MB max string
size_t max_table_entries = 100000; // Prevent hash DoS
uint64_t instructions_per_call = 1000000; // ~10ms execution
int stack_depth = 200; // Recursion limit
};
```
### 2. SandboxContext Structure
```cpp
// lua_sandbox.h
struct SandboxContext {
std::string app_id;
std::string app_path;
std::vector<std::string> permissions;
bool is_system_app = false;
};
```
### 3. LuaSandbox Class
```cpp
// lua_sandbox.h
class LuaSandbox {
public:
explicit LuaSandbox(const SandboxContext& context,
const SandboxLimits& limits = {});
~LuaSandbox();
// Non-copyable
LuaSandbox(const LuaSandbox&) = delete;
LuaSandbox& operator=(const LuaSandbox&) = delete;
// Load and execute Lua code
bool LoadString(const std::string& code, const std::string& chunk_name = "chunk");
bool LoadFile(const std::string& path);
// State access
lua_State* GetState() const { return m_L; }
const std::string& GetLastError() const { return m_last_error; }
// Resource usage
size_t GetMemoryUsed() const { return m_memory_used; }
uint64_t GetInstructionsUsed() const { return m_instructions_used; }
// Context access
const SandboxContext& GetContext() const { return m_context; }
const std::string& app_id() const { return m_context.app_id; }
// Reset instruction counter (call before each event handler)
void ResetInstructionCount();
private:
void SetupSandbox();
void RemoveDangerousGlobals();
void ProtectBuiltinTables();
void SetupInstructionHook();
static void* SandboxAlloc(void* ud, void* ptr, size_t osize, size_t nsize);
static void InstructionHook(lua_State* L, lua_Debug* ar);
lua_State* m_L = nullptr;
SandboxContext m_context;
SandboxLimits m_limits;
size_t m_memory_used = 0;
uint64_t m_instructions_used = 0;
std::string m_last_error;
};
```
### 4. Custom Memory Allocator
```cpp
// lua_sandbox.cpp
void* LuaSandbox::SandboxAlloc(void* ud, void* ptr, size_t osize, size_t nsize) {
auto* sandbox = static_cast<LuaSandbox*>(ud);
// Calculate new usage
size_t new_usage = sandbox->m_memory_used - osize + nsize;
// Check limit (only when allocating, not freeing)
if (nsize > 0 && new_usage > sandbox->m_limits.memory_bytes) {
return nullptr; // Allocation fails, triggers Lua memory error
}
// Update tracking
sandbox->m_memory_used = new_usage;
// Free
if (nsize == 0) {
free(ptr);
return nullptr;
}
// Alloc or realloc
return realloc(ptr, nsize);
}
```
### 5. Instruction Hook
```cpp
// lua_sandbox.cpp
void LuaSandbox::InstructionHook(lua_State* L, lua_Debug* ar) {
// Get sandbox from registry
lua_getfield(L, LUA_REGISTRYINDEX, "__sandbox");
auto* sandbox = static_cast<LuaSandbox*>(lua_touserdata(L, -1));
lua_pop(L, 1);
if (!sandbox) return;
// Increment by hook interval (1000 instructions)
sandbox->m_instructions_used += 1000;
// Check limit
if (sandbox->m_instructions_used > sandbox->m_limits.instructions_per_call) {
luaL_error(L, "instruction limit exceeded (%llu instructions)",
sandbox->m_instructions_used);
}
}
void LuaSandbox::SetupInstructionHook() {
lua_sethook(m_L, InstructionHook, LUA_MASKCOUNT, 1000);
}
```
### 6. Dangerous Globals Removal
```cpp
// lua_sandbox.cpp
void LuaSandbox::RemoveDangerousGlobals() {
const char* dangerous_globals[] = {
// Code execution
"dofile", "loadfile", "load", "loadstring",
// Raw access (bypass metatables)
"rawget", "rawset", "rawequal", "rawlen",
// Metatable manipulation (removed for protection)
"getmetatable", "setmetatable",
// GC manipulation
"collectgarbage",
// Dangerous libraries
"os", "io", "debug", "package",
// LuaJIT/FFI (if present)
"ffi", "jit", "newproxy",
nullptr
};
for (const char** p = dangerous_globals; *p; ++p) {
lua_pushnil(m_L);
lua_setglobal(m_L, *p);
}
// Remove require (we'll add safe version later)
lua_pushnil(m_L);
lua_setglobal(m_L, "require");
// Remove string.dump (creates bytecode)
lua_getglobal(m_L, "string");
if (lua_istable(m_L, -1)) {
lua_pushnil(m_L);
lua_setfield(m_L, -2, "dump");
}
lua_pop(m_L, 1);
}
```
### 7. Metatable Protection
```cpp
// lua_sandbox.cpp
void LuaSandbox::ProtectBuiltinTables() {
// Protect string metatable
lua_pushstring(m_L, "");
if (lua_getmetatable(m_L, -1)) {
lua_pushstring(m_L, "string");
lua_setfield(m_L, -2, "__metatable");
lua_pop(m_L, 1); // pop metatable
}
lua_pop(m_L, 1); // pop string
// Freeze _G with protective metatable
lua_pushglobaltable(m_L);
lua_newtable(m_L); // metatable for _G
// __metatable prevents access via getmetatable
lua_pushstring(m_L, "globals");
lua_setfield(m_L, -2, "__metatable");
// __newindex prevents modification
lua_pushcfunction(m_L, [](lua_State* L) -> int {
const char* key = lua_tostring(L, 2);
return luaL_error(L, "cannot modify global '%s'", key ? key : "(unknown)");
});
lua_setfield(m_L, -2, "__newindex");
lua_setmetatable(m_L, -2);
lua_pop(m_L, 1); // pop _G
}
```
### 8. Load with Bytecode Prevention
```cpp
// lua_sandbox.cpp
bool LuaSandbox::LoadString(const std::string& code, const std::string& chunk_name) {
// Reset instruction count for this execution
ResetInstructionCount();
// Load as TEXT ONLY - "t" mode rejects bytecode
int result = luaL_loadbufferx(m_L, code.c_str(), code.size(),
chunk_name.c_str(), "t");
if (result != LUA_OK) {
m_last_error = lua_tostring(m_L, -1);
lua_pop(m_L, 1);
return false;
}
// Execute
result = lua_pcall(m_L, 0, 0, 0);
if (result != LUA_OK) {
m_last_error = lua_tostring(m_L, -1);
lua_pop(m_L, 1);
return false;
}
return true;
}
```
---
## Test Cases
Each test has a corresponding Lua script and C++ test function.
### Test 1: Dangerous Globals Removed
**Script**: `scripts/test_globals_removed.lua`
```lua
-- Test that dangerous globals are nil
local dangerous = {
"os", "io", "debug", "package", "ffi", "jit",
"dofile", "loadfile", "load", "loadstring",
"rawget", "rawset", "rawequal", "rawlen",
"collectgarbage", "newproxy", "require"
}
for _, name in ipairs(dangerous) do
local value = _G[name]
if value ~= nil then
error("FAIL: " .. name .. " should be nil but is " .. type(value))
end
end
print("PASS: All dangerous globals removed")
```
**C++ Test**:
```cpp
TEST(LuaSandbox, DangerousGlobalsRemoved) {
LuaSandbox sandbox(TestContext());
EXPECT_TRUE(sandbox.LoadFile("scripts/test_globals_removed.lua"));
}
```
### Test 2: Bytecode Rejected
**Script**: `scripts/test_bytecode_rejected.lua`
```lua
-- This script tests that bytecode loading is blocked
-- The actual bytecode test is done from C++ side
print("PASS: Text loading works")
```
**C++ Test**:
```cpp
TEST(LuaSandbox, BytecodeRejected) {
LuaSandbox sandbox(TestContext());
// Lua bytecode starts with signature: \x1bLua
std::string bytecode = "\x1bLua\x54\x00\x19\x93"; // Fake bytecode header
EXPECT_FALSE(sandbox.LoadString(bytecode, "bytecode_test"));
EXPECT_TRUE(sandbox.GetLastError().find("binary") != std::string::npos ||
sandbox.GetLastError().find("attempt to load") != std::string::npos);
}
```
### Test 3: Memory Limit Enforced
**Script**: `scripts/test_memory_limit.lua`
```lua
-- This script intentionally tries to exhaust memory
-- Should fail with memory error
local t = {}
local i = 0
while true do
i = i + 1
t[i] = string.rep("x", 100000) -- 100KB strings
if i > 1000 then
error("FAIL: Should have hit memory limit by now")
end
end
```
**C++ Test**:
```cpp
TEST(LuaSandbox, MemoryLimitEnforced) {
SandboxLimits limits;
limits.memory_bytes = 1024 * 1024; // 1 MB limit
LuaSandbox sandbox(TestContext(), limits);
EXPECT_FALSE(sandbox.LoadFile("scripts/test_memory_limit.lua"));
// Should fail due to memory allocation failure
}
```
### Test 4: CPU Limit Enforced
**Script**: `scripts/test_cpu_limit.lua`
```lua
-- Infinite loop - should be stopped by instruction limit
while true do
-- busy loop
end
error("FAIL: Should never reach here")
```
**C++ Test**:
```cpp
TEST(LuaSandbox, CPULimitEnforced) {
SandboxLimits limits;
limits.instructions_per_call = 10000; // Very low limit
LuaSandbox sandbox(TestContext(), limits);
EXPECT_FALSE(sandbox.LoadFile("scripts/test_cpu_limit.lua"));
EXPECT_TRUE(sandbox.GetLastError().find("instruction") != std::string::npos);
}
```
### Test 5: Metatable Protected
**Script**: `scripts/test_metatable_protected.lua`
```lua
-- Test 1: String metatable is protected
local mt = getmetatable("")
if mt ~= "string" then
error("FAIL: string metatable should return 'string', got " .. tostring(mt))
end
-- Test 2: Cannot modify global environment
local ok, err = pcall(function()
_G.my_new_global = "test"
end)
if ok then
error("FAIL: Should not be able to add globals")
end
-- Test 3: Cannot modify existing globals
local ok2, err2 = pcall(function()
_G.print = nil
end)
if ok2 then
error("FAIL: Should not be able to modify print")
end
print("PASS: Metatables protected")
```
**C++ Test**:
```cpp
TEST(LuaSandbox, MetatableProtected) {
LuaSandbox sandbox(TestContext());
EXPECT_TRUE(sandbox.LoadFile("scripts/test_metatable_protected.lua"));
}
```
### Test 6: Safe Operations Work
**Script**: `scripts/test_safe_operations.lua`
```lua
-- Test that safe operations still work
-- Math operations
local x = math.sin(1.5) + math.floor(3.7)
assert(type(x) == "number", "Math failed")
-- String operations
local s = string.format("hello %d", 42)
assert(s == "hello 42", "String format failed")
local upper = string.upper("test")
assert(upper == "TEST", "String upper failed")
-- Table operations
local t = {1, 2, 3}
table.insert(t, 4)
assert(#t == 4, "Table insert failed")
table.sort(t)
assert(t[1] == 1 and t[4] == 4, "Table sort failed")
-- Iteration
local count = 0
for i, v in ipairs(t) do count = count + 1 end
assert(count == 4, "ipairs failed")
count = 0
for k, v in pairs({a=1, b=2, c=3}) do count = count + 1 end
assert(count == 3, "pairs failed")
-- Error handling
local ok, err = pcall(function() error("test error") end)
assert(not ok, "pcall should catch error")
assert(err:find("test error"), "Error message wrong")
-- Type checks
assert(type({}) == "table", "type table failed")
assert(type("") == "string", "type string failed")
assert(type(123) == "number", "type number failed")
assert(type(function() end) == "function", "type function failed")
-- tonumber/tostring
assert(tonumber("42") == 42, "tonumber failed")
assert(tostring(42) == "42", "tostring failed")
-- select
local a, b = select(2, 1, 2, 3)
assert(a == 2 and b == 3, "select failed")
-- assert works
assert(true, "assert failed")
print("PASS: All safe operations work")
```
### Test 7: string.dump Removed
**Script**: `scripts/test_string_dump_removed.lua`
```lua
-- string.dump can create bytecode, should be removed
if string.dump ~= nil then
error("FAIL: string.dump should be nil")
end
print("PASS: string.dump removed")
```
---
## Build & Test Commands
### Build
```bash
cd sandbox-test
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake
cmake --build build --config Debug
```
### Run All Tests
```bash
./build/Debug/sandbox-test.exe
```
### Run Single Test
```bash
./build/Debug/sandbox-test.exe --test DangerousGlobalsRemoved
```
### Test Output Format
```json
{
"name": "Lua Sandbox Security Tests",
"timestamp": "2024-01-15T10:30:00Z",
"summary": {
"passed": 7,
"failed": 0,
"total": 7
},
"tests": [
{
"name": "DangerousGlobalsRemoved",
"status": "passed",
"duration_ms": 5
},
...
]
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `TEST(LuaSandbox, DangerousGlobalsRemoved)` - os/io/debug/etc are nil
- [x] `TEST(LuaSandbox, BytecodeRejected)` - Binary chunks rejected
- [x] `TEST(LuaSandbox, MemoryLimitEnforced)` - OOM before system impact
- [x] `TEST(LuaSandbox, CPULimitEnforced)` - Infinite loops stopped
- [x] `TEST(LuaSandbox, MetatableProtected)` - _G frozen, string mt protected
- [x] `TEST(LuaSandbox, SafeOperationsWork)` - Normal Lua code works
- [x] `TEST(LuaSandbox, StringDumpRemoved)` - string.dump is nil
---
## Security Audit Checklist
After implementation, verify:
- [x] No way to access `os.execute` or equivalent
- [x] No way to load bytecode
- [x] No way to read arbitrary files
- [x] Memory exhaustion causes graceful failure
- [x] CPU exhaustion causes graceful failure
- [x] Cannot escape sandbox via metatable tricks
- [x] Cannot modify protected globals
---
## Next Steps
After Milestone 1 passes:
1. Integrate with kernel (replace global lua_State)
2. Add permission system (Milestone 2)
3. Add audit logging (Milestone 3)

484
docs/SANDBOX_MILESTONE_10.md Normal file
View File

@@ -0,0 +1,484 @@
# Milestone 10: Network - WebSocket
**Status**: Complete
**Goal**: Secure WebSocket connections with same validation as HTTP.
---
## Overview
This milestone extends the network system with WebSocket support:
- Reuse HttpValidator for URL/domain validation (wss:// only)
- Connection limits per app
- Message size limits
- Idle timeout handling
- Clean disconnection on app stop
### Key Deliverables
1. **WebSocketManager class** - Connection pool management
2. **WebSocket class** - Individual connection wrapper
3. **Lua websocket API** - `network.websocket()` function
4. **Mock implementation** - For desktop testing
---
## File Structure
```
src/main/cpp/sandbox/
├── http_validator.h # Existing - add WSS support
├── http_validator.cpp # Existing - add WSS support
├── websocket_manager.h # NEW - WebSocket pool
└── websocket_manager.cpp # NEW - Connection management
```
---
## Implementation Details
### 1. WebSocketManager Class
```cpp
// websocket_manager.h
#pragma once
#include <string>
#include <map>
#include <memory>
#include <mutex>
#include <functional>
#include <vector>
#include "http_validator.h"
struct lua_State;
namespace mosis {
struct WebSocketLimits {
int max_connections_per_app = 5;
size_t max_message_size = 1 * 1024 * 1024; // 1 MB
int idle_timeout_ms = 5 * 60 * 1000; // 5 minutes
int connect_timeout_ms = 30000; // 30 seconds
};
enum class WebSocketState {
Connecting,
Open,
Closing,
Closed
};
class WebSocket {
public:
using MessageCallback = std::function<void(const std::string& data, bool binary)>;
using CloseCallback = std::function<void(int code, const std::string& reason)>;
using ErrorCallback = std::function<void(const std::string& error)>;
using OpenCallback = std::function<void()>;
WebSocket(int id, const std::string& url);
~WebSocket();
int GetId() const { return m_id; }
const std::string& GetUrl() const { return m_url; }
WebSocketState GetState() const { return m_state; }
// Send message (returns false if not connected or message too large)
bool Send(const std::string& data, bool binary = false);
// Close connection
void Close(int code = 1000, const std::string& reason = "");
// Event callbacks
void SetOnOpen(OpenCallback cb) { m_on_open = std::move(cb); }
void SetOnMessage(MessageCallback cb) { m_on_message = std::move(cb); }
void SetOnClose(CloseCallback cb) { m_on_close = std::move(cb); }
void SetOnError(ErrorCallback cb) { m_on_error = std::move(cb); }
// For mock mode - simulate events
void SimulateOpen();
void SimulateMessage(const std::string& data, bool binary);
void SimulateClose(int code, const std::string& reason);
void SimulateError(const std::string& error);
private:
int m_id;
std::string m_url;
WebSocketState m_state;
size_t m_max_message_size;
OpenCallback m_on_open;
MessageCallback m_on_message;
CloseCallback m_on_close;
ErrorCallback m_on_error;
};
class WebSocketManager {
public:
WebSocketManager(const std::string& app_id, const WebSocketLimits& limits = WebSocketLimits{});
~WebSocketManager();
// Configure domain restrictions (reuses HttpValidator)
void SetAllowedDomains(const std::vector<std::string>& domains);
void ClearDomainRestrictions();
// Create new WebSocket connection
// Returns connection ID on success, -1 on failure (sets error)
std::shared_ptr<WebSocket> Connect(const std::string& url, std::string& error);
// Get connection by ID
std::shared_ptr<WebSocket> GetConnection(int id);
// Close specific connection
void CloseConnection(int id, int code = 1000, const std::string& reason = "");
// Close all connections (called on app stop)
void CloseAll();
// Stats
int GetActiveConnectionCount() const;
// Access validator for testing
HttpValidator& GetValidator() { return m_validator; }
// For testing: set mock mode
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
bool IsMockMode() const { return m_mock_mode; }
private:
std::string m_app_id;
WebSocketLimits m_limits;
HttpValidator m_validator;
std::map<int, std::shared_ptr<WebSocket>> m_connections;
int m_next_id = 1;
mutable std::mutex m_mutex;
bool m_mock_mode = true;
bool ValidateUrl(const std::string& url, std::string& error);
};
// Register websocket APIs
void RegisterWebSocketAPI(lua_State* L, WebSocketManager* manager);
} // namespace mosis
```
### 2. URL Validation Changes
The HttpValidator needs to accept both `https://` and `wss://` schemes:
```cpp
// In Validate():
if (parsed->scheme != "https" && parsed->scheme != "wss") {
error = "HTTPS or WSS required, got: " + parsed->scheme;
return std::nullopt;
}
```
### 3. Lua API
```lua
-- Create WebSocket connection
local ws, err = network.websocket("wss://api.example.com/socket")
if not ws then
print("Failed to connect:", err)
return
end
-- Event handlers
ws:on("open", function()
print("Connected!")
ws:send("Hello server")
end)
ws:on("message", function(data, binary)
print("Received:", data)
end)
ws:on("close", function(code, reason)
print("Closed:", code, reason)
end)
ws:on("error", function(err)
print("Error:", err)
end)
-- Send message
ws:send("Hello")
ws:send(binaryData, true) -- binary mode
-- Close connection
ws:close()
ws:close(1000, "Normal closure")
-- Get state
local state = ws:state() -- "connecting", "open", "closing", "closed"
```
### 4. Connection Lifecycle
```
Connect(url) ──► Validating ──► Connecting ──► Open
│ │ │
│ error │ error │ message
▼ ▼ ▼
Closed Closed Handler
│
│ close/error
▼
Closed
```
### 5. Limits Enforcement
| Limit | Default | Description |
|-------|---------|-------------|
| max_connections_per_app | 5 | Maximum concurrent WebSocket connections |
| max_message_size | 1 MB | Maximum message size (send or receive) |
| idle_timeout_ms | 5 min | Close idle connections |
| connect_timeout_ms | 30 sec | Connection establishment timeout |
---
## Test Cases
### Test 1: WebSocket URL Validation
```cpp
bool Test_WebSocketUrlValidation(std::string& error_msg) {
mosis::WebSocketManager manager("test.app");
manager.ClearDomainRestrictions();
std::string err;
// WSS should be allowed
auto ws = manager.Connect("wss://example.com/socket", err);
// In mock mode, connection will fail but validation should pass
EXPECT_TRUE(err.find("mock") != std::string::npos ||
err.find("disabled") != std::string::npos ||
ws != nullptr);
// WS (plain) should be blocked
ws = manager.Connect("ws://example.com/socket", err);
EXPECT_TRUE(ws == nullptr);
EXPECT_TRUE(err.find("WSS") != std::string::npos ||
err.find("HTTPS") != std::string::npos ||
err.find("required") != std::string::npos);
return true;
}
```
### Test 2: Connection Limits
```cpp
bool Test_WebSocketConnectionLimits(std::string& error_msg) {
mosis::WebSocketLimits limits;
limits.max_connections_per_app = 2;
mosis::WebSocketManager manager("test.app", limits);
manager.ClearDomainRestrictions();
std::string err;
// Create max connections
auto ws1 = manager.Connect("wss://example.com/socket1", err);
auto ws2 = manager.Connect("wss://example.com/socket2", err);
// Third should fail (or be limited)
auto ws3 = manager.Connect("wss://example.com/socket3", err);
EXPECT_TRUE(ws3 == nullptr);
EXPECT_TRUE(err.find("limit") != std::string::npos ||
err.find("connections") != std::string::npos);
return true;
}
```
### Test 3: WebSocket Blocks Private IPs
```cpp
bool Test_WebSocketBlocksPrivateIP(std::string& error_msg) {
mosis::WebSocketManager manager("test.app");
manager.ClearDomainRestrictions();
std::string err;
std::vector<std::string> private_urls = {
"wss://127.0.0.1/socket",
"wss://localhost/socket",
"wss://10.0.0.1/socket",
"wss://192.168.1.1/socket",
"wss://169.254.169.254/socket"
};
for (const auto& url : private_urls) {
auto ws = manager.Connect(url, err);
EXPECT_TRUE(ws == nullptr);
}
return true;
}
```
### Test 4: Domain Whitelist
```cpp
bool Test_WebSocketDomainWhitelist(std::string& error_msg) {
mosis::WebSocketManager manager("test.app");
manager.SetAllowedDomains({"api.example.com"});
std::string err;
// Allowed domain - should pass validation (may fail in mock mode for other reasons)
auto ws1 = manager.Connect("wss://api.example.com/socket", err);
bool allowed_passed = (ws1 != nullptr) ||
(err.find("mock") != std::string::npos) ||
(err.find("disabled") != std::string::npos);
EXPECT_TRUE(allowed_passed);
// Disallowed domain - should fail validation
auto ws2 = manager.Connect("wss://evil.com/socket", err);
EXPECT_TRUE(ws2 == nullptr);
EXPECT_TRUE(err.find("allowed") != std::string::npos ||
err.find("whitelist") != std::string::npos ||
err.find("Domain") != std::string::npos);
return true;
}
```
### Test 5: Message Size Limits
```cpp
bool Test_WebSocketMessageLimits(std::string& error_msg) {
mosis::WebSocketLimits limits;
limits.max_message_size = 1024; // 1 KB for testing
mosis::WebSocketManager manager("test.app", limits);
manager.ClearDomainRestrictions();
// Create a mock WebSocket directly to test send limits
mosis::WebSocket ws(1, "wss://example.com/socket");
// Small message should work (if connected)
// Large message should fail
std::string large_message(2048, 'X'); // 2 KB
bool send_result = ws.Send(large_message);
EXPECT_FALSE(send_result); // Should fail - not connected and/or too large
return true;
}
```
### Test 6: Close All Connections
```cpp
bool Test_WebSocketCloseAll(std::string& error_msg) {
mosis::WebSocketManager manager("test.app");
manager.ClearDomainRestrictions();
std::string err;
// Create some connections (may fail in mock but that's ok)
manager.Connect("wss://example.com/socket1", err);
manager.Connect("wss://example.com/socket2", err);
// Close all
manager.CloseAll();
// Should have no active connections
EXPECT_TRUE(manager.GetActiveConnectionCount() == 0);
return true;
}
```
### Test 7: Lua Integration
```cpp
bool Test_WebSocketLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::WebSocketManager manager("test.app");
manager.ClearDomainRestrictions();
mosis::RegisterWebSocketAPI(sandbox.GetState(), &manager);
std::string script = R"lua(
-- Test that network.websocket exists
if not network then
error("network global not found")
end
if not network.websocket then
error("network.websocket not found")
end
-- Test validation rejection (private IP)
local ws, err = network.websocket("wss://127.0.0.1/socket")
if ws then
error("expected private IP to be blocked")
end
)lua";
bool ok = sandbox.LoadString(script, "websocket_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_WebSocketUrlValidation` - WSS required, WS blocked
- [x] `Test_WebSocketConnectionLimits` - Per-app connection limits enforced
- [x] `Test_WebSocketBlocksPrivateIP` - SSRF prevention works
- [x] `Test_WebSocketDomainWhitelist` - Domain restrictions work
- [x] `Test_WebSocketMessageLimits` - Message size limits enforced
- [x] `Test_WebSocketCloseAll` - Cleanup works
- [x] `Test_WebSocketLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 9 (HttpValidator)
---
## Notes
### Desktop vs Android Implementation
For desktop testing, WebSocketManager operates in mock mode:
- URL validation runs normally
- Connection limits are tracked
- Actual WebSocket connections are not made
- Events can be simulated for testing
On Android, the real implementation would:
1. Use Java WebSocket client through JNI
2. Handle background thread for socket I/O
3. Marshal events back to Lua thread
### Security Considerations
1. **Same-origin**: WSS URLs validated same as HTTPS
2. **Connection limits**: Prevent resource exhaustion
3. **Message limits**: Prevent memory exhaustion
4. **Idle timeout**: Automatic cleanup of abandoned connections
5. **Clean shutdown**: All connections closed on app stop
---
## Next Steps
After Milestone 10 passes:
1. Milestone 11: Virtual Hardware - Camera

521
docs/SANDBOX_MILESTONE_11.md Normal file
View File

@@ -0,0 +1,521 @@
# Milestone 11: Virtual Hardware - Camera
**Status**: Complete
**Goal**: Camera access with mandatory recording indicators and security controls.
---
## Overview
This milestone implements secure camera access for Lua apps:
- Permission required (`camera` permission)
- User gesture required to start session
- Mandatory recording indicator (system-controlled, cannot be hidden)
- Single session per app
- Frame rate limiting
- Automatic cleanup on app stop
### Key Deliverables
1. **CameraInterface class** - Session management, permission checks
2. **CameraSession class** - Active camera session wrapper
3. **Lua camera API** - `camera.start()`, session methods
4. **Recording indicator** - System-level UI notification
---
## File Structure
```
src/main/cpp/sandbox/
├── camera_interface.h # NEW - Camera API header
└── camera_interface.cpp # NEW - Camera implementation
```
---
## Implementation Details
### 1. CameraInterface Class
```cpp
// camera_interface.h
#pragma once
#include <string>
#include <memory>
#include <functional>
#include <mutex>
#include <atomic>
#include <vector>
struct lua_State;
namespace mosis {
class PermissionGate;
enum class CameraFacing {
Front,
Back
};
enum class CameraResolution {
VGA, // 640x480
HD, // 1280x720
FullHD // 1920x1080
};
struct CameraConfig {
CameraFacing facing = CameraFacing::Back;
CameraResolution resolution = CameraResolution::HD;
int max_fps = 30;
};
struct CameraFrame {
std::vector<uint8_t> data; // RGBA or JPEG data
int width;
int height;
uint64_t timestamp_ms;
bool is_jpeg;
};
class CameraSession {
public:
using FrameCallback = std::function<void(const CameraFrame& frame)>;
CameraSession(int id, const CameraConfig& config);
~CameraSession();
int GetId() const { return m_id; }
const CameraConfig& GetConfig() const { return m_config; }
bool IsActive() const { return m_active; }
// Capture single photo (returns copy of current frame)
CameraFrame Capture();
// Set frame callback (for preview)
void SetOnFrame(FrameCallback cb) { m_on_frame = std::move(cb); }
// Stop session
void Stop();
// For mock mode - simulate frame arrival
void SimulateFrame(const CameraFrame& frame);
private:
int m_id;
CameraConfig m_config;
std::atomic<bool> m_active{true};
FrameCallback m_on_frame;
CameraFrame m_last_frame;
std::mutex m_mutex;
};
class CameraInterface {
public:
CameraInterface(const std::string& app_id, PermissionGate* permissions);
~CameraInterface();
// Start camera session
// Returns session on success, nullptr on failure (sets error)
// Requires camera permission and user gesture
std::shared_ptr<CameraSession> StartSession(const CameraConfig& config, std::string& error);
// Stop active session
void StopSession();
// Check if session is active
bool HasActiveSession() const;
// Check if recording indicator should be shown
bool IsIndicatorVisible() const;
// Cleanup on app stop
void Shutdown();
// For testing
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
bool IsMockMode() const { return m_mock_mode; }
// Simulate user gesture for testing
void SimulateUserGesture();
private:
std::string m_app_id;
PermissionGate* m_permissions;
std::shared_ptr<CameraSession> m_active_session;
mutable std::mutex m_mutex;
bool m_mock_mode = true;
std::atomic<bool> m_indicator_visible{false};
// Track user gesture timing
uint64_t m_last_gesture_time = 0;
static constexpr uint64_t GESTURE_VALIDITY_MS = 5000; // 5 seconds
bool HasRecentUserGesture() const;
};
// Register camera.* APIs as globals
void RegisterCameraAPI(lua_State* L, CameraInterface* camera);
} // namespace mosis
```
### 2. Permission Requirements
Camera access requires:
1. `camera` permission declared in manifest
2. Permission granted by user (dangerous permission)
3. Recent user gesture (within 5 seconds)
```cpp
// Permission check flow
bool CameraInterface::StartSession(...) {
// 1. Check permission
if (!m_permissions->Check("camera")) {
error = "Camera permission not granted";
return nullptr;
}
// 2. Check user gesture
if (!HasRecentUserGesture()) {
error = "Camera requires user gesture";
return nullptr;
}
// 3. Check no existing session
if (m_active_session) {
error = "Camera session already active";
return nullptr;
}
// ... create session
}
```
### 3. Recording Indicator
The recording indicator is mandatory and system-controlled:
- Shown whenever camera session is active
- Cannot be hidden or obscured by app
- Positioned in system UI area
- Shows camera icon with "Recording" text
```cpp
// Indicator control
void CameraInterface::ShowIndicator() {
m_indicator_visible = true;
// In real implementation: notify system UI
}
void CameraInterface::HideIndicator() {
m_indicator_visible = false;
// In real implementation: notify system UI
}
```
### 4. Lua API
```lua
-- Start camera session (requires permission + user gesture)
local session, err = camera.start({
facing = "back", -- "front" or "back"
resolution = "720p", -- "480p", "720p", or "1080p"
fps = 30 -- max frames per second
})
if not session then
print("Failed to start camera:", err)
return
end
-- Capture single photo
local photo = session:capture()
if photo then
print("Captured", photo.width, "x", photo.height, "image")
-- photo.data contains JPEG bytes
end
-- Set frame callback (for preview)
session:on("frame", function(frame)
-- frame.data, frame.width, frame.height, frame.timestamp
end)
-- Stop session
session:stop()
-- Check if camera is active
if camera.isActive() then
print("Camera is running")
end
```
### 5. Frame Rate Limiting
To prevent resource abuse:
- Maximum 30 FPS
- Minimum 16ms between frames
- Frame delivery is asynchronous
```cpp
struct FrameRateLimiter {
int max_fps = 30;
uint64_t min_frame_interval_ms = 33; // ~30 FPS
uint64_t last_frame_time = 0;
bool ShouldDeliverFrame() {
uint64_t now = GetCurrentTimeMs();
if (now - last_frame_time >= min_frame_interval_ms) {
last_frame_time = now;
return true;
}
return false;
}
};
```
### 6. Session Lifecycle
```
User Gesture ──► camera.start() ──► Session Active ──► Indicator Shown
│ │
│ error │ session:stop()
▼ ▼
nil, err Session Closed ──► Indicator Hidden
App Stop ──► CameraInterface::Shutdown() ──► All Sessions Closed
```
---
## Test Cases
### Test 1: Requires Permission
```cpp
bool Test_CameraRequiresPermission(std::string& error_msg) {
// Create permission gate WITHOUT camera permission
PermissionGate permissions("test.app", {}, false);
mosis::CameraInterface camera("test.app", &permissions);
camera.SimulateUserGesture();
std::string err;
mosis::CameraConfig config;
auto session = camera.StartSession(config, err);
EXPECT_TRUE(session == nullptr);
EXPECT_TRUE(err.find("permission") != std::string::npos);
return true;
}
```
### Test 2: Requires User Gesture
```cpp
bool Test_CameraRequiresUserGesture(std::string& error_msg) {
// Create permission gate WITH camera permission
PermissionGate permissions("test.app", {"camera"}, false);
permissions.Grant("camera");
mosis::CameraInterface camera("test.app", &permissions);
// Note: NOT calling SimulateUserGesture()
std::string err;
mosis::CameraConfig config;
auto session = camera.StartSession(config, err);
EXPECT_TRUE(session == nullptr);
EXPECT_TRUE(err.find("gesture") != std::string::npos);
return true;
}
```
### Test 3: Shows Indicator
```cpp
bool Test_CameraShowsIndicator(std::string& error_msg) {
PermissionGate permissions("test.app", {"camera"}, false);
permissions.Grant("camera");
mosis::CameraInterface camera("test.app", &permissions);
camera.SimulateUserGesture();
EXPECT_FALSE(camera.IsIndicatorVisible());
std::string err;
mosis::CameraConfig config;
auto session = camera.StartSession(config, err);
// Indicator should be visible while session is active
EXPECT_TRUE(camera.IsIndicatorVisible());
session->Stop();
// Indicator should be hidden after stop
EXPECT_FALSE(camera.IsIndicatorVisible());
return true;
}
```
### Test 4: Single Session Only
```cpp
bool Test_CameraSingleSession(std::string& error_msg) {
PermissionGate permissions("test.app", {"camera"}, false);
permissions.Grant("camera");
mosis::CameraInterface camera("test.app", &permissions);
camera.SimulateUserGesture();
std::string err;
mosis::CameraConfig config;
// First session should succeed
auto session1 = camera.StartSession(config, err);
EXPECT_TRUE(session1 != nullptr);
// Second session should fail
camera.SimulateUserGesture();
auto session2 = camera.StartSession(config, err);
EXPECT_TRUE(session2 == nullptr);
EXPECT_TRUE(err.find("active") != std::string::npos ||
err.find("already") != std::string::npos);
return true;
}
```
### Test 5: Stops On App Stop
```cpp
bool Test_CameraStopsOnAppStop(std::string& error_msg) {
PermissionGate permissions("test.app", {"camera"}, false);
permissions.Grant("camera");
mosis::CameraInterface camera("test.app", &permissions);
camera.SimulateUserGesture();
std::string err;
mosis::CameraConfig config;
auto session = camera.StartSession(config, err);
EXPECT_TRUE(session != nullptr);
EXPECT_TRUE(camera.HasActiveSession());
// Simulate app stop
camera.Shutdown();
EXPECT_FALSE(camera.HasActiveSession());
EXPECT_FALSE(camera.IsIndicatorVisible());
return true;
}
```
### Test 6: Lua Integration
```cpp
bool Test_CameraLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
ctx.permissions = {"camera"};
LuaSandbox sandbox(ctx);
PermissionGate permissions("test.app", {"camera"}, false);
permissions.Grant("camera");
mosis::CameraInterface camera("test.app", &permissions);
camera.SimulateUserGesture();
mosis::RegisterCameraAPI(sandbox.GetState(), &camera);
std::string script = R"lua(
-- Test that camera global exists
if not camera then
error("camera global not found")
end
if not camera.start then
error("camera.start not found")
end
if not camera.isActive then
error("camera.isActive not found")
end
-- isActive should be false initially
if camera.isActive() then
error("camera should not be active initially")
end
)lua";
bool ok = sandbox.LoadString(script, "camera_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_CameraRequiresPermission` - Permission check works
- [x] `Test_CameraRequiresUserGesture` - User gesture required
- [x] `Test_CameraShowsIndicator` - Recording indicator shown/hidden
- [x] `Test_CameraSingleSession` - Only one session allowed
- [x] `Test_CameraStopsOnShutdown` - Cleanup on shutdown
- [x] `Test_CameraLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate + user gesture)
- Milestone 3 (AuditLog)
---
## Notes
### Desktop vs Android Implementation
For desktop testing, CameraInterface operates in mock mode:
- Permission and gesture checks run normally
- Indicator state is tracked but not displayed
- No actual camera hardware access
- Frames can be simulated for testing
On Android, the real implementation would:
1. Use Camera2 API through JNI
2. Display system-level recording indicator
3. Handle camera hardware lifecycle
4. Deliver frames through SurfaceTexture
### Security Considerations
1. **Permission**: Camera access is a dangerous permission requiring user grant
2. **User gesture**: Prevents background camera activation
3. **Indicator**: User always knows when camera is active
4. **Single session**: Prevents resource abuse
5. **Cleanup**: Sessions closed when app stops
### Future Integration
The camera will integrate with game engines to:
1. Stream frames to VR/AR display
2. Support mixed reality passthrough
3. Enable barcode/QR scanning
---
## Next Steps
After Milestone 11 passes:
1. Milestone 12: Virtual Hardware - Microphone

507
docs/SANDBOX_MILESTONE_12.md Normal file
View File

@@ -0,0 +1,507 @@
# Milestone 12: Virtual Hardware - Microphone
**Status**: Complete
**Goal**: Audio recording with mandatory recording indicators and security controls.
---
## Overview
This milestone implements secure microphone access for Lua apps:
- Permission required (`microphone` permission)
- User gesture required to start recording
- Mandatory recording indicator (system-controlled, cannot be hidden)
- Single recording session per app
- Sample rate limiting
- Automatic cleanup on app stop
### Key Deliverables
1. **MicrophoneInterface class** - Session management, permission checks
2. **RecordingSession class** - Active recording session wrapper
3. **Lua microphone API** - `microphone.start()`, session methods
4. **Recording indicator** - System-level UI notification
---
## File Structure
```
src/main/cpp/sandbox/
├── microphone_interface.h # NEW - Microphone API header
└── microphone_interface.cpp # NEW - Microphone implementation
```
---
## Implementation Details
### 1. MicrophoneInterface Class
```cpp
// microphone_interface.h
#pragma once
#include <string>
#include <memory>
#include <functional>
#include <mutex>
#include <atomic>
#include <vector>
#include <chrono>
struct lua_State;
namespace mosis {
class PermissionGate;
enum class AudioFormat {
PCM_16BIT,
PCM_FLOAT
};
struct RecordingConfig {
int sample_rate = 44100; // 8000, 16000, 22050, 44100, 48000
int channels = 1; // 1 (mono) or 2 (stereo)
AudioFormat format = AudioFormat::PCM_16BIT;
};
struct AudioBuffer {
std::vector<uint8_t> data;
int sample_rate;
int channels;
AudioFormat format;
uint64_t timestamp_ms;
int sample_count;
};
class RecordingSession {
public:
using BufferCallback = std::function<void(const AudioBuffer& buffer)>;
RecordingSession(int id, const RecordingConfig& config);
~RecordingSession();
int GetId() const { return m_id; }
const RecordingConfig& GetConfig() const { return m_config; }
bool IsActive() const { return m_active; }
// Get accumulated audio data
AudioBuffer GetBuffer();
// Set buffer callback (for streaming)
void SetOnBuffer(BufferCallback cb) { m_on_buffer = std::move(cb); }
// Stop recording
void Stop();
// For mock mode - simulate audio data arrival
void SimulateBuffer(const AudioBuffer& buffer);
private:
int m_id;
RecordingConfig m_config;
std::atomic<bool> m_active{true};
BufferCallback m_on_buffer;
AudioBuffer m_accumulated;
mutable std::mutex m_mutex;
};
class MicrophoneInterface {
public:
MicrophoneInterface(const std::string& app_id, PermissionGate* permissions);
~MicrophoneInterface();
// Start recording session
// Returns session on success, nullptr on failure (sets error)
// Requires microphone permission and user gesture
std::shared_ptr<RecordingSession> StartSession(const RecordingConfig& config, std::string& error);
// Stop active session
void StopSession();
// Check if session is active
bool HasActiveSession() const;
// Check if recording indicator should be shown
bool IsIndicatorVisible() const;
// Cleanup on app stop
void Shutdown();
// For testing
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
bool IsMockMode() const { return m_mock_mode; }
// Simulate user gesture for testing
void SimulateUserGesture();
private:
std::string m_app_id;
PermissionGate* m_permissions;
std::shared_ptr<RecordingSession> m_active_session;
mutable std::mutex m_mutex;
bool m_mock_mode = true;
std::atomic<bool> m_indicator_visible{false};
int m_next_session_id = 1;
// Track user gesture timing
std::chrono::steady_clock::time_point m_last_gesture_time;
bool m_has_gesture = false;
static constexpr int GESTURE_VALIDITY_MS = 5000; // 5 seconds
bool HasRecentUserGesture() const;
void ShowIndicator();
void HideIndicator();
};
// Register microphone.* APIs as globals
void RegisterMicrophoneAPI(lua_State* L, MicrophoneInterface* microphone);
} // namespace mosis
```
### 2. Permission Requirements
Microphone access requires:
1. `microphone` permission declared in manifest
2. Permission granted by user (dangerous permission)
3. Recent user gesture (within 5 seconds)
```cpp
// Permission check flow
bool MicrophoneInterface::StartSession(...) {
// 1. Check permission
if (!m_permissions->HasPermission("microphone")) {
error = "Microphone permission not granted";
return nullptr;
}
// 2. Check user gesture
if (!HasRecentUserGesture()) {
error = "Microphone requires user gesture";
return nullptr;
}
// 3. Check no existing session
if (m_active_session) {
error = "Recording session already active";
return nullptr;
}
// ... create session
}
```
### 3. Recording Indicator
The recording indicator is mandatory and system-controlled:
- Shown whenever microphone session is active
- Cannot be hidden or obscured by app
- Positioned in system UI area
- Shows microphone icon with "Recording" text
### 4. Lua API
```lua
-- Start recording session (requires permission + user gesture)
local session, err = microphone.start({
sampleRate = 44100, -- 8000, 16000, 22050, 44100, 48000
channels = 1, -- 1 (mono) or 2 (stereo)
format = "pcm16" -- "pcm16" or "float"
})
if not session then
print("Failed to start recording:", err)
return
end
-- Set buffer callback (for streaming audio)
session:on("buffer", function(buffer)
-- buffer.data, buffer.sampleRate, buffer.channels, buffer.sampleCount
end)
-- Get accumulated audio data
local audio = session:getBuffer()
if audio then
print("Recorded", audio.sampleCount, "samples at", audio.sampleRate, "Hz")
end
-- Stop recording
session:stop()
-- Check if microphone is active
if microphone.isActive() then
print("Microphone is recording")
end
```
### 5. Sample Rate Validation
To prevent resource abuse:
- Allowed sample rates: 8000, 16000, 22050, 44100, 48000
- Maximum: 48000 Hz
- Channels: 1 (mono) or 2 (stereo)
### 6. Session Lifecycle
```
User Gesture ──► microphone.start() ──► Session Active ──► Indicator Shown
│ │
│ error │ session:stop()
▼ ▼
nil, err Session Closed ──► Indicator Hidden
App Stop ──► MicrophoneInterface::Shutdown() ──► All Sessions Closed
```
---
## Test Cases
### Test 1: Requires Permission
```cpp
bool Test_MicrophoneRequiresPermission(std::string& error_msg) {
// Create context WITHOUT microphone permission
SandboxContext ctx;
ctx.app_id = "test.app";
ctx.permissions = {};
ctx.is_system_app = false;
PermissionGate permissions(ctx);
mosis::MicrophoneInterface mic("test.app", &permissions);
mic.SimulateUserGesture();
std::string err;
mosis::RecordingConfig config;
auto session = mic.StartSession(config, err);
EXPECT_TRUE(session == nullptr);
EXPECT_TRUE(err.find("permission") != std::string::npos);
return true;
}
```
### Test 2: Requires User Gesture
```cpp
bool Test_MicrophoneRequiresUserGesture(std::string& error_msg) {
SandboxContext ctx;
ctx.app_id = "test.app";
ctx.permissions = {"microphone"};
ctx.is_system_app = false;
PermissionGate permissions(ctx);
permissions.GrantPermission("microphone");
mosis::MicrophoneInterface mic("test.app", &permissions);
// Note: NOT calling SimulateUserGesture()
std::string err;
mosis::RecordingConfig config;
auto session = mic.StartSession(config, err);
EXPECT_TRUE(session == nullptr);
EXPECT_TRUE(err.find("gesture") != std::string::npos);
return true;
}
```
### Test 3: Shows Indicator
```cpp
bool Test_MicrophoneShowsIndicator(std::string& error_msg) {
SandboxContext ctx;
ctx.app_id = "test.app";
ctx.permissions = {"microphone"};
ctx.is_system_app = false;
PermissionGate permissions(ctx);
permissions.GrantPermission("microphone");
mosis::MicrophoneInterface mic("test.app", &permissions);
mic.SimulateUserGesture();
EXPECT_FALSE(mic.IsIndicatorVisible());
std::string err;
mosis::RecordingConfig config;
auto session = mic.StartSession(config, err);
EXPECT_TRUE(mic.IsIndicatorVisible());
mic.StopSession();
EXPECT_FALSE(mic.IsIndicatorVisible());
return true;
}
```
### Test 4: Single Session Only
```cpp
bool Test_MicrophoneSingleSession(std::string& error_msg) {
SandboxContext ctx;
ctx.app_id = "test.app";
ctx.permissions = {"microphone"};
ctx.is_system_app = false;
PermissionGate permissions(ctx);
permissions.GrantPermission("microphone");
mosis::MicrophoneInterface mic("test.app", &permissions);
mic.SimulateUserGesture();
std::string err;
mosis::RecordingConfig config;
// First session should succeed
auto session1 = mic.StartSession(config, err);
EXPECT_TRUE(session1 != nullptr);
// Second session should fail
mic.SimulateUserGesture();
auto session2 = mic.StartSession(config, err);
EXPECT_TRUE(session2 == nullptr);
EXPECT_TRUE(err.find("active") != std::string::npos ||
err.find("already") != std::string::npos);
return true;
}
```
### Test 5: Stops On App Stop
```cpp
bool Test_MicrophoneStopsOnShutdown(std::string& error_msg) {
SandboxContext ctx;
ctx.app_id = "test.app";
ctx.permissions = {"microphone"};
ctx.is_system_app = false;
PermissionGate permissions(ctx);
permissions.GrantPermission("microphone");
mosis::MicrophoneInterface mic("test.app", &permissions);
mic.SimulateUserGesture();
std::string err;
mosis::RecordingConfig config;
auto session = mic.StartSession(config, err);
EXPECT_TRUE(session != nullptr);
EXPECT_TRUE(mic.HasActiveSession());
// Simulate app stop
mic.Shutdown();
EXPECT_FALSE(mic.HasActiveSession());
EXPECT_FALSE(mic.IsIndicatorVisible());
return true;
}
```
### Test 6: Lua Integration
```cpp
bool Test_MicrophoneLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
ctx.permissions = {"microphone"};
LuaSandbox sandbox(ctx);
PermissionGate permissions(ctx);
permissions.GrantPermission("microphone");
mosis::MicrophoneInterface mic("test.app", &permissions);
mic.SimulateUserGesture();
mosis::RegisterMicrophoneAPI(sandbox.GetState(), &mic);
std::string script = R"lua(
-- Test that microphone global exists
if not microphone then
error("microphone global not found")
end
if not microphone.start then
error("microphone.start not found")
end
if not microphone.isActive then
error("microphone.isActive not found")
end
-- isActive should be false initially
if microphone.isActive() then
error("microphone should not be active initially")
end
)lua";
bool ok = sandbox.LoadString(script, "microphone_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_MicrophoneRequiresPermission` - Permission check works
- [x] `Test_MicrophoneRequiresUserGesture` - User gesture required
- [x] `Test_MicrophoneShowsIndicator` - Recording indicator shown/hidden
- [x] `Test_MicrophoneSingleSession` - Only one session allowed
- [x] `Test_MicrophoneStopsOnShutdown` - Cleanup on shutdown
- [x] `Test_MicrophoneLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate + user gesture)
- Milestone 3 (AuditLog)
---
## Notes
### Desktop vs Android Implementation
For desktop testing, MicrophoneInterface operates in mock mode:
- Permission and gesture checks run normally
- Indicator state is tracked but not displayed
- No actual microphone hardware access
- Audio buffers can be simulated for testing
On Android, the real implementation would:
1. Use AudioRecord API through JNI
2. Display system-level recording indicator
3. Handle audio hardware lifecycle
4. Deliver audio buffers through native callbacks
### Security Considerations
1. **Permission**: Microphone access is a dangerous permission requiring user grant
2. **User gesture**: Prevents background audio recording
3. **Indicator**: User always knows when microphone is active
4. **Single session**: Prevents resource abuse
5. **Cleanup**: Sessions closed when app stops
### Future Integration
The microphone will integrate with:
1. Voice communication in multiplayer games
2. Voice commands/speech recognition
3. Audio messaging features
---
## Next Steps
After Milestone 12 passes:
1. Milestone 13: Virtual Hardware - Audio Output

467
docs/SANDBOX_MILESTONE_13.md Normal file
View File

@@ -0,0 +1,467 @@
# Milestone 13: Virtual Hardware - Audio Output
**Status**: Complete
**Goal**: Safe audio playback with volume limits and concurrent sound management.
---
## Overview
This milestone implements secure audio output for Lua apps:
- No special permission required (normal capability)
- System volume limit (app cannot exceed system volume)
- Concurrent sound limit (10 per app)
- Automatic cleanup on app stop
- Support for PCM and basic audio formats
### Key Deliverables
1. **AudioOutputInterface class** - Sound playback management
2. **SoundPlayer class** - Individual sound instance
3. **Lua audio API** - `audio.play()`, player methods
4. **Volume enforcement** - System volume cap
---
## File Structure
```
src/main/cpp/sandbox/
├── audio_output.h # NEW - Audio output API header
└── audio_output.cpp # NEW - Audio output implementation
```
---
## Implementation Details
### 1. AudioOutputInterface Class
```cpp
// audio_output.h
#pragma once
#include <string>
#include <memory>
#include <functional>
#include <mutex>
#include <atomic>
#include <vector>
#include <unordered_map>
struct lua_State;
namespace mosis {
enum class AudioState {
Playing,
Paused,
Stopped
};
struct PlaybackConfig {
float volume = 1.0f; // 0.0 to 1.0
bool loop = false;
float pitch = 1.0f; // 0.5 to 2.0
float pan = 0.0f; // -1.0 (left) to 1.0 (right)
};
struct AudioData {
std::vector<uint8_t> data;
int sample_rate = 44100;
int channels = 1;
int bits_per_sample = 16;
};
class SoundPlayer {
public:
using EndCallback = std::function<void()>;
SoundPlayer(int id, const AudioData& audio, const PlaybackConfig& config);
~SoundPlayer();
int GetId() const { return m_id; }
AudioState GetState() const { return m_state; }
float GetVolume() const { return m_volume; }
bool IsLooping() const { return m_loop; }
// Control
void Play();
void Pause();
void Stop();
void SetVolume(float volume);
void SetPitch(float pitch);
void SetPan(float pan);
// Callback when playback ends
void SetOnEnd(EndCallback cb) { m_on_end = std::move(cb); }
// For mock mode - simulate playback completion
void SimulateEnd();
// Get playback position (0.0 to 1.0)
float GetPosition() const { return m_position; }
private:
int m_id;
AudioData m_audio;
std::atomic<AudioState> m_state{AudioState::Stopped};
float m_volume = 1.0f;
float m_pitch = 1.0f;
float m_pan = 0.0f;
bool m_loop = false;
float m_position = 0.0f;
EndCallback m_on_end;
mutable std::mutex m_mutex;
};
class AudioOutputInterface {
public:
AudioOutputInterface(const std::string& app_id);
~AudioOutputInterface();
// Play a sound
// Returns player on success, nullptr on failure (sets error)
std::shared_ptr<SoundPlayer> Play(const AudioData& audio, const PlaybackConfig& config, std::string& error);
// Stop a specific player
void StopPlayer(int player_id);
// Stop all sounds for this app
void StopAll();
// Get active player count
size_t GetActivePlayerCount() const;
// Get system volume (0.0 to 1.0)
float GetSystemVolume() const { return m_system_volume; }
// Set system volume (for testing)
void SetSystemVolume(float volume);
// Cleanup on app stop
void Shutdown();
// For testing
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
bool IsMockMode() const { return m_mock_mode; }
// Get player by ID
std::shared_ptr<SoundPlayer> GetPlayer(int id);
private:
std::string m_app_id;
std::unordered_map<int, std::shared_ptr<SoundPlayer>> m_players;
mutable std::mutex m_mutex;
bool m_mock_mode = true;
float m_system_volume = 1.0f;
int m_next_player_id = 1;
static constexpr int MAX_CONCURRENT_SOUNDS = 10;
void CleanupStoppedPlayers();
};
// Register audio.* APIs as globals
void RegisterAudioAPI(lua_State* L, AudioOutputInterface* audio);
} // namespace mosis
```
### 2. Volume Enforcement
App volume is capped at system volume:
- Effective volume = app_volume × system_volume
- If system volume is 0.5 and app sets 1.0, actual volume is 0.5
- Apps cannot play louder than system setting
### 3. Concurrent Sound Limit
Maximum 10 simultaneous sounds per app:
- Prevents resource exhaustion
- New sounds fail when limit reached
- Stopped sounds don't count toward limit
### 4. Lua API
```lua
-- Play a sound
local player = audio.play(soundData, {
volume = 0.8, -- 0.0 to 1.0 (default 1.0)
loop = false, -- Loop playback (default false)
pitch = 1.0, -- Playback speed (default 1.0)
pan = 0.0 -- -1.0 left, 0.0 center, 1.0 right
})
if not player then
print("Failed to play sound")
return
end
-- Player methods
player:pause()
player:resume()
player:stop()
player:setVolume(0.5)
player:setPitch(1.2)
player:setPan(-0.5)
-- Check state
local state = player:getState() -- "playing", "paused", "stopped"
local pos = player:getPosition() -- 0.0 to 1.0
-- Event callback
player:on("ended", function()
print("Sound finished playing")
end)
-- Stop all sounds
audio.stopAll()
-- Get active sound count
local count = audio.getActiveCount()
```
### 5. Audio Data Format
For mock mode testing, audio data is a table:
```lua
local soundData = {
data = binaryString, -- Raw PCM data
sampleRate = 44100, -- Hz
channels = 1, -- 1 (mono) or 2 (stereo)
bitsPerSample = 16 -- 8 or 16
}
```
In real implementation, this would interface with Android AudioTrack or game engine audio systems.
---
## Test Cases
### Test 1: Basic Playback
```cpp
bool Test_AudioPlaysSound(std::string& error_msg) {
mosis::AudioOutputInterface audio("test.app");
mosis::AudioData data;
data.sample_rate = 44100;
data.channels = 1;
data.bits_per_sample = 16;
data.data = std::vector<uint8_t>(1000); // Dummy data
mosis::PlaybackConfig config;
std::string err;
auto player = audio.Play(data, config, err);
EXPECT_TRUE(player != nullptr);
EXPECT_TRUE(player->GetState() == mosis::AudioState::Playing);
return true;
}
```
### Test 2: Volume Limit
```cpp
bool Test_AudioRespectsSystemVolume(std::string& error_msg) {
mosis::AudioOutputInterface audio("test.app");
audio.SetSystemVolume(0.5f);
mosis::AudioData data;
data.data = std::vector<uint8_t>(1000);
mosis::PlaybackConfig config;
config.volume = 1.0f; // App requests full volume
std::string err;
auto player = audio.Play(data, config, err);
// Player is created with requested volume
EXPECT_TRUE(player != nullptr);
EXPECT_TRUE(player->GetVolume() == 1.0f);
// But effective volume is capped by system
// (In real implementation, audio subsystem applies this)
return true;
}
```
### Test 3: Concurrent Limit
```cpp
bool Test_AudioLimitsConcurrent(std::string& error_msg) {
mosis::AudioOutputInterface audio("test.app");
mosis::AudioData data;
data.data = std::vector<uint8_t>(1000);
mosis::PlaybackConfig config;
// Create MAX_CONCURRENT_SOUNDS players
std::vector<std::shared_ptr<mosis::SoundPlayer>> players;
for (int i = 0; i < 10; i++) {
std::string err;
auto player = audio.Play(data, config, err);
EXPECT_TRUE(player != nullptr);
players.push_back(player);
}
// 11th should fail
std::string err;
auto extra = audio.Play(data, config, err);
EXPECT_TRUE(extra == nullptr);
EXPECT_TRUE(err.find("limit") != std::string::npos ||
err.find("concurrent") != std::string::npos);
return true;
}
```
### Test 4: Stop All
```cpp
bool Test_AudioStopAll(std::string& error_msg) {
mosis::AudioOutputInterface audio("test.app");
mosis::AudioData data;
data.data = std::vector<uint8_t>(1000);
mosis::PlaybackConfig config;
std::string err;
auto player1 = audio.Play(data, config, err);
auto player2 = audio.Play(data, config, err);
EXPECT_TRUE(audio.GetActivePlayerCount() == 2);
audio.StopAll();
EXPECT_TRUE(audio.GetActivePlayerCount() == 0);
return true;
}
```
### Test 5: Cleanup on Shutdown
```cpp
bool Test_AudioStopsOnShutdown(std::string& error_msg) {
mosis::AudioOutputInterface audio("test.app");
mosis::AudioData data;
data.data = std::vector<uint8_t>(1000);
mosis::PlaybackConfig config;
std::string err;
auto player = audio.Play(data, config, err);
EXPECT_TRUE(player != nullptr);
// Simulate app stop
audio.Shutdown();
EXPECT_TRUE(audio.GetActivePlayerCount() == 0);
return true;
}
```
### Test 6: Lua Integration
```cpp
bool Test_AudioLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::AudioOutputInterface audio("test.app");
mosis::RegisterAudioAPI(sandbox.GetState(), &audio);
std::string script = R"lua(
-- Test that audio global exists
if not audio then
error("audio global not found")
end
if not audio.play then
error("audio.play not found")
end
if not audio.stopAll then
error("audio.stopAll not found")
end
if not audio.getActiveCount then
error("audio.getActiveCount not found")
end
-- Active count should be 0 initially
if audio.getActiveCount() ~= 0 then
error("should have no active sounds initially")
end
)lua";
bool ok = sandbox.LoadString(script, "audio_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_AudioPlaysSound` - Basic playback works
- [x] `Test_AudioRespectsSystemVolume` - Volume capped by system
- [x] `Test_AudioLimitsConcurrent` - 10 sound limit enforced
- [x] `Test_AudioStopAll` - Stop all sounds works
- [x] `Test_AudioStopsOnShutdown` - Cleanup on shutdown
- [x] `Test_AudioLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
---
## Notes
### Desktop vs Android Implementation
For desktop testing, AudioOutputInterface operates in mock mode:
- Players are created and tracked
- State transitions work (play/pause/stop)
- No actual audio hardware access
- Playback completion can be simulated for testing
On Android, the real implementation would:
1. Use AudioTrack API through JNI
2. Handle audio focus properly
3. Respect system volume settings
4. Mix multiple sounds appropriately
### Security Considerations
1. **No permission required**: Audio playback is a normal capability
2. **Volume limit**: Cannot exceed system volume setting
3. **Concurrent limit**: Prevents resource exhaustion
4. **Cleanup**: All sounds stopped when app stops
### Future Integration
Audio output will integrate with:
1. Game sound effects
2. UI feedback sounds
3. Music playback
4. Voice chat output (Milestone 12 companion)
---
## Next Steps
After Milestone 13 passes:
1. Milestone 14: Virtual Hardware - Location

561
docs/SANDBOX_MILESTONE_14.md Normal file
View File

@@ -0,0 +1,561 @@
# Milestone 14: Virtual Hardware - Location
**Status**: Complete
**Goal**: Location access with privacy controls and precision levels.
---
## Overview
This milestone implements secure location access for Lua apps:
- Two permission levels: coarse (city-level) and fine (GPS precision)
- Rate limiting to prevent tracking abuse
- Automatic precision reduction for coarse permission
- Watch mode for continuous updates
- Automatic cleanup on app stop
### Key Deliverables
1. **LocationInterface class** - Location access management
2. **LocationWatch class** - Continuous position monitoring
3. **Lua location API** - `location.getCurrentPosition()`, `location.watchPosition()`
4. **Precision enforcement** - Coarse vs fine accuracy
---
## File Structure
```
src/main/cpp/sandbox/
├── location_interface.h # NEW - Location API header
└── location_interface.cpp # NEW - Location implementation
```
---
## Implementation Details
### 1. LocationInterface Class
```cpp
// location_interface.h
#pragma once
#include <string>
#include <memory>
#include <functional>
#include <mutex>
#include <atomic>
#include <vector>
#include <unordered_map>
#include <chrono>
struct lua_State;
namespace mosis {
struct Position {
double latitude = 0.0;
double longitude = 0.0;
double accuracy = 0.0; // meters
double altitude = 0.0; // meters (optional)
double altitude_accuracy = 0.0; // meters (optional)
double heading = 0.0; // degrees (optional)
double speed = 0.0; // m/s (optional)
int64_t timestamp = 0; // milliseconds since epoch
};
struct LocationOptions {
bool enable_high_accuracy = false;
int timeout_ms = 30000; // max time to wait
int max_age_ms = 0; // accept cached position this old
};
enum class LocationError {
None,
PermissionDenied,
PositionUnavailable,
Timeout,
RateLimited,
AlreadyWatching
};
class LocationWatch {
public:
using PositionCallback = std::function<void(const Position&)>;
using ErrorCallback = std::function<void(LocationError, const std::string&)>;
LocationWatch(int id, const LocationOptions& options);
~LocationWatch();
int GetId() const { return m_id; }
bool IsActive() const { return m_active; }
void Start();
void Stop();
void SetOnPosition(PositionCallback cb) { m_on_position = std::move(cb); }
void SetOnError(ErrorCallback cb) { m_on_error = std::move(cb); }
// For mock mode - simulate position update
void SimulatePosition(const Position& pos);
void SimulateError(LocationError error, const std::string& message);
private:
int m_id;
LocationOptions m_options;
std::atomic<bool> m_active{false};
PositionCallback m_on_position;
ErrorCallback m_on_error;
mutable std::mutex m_mutex;
};
class LocationInterface {
public:
LocationInterface(const std::string& app_id);
~LocationInterface();
// Check if app has location permission
bool HasFinePermission() const { return m_has_fine_permission; }
bool HasCoarsePermission() const { return m_has_coarse_permission; }
void SetFinePermission(bool granted) { m_has_fine_permission = granted; }
void SetCoarsePermission(bool granted) { m_has_coarse_permission = granted; }
// Get current position (one-shot)
// Returns true if request started, false on error
bool GetCurrentPosition(
const LocationOptions& options,
std::function<void(const Position&)> success,
std::function<void(LocationError, const std::string&)> error,
std::string& out_error
);
// Watch position (continuous)
// Returns watch on success, nullptr on failure
std::shared_ptr<LocationWatch> WatchPosition(
const LocationOptions& options,
std::string& error
);
// Stop a specific watch
void ClearWatch(int watch_id);
// Stop all watches for this app
void ClearAllWatches();
// Get active watch count
size_t GetActiveWatchCount() const;
// Cleanup on app stop
void Shutdown();
// For testing
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
bool IsMockMode() const { return m_mock_mode; }
// Mock mode: set the position to return
void SetMockPosition(const Position& pos) { m_mock_position = pos; }
// Get watch by ID
std::shared_ptr<LocationWatch> GetWatch(int id);
// For rate limiting
bool CheckRateLimit();
private:
std::string m_app_id;
std::unordered_map<int, std::shared_ptr<LocationWatch>> m_watches;
mutable std::mutex m_mutex;
bool m_mock_mode = true;
bool m_has_fine_permission = false;
bool m_has_coarse_permission = false;
Position m_mock_position;
int m_next_watch_id = 1;
std::chrono::steady_clock::time_point m_last_request_time;
static constexpr int MAX_WATCHES = 5;
static constexpr int RATE_LIMIT_MS = 1000; // 1 request per second
static constexpr double COARSE_ACCURACY_M = 1000.0; // 1km for coarse
// Apply precision reduction for coarse permission
Position ApplyPrecision(const Position& pos) const;
void CleanupStoppedWatches();
};
// Register location.* APIs as globals
void RegisterLocationAPI(lua_State* L, LocationInterface* location);
} // namespace mosis
```
### 2. Permission Levels
Two permission levels with different accuracy:
- `location.fine` - Full GPS precision (meters)
- `location.coarse` - City-level precision (~1km)
### 3. Precision Reduction for Coarse
When only coarse permission is granted:
- Latitude/longitude rounded to ~1km grid
- Accuracy reported as minimum 1000 meters
- Altitude, heading, speed not provided
```cpp
Position LocationInterface::ApplyPrecision(const Position& pos) const {
if (m_has_fine_permission) {
return pos; // Full precision
}
// Coarse: round to ~1km (0.01 degrees ≈ 1.1km)
Position coarse = pos;
coarse.latitude = std::round(pos.latitude * 100.0) / 100.0;
coarse.longitude = std::round(pos.longitude * 100.0) / 100.0;
coarse.accuracy = std::max(pos.accuracy, COARSE_ACCURACY_M);
coarse.altitude = 0.0;
coarse.altitude_accuracy = 0.0;
coarse.heading = 0.0;
coarse.speed = 0.0;
return coarse;
}
```
### 4. Rate Limiting
Maximum 1 location request per second per app:
- Prevents continuous tracking abuse
- Applies to both getCurrentPosition and watchPosition updates
### 5. Lua API
```lua
-- Get current position (one-shot)
location.getCurrentPosition(function(pos)
print("Lat:", pos.latitude)
print("Lon:", pos.longitude)
print("Accuracy:", pos.accuracy, "meters")
print("Timestamp:", pos.timestamp)
end, function(error, message)
print("Error:", error, message)
end, {
enableHighAccuracy = true, -- Request fine permission
timeout = 30000, -- Max wait time in ms
maxAge = 0 -- Don't accept cached position
})
-- Watch position (continuous updates)
local watch = location.watchPosition(function(pos)
print("Updated position:", pos.latitude, pos.longitude)
end, function(error, message)
print("Watch error:", error, message)
end, {
enableHighAccuracy = false -- Use coarse permission
})
-- Stop watching
watch:stop()
-- Or clear by ID
location.clearWatch(watch:getId())
-- Get active watch count
local count = location.getWatchCount()
```
### 6. Error Codes
```lua
-- Error values passed to error callback
"PERMISSION_DENIED" -- No location permission
"POSITION_UNAVAILABLE" -- Location not available
"TIMEOUT" -- Request timed out
"RATE_LIMITED" -- Too many requests
"ALREADY_WATCHING" -- Max watches reached
```
---
## Test Cases
### Test 1: Requires Permission
```cpp
bool Test_LocationRequiresPermission(std::string& error_msg) {
mosis::LocationInterface location("test.app");
// No permissions granted
mosis::LocationOptions options;
std::string err;
bool started = location.GetCurrentPosition(
options,
[](const mosis::Position&) {},
[](mosis::LocationError, const std::string&) {},
err
);
EXPECT_TRUE(!started);
EXPECT_TRUE(err.find("permission") != std::string::npos);
return true;
}
```
### Test 2: Coarse Reduces Precision
```cpp
bool Test_LocationCoarseReducesPrecision(std::string& error_msg) {
mosis::LocationInterface location("test.app");
location.SetCoarsePermission(true); // Only coarse, not fine
// Set mock position with high precision
mosis::Position precise;
precise.latitude = 37.7749295;
precise.longitude = -122.4194155;
precise.accuracy = 5.0;
precise.altitude = 100.0;
location.SetMockPosition(precise);
bool got_position = false;
mosis::Position received;
mosis::LocationOptions options;
std::string err;
location.GetCurrentPosition(
options,
[&](const mosis::Position& pos) {
got_position = true;
received = pos;
},
[](mosis::LocationError, const std::string&) {},
err
);
// Trigger mock callback
// In mock mode, position is returned immediately
EXPECT_TRUE(got_position);
// Should be rounded to ~0.01 degrees
EXPECT_TRUE(std::abs(received.latitude - 37.77) < 0.001);
EXPECT_TRUE(std::abs(received.longitude - (-122.42)) < 0.001);
// Accuracy should be at least 1000m
EXPECT_TRUE(received.accuracy >= 1000.0);
// Fine details should be zeroed
EXPECT_TRUE(received.altitude == 0.0);
return true;
}
```
### Test 3: Rate Limiting
```cpp
bool Test_LocationRateLimits(std::string& error_msg) {
mosis::LocationInterface location("test.app");
location.SetCoarsePermission(true);
mosis::Position pos;
pos.latitude = 37.77;
pos.longitude = -122.42;
location.SetMockPosition(pos);
mosis::LocationOptions options;
// First request should succeed
std::string err;
bool ok1 = location.GetCurrentPosition(
options,
[](const mosis::Position&) {},
[](mosis::LocationError, const std::string&) {},
err
);
EXPECT_TRUE(ok1);
// Immediate second request should be rate limited
bool ok2 = location.GetCurrentPosition(
options,
[](const mosis::Position&) {},
[](mosis::LocationError, const std::string&) {},
err
);
EXPECT_TRUE(!ok2);
EXPECT_TRUE(err.find("rate") != std::string::npos ||
err.find("limit") != std::string::npos);
return true;
}
```
### Test 4: Watch Position
```cpp
bool Test_LocationWatchPosition(std::string& error_msg) {
mosis::LocationInterface location("test.app");
location.SetFinePermission(true);
std::string err;
auto watch = location.WatchPosition({}, err);
EXPECT_TRUE(watch != nullptr);
EXPECT_TRUE(watch->IsActive());
EXPECT_TRUE(location.GetActiveWatchCount() == 1);
watch->Stop();
EXPECT_TRUE(!watch->IsActive());
return true;
}
```
### Test 5: Watch Limit
```cpp
bool Test_LocationWatchLimit(std::string& error_msg) {
mosis::LocationInterface location("test.app");
location.SetCoarsePermission(true);
std::vector<std::shared_ptr<mosis::LocationWatch>> watches;
std::string err;
// Create MAX_WATCHES (5) watches
for (int i = 0; i < 5; i++) {
auto watch = location.WatchPosition({}, err);
EXPECT_TRUE(watch != nullptr);
watches.push_back(watch);
}
// 6th should fail
auto extra = location.WatchPosition({}, err);
EXPECT_TRUE(extra == nullptr);
EXPECT_TRUE(err.find("limit") != std::string::npos ||
err.find("maximum") != std::string::npos);
return true;
}
```
### Test 6: Cleanup on Shutdown
```cpp
bool Test_LocationCleansUpOnShutdown(std::string& error_msg) {
mosis::LocationInterface location("test.app");
location.SetCoarsePermission(true);
std::string err;
auto watch1 = location.WatchPosition({}, err);
auto watch2 = location.WatchPosition({}, err);
EXPECT_TRUE(location.GetActiveWatchCount() == 2);
location.Shutdown();
EXPECT_TRUE(location.GetActiveWatchCount() == 0);
EXPECT_TRUE(!watch1->IsActive());
EXPECT_TRUE(!watch2->IsActive());
return true;
}
```
### Test 7: Lua Integration
```cpp
bool Test_LocationLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::LocationInterface location("test.app");
mosis::RegisterLocationAPI(sandbox.GetState(), &location);
std::string script = R"lua(
-- Test that location global exists
if not location then
error("location global not found")
end
if not location.getCurrentPosition then
error("location.getCurrentPosition not found")
end
if not location.watchPosition then
error("location.watchPosition not found")
end
if not location.clearWatch then
error("location.clearWatch not found")
end
if not location.getWatchCount then
error("location.getWatchCount not found")
end
-- Watch count should be 0 initially
if location.getWatchCount() ~= 0 then
error("should have no active watches initially")
end
)lua";
bool ok = sandbox.LoadString(script, "location_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_LocationRequiresPermission` - Permission required
- [x] `Test_LocationCoarseReducesPrecision` - Coarse reduces precision
- [x] `Test_LocationRateLimits` - Rate limiting enforced
- [x] `Test_LocationWatchPosition` - Watch mode works
- [x] `Test_LocationWatchLimit` - Watch limit enforced
- [x] `Test_LocationCleansUpOnShutdown` - Cleanup on shutdown
- [x] `Test_LocationLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate)
- Milestone 3 (RateLimiter)
---
## Notes
### Desktop vs Android Implementation
For desktop testing, LocationInterface operates in mock mode:
- Positions are set programmatically via `SetMockPosition()`
- Watches track state but don't receive real updates
- Rate limiting is enforced normally
On Android, the real implementation would:
1. Use FusedLocationProviderClient via JNI
2. Request appropriate permissions at runtime
3. Handle GPS/network provider selection
4. Battery-efficient location batching
### Security Considerations
1. **Permission required**: Either coarse or fine permission needed
2. **Precision enforcement**: Coarse permission reduces accuracy to ~1km
3. **Rate limiting**: 1 request per second prevents tracking abuse
4. **Watch limit**: Max 5 watches per app prevents resource exhaustion
5. **Cleanup**: All watches stopped when app stops
### Privacy Features
1. **Coarse mode**: City-level accuracy (~1km) for apps that don't need precision
2. **No background tracking**: Location only available while app is active
3. **Audit logging**: All location requests logged for security review
---
## Next Steps
After Milestone 14 passes:
1. Milestone 15: Virtual Hardware - Sensors

501
docs/SANDBOX_MILESTONE_15.md Normal file
View File

@@ -0,0 +1,501 @@
# Milestone 15: Virtual Hardware - Sensors
**Status**: Complete
**Goal**: Motion sensors with fingerprinting prevention.
---
## Overview
This milestone implements secure sensor access for Lua apps:
- Motion sensors (accelerometer, gyroscope, magnetometer) require `sensors.motion` permission
- Environmental sensors (proximity, ambient light) are auto-granted
- Precision reduction to prevent device fingerprinting
- Frequency limiting to 60 Hz maximum
- Subscription-based API for continuous updates
- Automatic cleanup on app stop
### Key Deliverables
1. **SensorInterface class** - Sensor subscription management
2. **SensorSubscription class** - Individual sensor stream
3. **Lua sensors API** - `sensors.subscribe()`, subscription methods
4. **Fingerprinting prevention** - Precision reduction and frequency limits
---
## File Structure
```
src/main/cpp/sandbox/
├── sensor_interface.h # NEW - Sensor API header
└── sensor_interface.cpp # NEW - Sensor implementation
```
---
## Implementation Details
### 1. SensorInterface Class
```cpp
// sensor_interface.h
#pragma once
#include <string>
#include <memory>
#include <functional>
#include <mutex>
#include <atomic>
#include <vector>
#include <unordered_map>
#include <chrono>
struct lua_State;
namespace mosis {
enum class SensorType {
Accelerometer, // Requires sensors.motion
Gyroscope, // Requires sensors.motion
Magnetometer, // Requires sensors.motion
Proximity, // Auto-granted
AmbientLight // Auto-granted
};
struct SensorData {
double x = 0.0; // Primary axis / value
double y = 0.0; // Secondary axis (if applicable)
double z = 0.0; // Tertiary axis (if applicable)
int64_t timestamp = 0; // Milliseconds since epoch
};
struct SensorOptions {
int frequency = 60; // Hz, capped at 60
};
class SensorSubscription {
public:
using DataCallback = std::function<void(const SensorData&)>;
SensorSubscription(int id, SensorType type, const SensorOptions& options);
~SensorSubscription();
int GetId() const { return m_id; }
SensorType GetType() const { return m_type; }
bool IsActive() const { return m_active; }
int GetFrequency() const { return m_frequency; }
void Start();
void Stop();
void SetOnData(DataCallback cb) { m_on_data = std::move(cb); }
// For mock mode - simulate sensor data
void SimulateData(const SensorData& data);
private:
int m_id;
SensorType m_type;
int m_frequency;
std::atomic<bool> m_active{false};
DataCallback m_on_data;
mutable std::mutex m_mutex;
};
class SensorInterface {
public:
SensorInterface(const std::string& app_id);
~SensorInterface();
// Check if app has motion sensor permission
bool HasMotionPermission() const { return m_has_motion_permission; }
void SetMotionPermission(bool granted) { m_has_motion_permission = granted; }
// Subscribe to a sensor
// Returns subscription on success, nullptr on failure
std::shared_ptr<SensorSubscription> Subscribe(
SensorType type,
const SensorOptions& options,
std::string& error
);
// Subscribe by sensor name
std::shared_ptr<SensorSubscription> Subscribe(
const std::string& sensor_name,
const SensorOptions& options,
std::string& error
);
// Stop a specific subscription
void Unsubscribe(int subscription_id);
// Stop all subscriptions for this app
void UnsubscribeAll();
// Get active subscription count
size_t GetActiveSubscriptionCount() const;
// Cleanup on app stop
void Shutdown();
// For testing
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
bool IsMockMode() const { return m_mock_mode; }
// Get subscription by ID
std::shared_ptr<SensorSubscription> GetSubscription(int id);
// Check if sensor type requires permission
static bool RequiresPermission(SensorType type);
// Parse sensor name to type
static bool ParseSensorType(const std::string& name, SensorType& out_type);
private:
std::string m_app_id;
std::unordered_map<int, std::shared_ptr<SensorSubscription>> m_subscriptions;
mutable std::mutex m_mutex;
bool m_mock_mode = true;
bool m_has_motion_permission = false;
int m_next_subscription_id = 1;
static constexpr int MAX_SUBSCRIPTIONS = 10;
static constexpr int MAX_FREQUENCY = 60; // Hz
// Apply precision reduction to prevent fingerprinting
SensorData ApplyPrecision(const SensorData& data) const;
void CleanupStoppedSubscriptions();
};
// Register sensors.* APIs as globals
void RegisterSensorAPI(lua_State* L, SensorInterface* sensors);
} // namespace mosis
```
### 2. Sensor Types and Permissions
| Sensor | Permission Required | Data Fields |
|--------|---------------------|-------------|
| Accelerometer | `sensors.motion` | x, y, z (m/s²) |
| Gyroscope | `sensors.motion` | x, y, z (rad/s) |
| Magnetometer | `sensors.motion` | x, y, z (µT) |
| Proximity | None (auto-granted) | x (distance in cm, or 0/1 for near/far) |
| Ambient Light | None (auto-granted) | x (lux) |
### 3. Fingerprinting Prevention
Values are rounded to 2 decimal places to prevent device fingerprinting:
```cpp
SensorData SensorInterface::ApplyPrecision(const SensorData& data) const {
SensorData result = data;
result.x = std::round(data.x * 100.0) / 100.0;
result.y = std::round(data.y * 100.0) / 100.0;
result.z = std::round(data.z * 100.0) / 100.0;
return result;
}
```
### 4. Frequency Limiting
Maximum update frequency is capped at 60 Hz:
- Prevents high-frequency sensor abuse
- Reduces battery impact
- Sufficient for most app use cases
### 5. Lua API
```lua
-- Subscribe to accelerometer (requires sensors.motion permission)
local sub = sensors.subscribe("accelerometer", function(data)
print("Accel:", data.x, data.y, data.z)
print("Time:", data.timestamp)
end, {
frequency = 60 -- Hz, max 60
})
-- Check if subscription is active
if sub:isActive() then
print("Subscription ID:", sub:getId())
end
-- Stop subscription
sub:stop()
-- Subscribe to proximity (auto-granted, no permission needed)
local proxSub = sensors.subscribe("proximity", function(data)
if data.x < 5 then
print("Object is near")
end
end)
-- Get active subscription count
local count = sensors.getSubscriptionCount()
-- Unsubscribe all
sensors.unsubscribeAll()
```
### 6. Sensor Names
```lua
-- Valid sensor names
"accelerometer" -- Accelerometer (requires permission)
"gyroscope" -- Gyroscope (requires permission)
"magnetometer" -- Magnetometer (requires permission)
"proximity" -- Proximity sensor (auto-granted)
"light" -- Ambient light sensor (auto-granted)
```
---
## Test Cases
### Test 1: Requires Permission for Motion Sensors
```cpp
bool Test_SensorRequiresPermission(std::string& error_msg) {
mosis::SensorInterface sensors("test.app");
// No permissions granted
std::string err;
auto sub = sensors.Subscribe("accelerometer", {}, err);
EXPECT_TRUE(sub == nullptr);
EXPECT_TRUE(err.find("permission") != std::string::npos);
return true;
}
```
### Test 2: Auto-Grants Environmental Sensors
```cpp
bool Test_SensorAutoGrantsEnvironmental(std::string& error_msg) {
mosis::SensorInterface sensors("test.app");
// No motion permission
std::string err;
// Proximity should work without permission
auto proxSub = sensors.Subscribe("proximity", {}, err);
EXPECT_TRUE(proxSub != nullptr);
EXPECT_TRUE(proxSub->IsActive());
// Light should work without permission
auto lightSub = sensors.Subscribe("light", {}, err);
EXPECT_TRUE(lightSub != nullptr);
EXPECT_TRUE(lightSub->IsActive());
return true;
}
```
### Test 3: Reduces Precision
```cpp
bool Test_SensorReducesPrecision(std::string& error_msg) {
mosis::SensorInterface sensors("test.app");
sensors.SetMotionPermission(true);
std::string err;
auto sub = sensors.Subscribe("accelerometer", {}, err);
EXPECT_TRUE(sub != nullptr);
mosis::SensorData received;
bool got_data = false;
sub->SetOnData([&](const mosis::SensorData& data) {
got_data = true;
received = data;
});
// Simulate high-precision data
mosis::SensorData precise;
precise.x = 9.80665123456;
precise.y = 0.12345678901;
precise.z = -0.98765432109;
sub->SimulateData(precise);
EXPECT_TRUE(got_data);
// Values should be rounded to 2 decimal places
EXPECT_TRUE(std::abs(received.x - 9.81) < 0.001);
EXPECT_TRUE(std::abs(received.y - 0.12) < 0.001);
EXPECT_TRUE(std::abs(received.z - (-0.99)) < 0.001);
return true;
}
```
### Test 4: Limits Frequency
```cpp
bool Test_SensorLimitsFrequency(std::string& error_msg) {
mosis::SensorInterface sensors("test.app");
sensors.SetMotionPermission(true);
mosis::SensorOptions options;
options.frequency = 120; // Request 120 Hz
std::string err;
auto sub = sensors.Subscribe("gyroscope", options, err);
EXPECT_TRUE(sub != nullptr);
// Should be capped at 60 Hz
EXPECT_TRUE(sub->GetFrequency() <= 60);
return true;
}
```
### Test 5: Subscription Limit
```cpp
bool Test_SensorSubscriptionLimit(std::string& error_msg) {
mosis::SensorInterface sensors("test.app");
sensors.SetMotionPermission(true);
std::vector<std::shared_ptr<mosis::SensorSubscription>> subs;
std::string err;
// Create MAX_SUBSCRIPTIONS (10)
for (int i = 0; i < 10; i++) {
auto sub = sensors.Subscribe("accelerometer", {}, err);
EXPECT_TRUE(sub != nullptr);
subs.push_back(sub);
}
// 11th should fail
auto extra = sensors.Subscribe("accelerometer", {}, err);
EXPECT_TRUE(extra == nullptr);
EXPECT_TRUE(err.find("limit") != std::string::npos ||
err.find("maximum") != std::string::npos);
return true;
}
```
### Test 6: Cleanup on Shutdown
```cpp
bool Test_SensorCleansUpOnShutdown(std::string& error_msg) {
mosis::SensorInterface sensors("test.app");
sensors.SetMotionPermission(true);
std::string err;
auto sub1 = sensors.Subscribe("accelerometer", {}, err);
auto sub2 = sensors.Subscribe("gyroscope", {}, err);
EXPECT_TRUE(sensors.GetActiveSubscriptionCount() == 2);
sensors.Shutdown();
EXPECT_TRUE(sensors.GetActiveSubscriptionCount() == 0);
EXPECT_TRUE(!sub1->IsActive());
EXPECT_TRUE(!sub2->IsActive());
return true;
}
```
### Test 7: Lua Integration
```cpp
bool Test_SensorLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::SensorInterface sensors("test.app");
mosis::RegisterSensorAPI(sandbox.GetState(), &sensors);
std::string script = R"lua(
-- Test that sensors global exists
if not sensors then
error("sensors global not found")
end
if not sensors.subscribe then
error("sensors.subscribe not found")
end
if not sensors.unsubscribeAll then
error("sensors.unsubscribeAll not found")
end
if not sensors.getSubscriptionCount then
error("sensors.getSubscriptionCount not found")
end
-- Subscription count should be 0 initially
if sensors.getSubscriptionCount() ~= 0 then
error("should have no active subscriptions initially")
end
)lua";
bool ok = sandbox.LoadString(script, "sensor_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_SensorRequiresPermission` - Motion sensors require permission
- [x] `Test_SensorAutoGrantsEnvironmental` - Proximity/light auto-granted
- [x] `Test_SensorReducesPrecision` - Values rounded to 2 decimals
- [x] `Test_SensorLimitsFrequency` - Frequency capped at 60 Hz
- [x] `Test_SensorSubscriptionLimit` - Max 10 subscriptions enforced
- [x] `Test_SensorCleansUpOnShutdown` - Cleanup on shutdown
- [x] `Test_SensorLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate)
---
## Notes
### Desktop vs Android Implementation
For desktop testing, SensorInterface operates in mock mode:
- Subscriptions track state but don't receive real sensor data
- Data can be simulated via `SimulateData()` for testing
- Precision reduction and frequency limits are enforced normally
On Android, the real implementation would:
1. Use SensorManager via JNI
2. Register sensor event listeners
3. Apply precision reduction before callbacks
4. Enforce frequency via sensor delay settings
### Security Considerations
1. **Permission required**: Motion sensors need `sensors.motion` permission
2. **Precision reduction**: Values rounded to 2 decimals to prevent fingerprinting
3. **Frequency limit**: Maximum 60 Hz prevents high-frequency abuse
4. **Subscription limit**: Max 10 subscriptions per app
5. **Cleanup**: All subscriptions stopped when app stops
### Privacy Features
1. **Reduced precision**: Prevents unique device identification via sensor calibration
2. **No raw sensor access**: Apps only get processed, reduced-precision data
3. **Audit logging**: All sensor subscriptions logged for security review
---
## Next Steps
After Milestone 15 passes:
1. Milestone 16: Virtual Hardware - Bluetooth

566
docs/SANDBOX_MILESTONE_16.md Normal file
View File

@@ -0,0 +1,566 @@
# Milestone 16: Virtual Hardware - Bluetooth
**Status**: Complete
**Goal**: Bluetooth discovery and pairing with user consent.
---
## Overview
This milestone implements secure Bluetooth access for Lua apps:
- Discovery requires user consent (user gesture)
- Permission required: `bluetooth` for discovery and connection
- Device info limited to name and address (no UUIDs)
- Connection limit: 5 active connections per app
- Automatic cleanup on app stop
### Key Deliverables
1. **BluetoothInterface class** - Bluetooth discovery and connection management
2. **BluetoothConnection class** - Individual connection handling
3. **Lua bluetooth API** - `bluetooth.startDiscovery()`, `bluetooth.connect()`
4. **Privacy protection** - Limited device information, user consent required
---
## File Structure
```
src/main/cpp/sandbox/
├── bluetooth_interface.h # NEW - Bluetooth API header
└── bluetooth_interface.cpp # NEW - Bluetooth implementation
```
---
## Implementation Details
### 1. BluetoothInterface Class
```cpp
// bluetooth_interface.h
#pragma once
#include <string>
#include <memory>
#include <functional>
#include <mutex>
#include <atomic>
#include <vector>
#include <unordered_map>
#include <chrono>
struct lua_State;
namespace mosis {
struct BluetoothDevice {
std::string name; // Device display name
std::string address; // MAC address
};
struct DiscoveryOptions {
int timeout_seconds = 30; // Discovery timeout
};
struct ConnectionOptions {
int timeout_ms = 10000; // Connection timeout
};
enum class BluetoothError {
None,
PermissionDenied,
UserConsentRequired,
DiscoveryFailed,
ConnectionFailed,
NotConnected,
ConnectionLimitReached,
Timeout,
AlreadyDiscovering
};
class BluetoothConnection {
public:
using DataCallback = std::function<void(const std::vector<uint8_t>&)>;
using ErrorCallback = std::function<void(BluetoothError, const std::string&)>;
BluetoothConnection(int id, const std::string& address);
~BluetoothConnection();
int GetId() const { return m_id; }
std::string GetAddress() const { return m_address; }
bool IsConnected() const { return m_connected; }
bool Send(const std::vector<uint8_t>& data, std::string& error);
void Close();
void SetOnData(DataCallback cb) { m_on_data = std::move(cb); }
void SetOnError(ErrorCallback cb) { m_on_error = std::move(cb); }
void SetOnDisconnect(std::function<void()> cb) { m_on_disconnect = std::move(cb); }
// For mock mode - simulate data receive
void SimulateData(const std::vector<uint8_t>& data);
void SimulateDisconnect();
private:
int m_id;
std::string m_address;
std::atomic<bool> m_connected{false};
DataCallback m_on_data;
ErrorCallback m_on_error;
std::function<void()> m_on_disconnect;
mutable std::mutex m_mutex;
};
class BluetoothInterface {
public:
BluetoothInterface(const std::string& app_id);
~BluetoothInterface();
// Permission checks
bool HasBluetoothPermission() const { return m_has_bluetooth_permission; }
void SetBluetoothPermission(bool granted) { m_has_bluetooth_permission = granted; }
// User gesture tracking
bool HasUserConsent() const { return m_has_user_consent; }
void SetUserConsent(bool consent) { m_has_user_consent = consent; }
// Start device discovery
// Requires permission AND user consent
bool StartDiscovery(
const DiscoveryOptions& options,
std::function<void(const std::vector<BluetoothDevice>&)> success,
std::function<void(BluetoothError, const std::string&)> error,
std::string& out_error
);
// Stop current discovery
void StopDiscovery();
// Check if discovery is in progress
bool IsDiscovering() const { return m_is_discovering; }
// Connect to a device
std::shared_ptr<BluetoothConnection> Connect(
const std::string& address,
const ConnectionOptions& options,
std::string& error
);
// Disconnect a specific connection
void Disconnect(int connection_id);
// Disconnect all connections
void DisconnectAll();
// Get active connection count
size_t GetActiveConnectionCount() const;
// Cleanup on app stop
void Shutdown();
// For testing
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
bool IsMockMode() const { return m_mock_mode; }
// Mock mode: set discovered devices
void SetMockDevices(const std::vector<BluetoothDevice>& devices) { m_mock_devices = devices; }
// Get connection by ID
std::shared_ptr<BluetoothConnection> GetConnection(int id);
private:
std::string m_app_id;
std::unordered_map<int, std::shared_ptr<BluetoothConnection>> m_connections;
mutable std::mutex m_mutex;
bool m_mock_mode = true;
bool m_has_bluetooth_permission = false;
bool m_has_user_consent = false;
std::atomic<bool> m_is_discovering{false};
std::vector<BluetoothDevice> m_mock_devices;
int m_next_connection_id = 1;
static constexpr int MAX_CONNECTIONS = 5;
void CleanupClosedConnections();
};
// Register bluetooth.* APIs as globals
void RegisterBluetoothAPI(lua_State* L, BluetoothInterface* bluetooth);
} // namespace mosis
```
### 2. Permission and Consent Requirements
| Action | Permission Required | User Consent Required |
|--------|--------------------|-----------------------|
| Start Discovery | `bluetooth` | Yes (user gesture) |
| Connect to Device | `bluetooth` | No (implicit from discovery) |
| Send Data | `bluetooth` | No (connection already established) |
| Receive Data | `bluetooth` | No (connection already established) |
### 3. Privacy Protection
Device information is limited to:
- Device name (display name)
- Device address (MAC)
NOT exposed:
- Full UUID list (prevents service fingerprinting)
- Device class
- Manufacturer data
- RSSI (prevents proximity tracking)
### 4. Lua API
```lua
-- Start device discovery (requires user gesture)
bluetooth.startDiscovery(function(devices)
for _, device in ipairs(devices) do
print("Found:", device.name, device.address)
end
end, function(error, message)
print("Discovery error:", error, message)
end, {
timeout = 30 -- seconds
})
-- Stop discovery
bluetooth.stopDiscovery()
-- Check if discovering
local discovering = bluetooth.isDiscovering()
-- Connect to a device
local conn = bluetooth.connect(address, function(data)
print("Received:", #data, "bytes")
end, function(error, message)
print("Connection error:", error, message)
end, {
timeout = 10000 -- ms
})
-- Check connection
if conn and conn:isConnected() then
-- Send data
conn:send({0x01, 0x02, 0x03})
-- Get connection info
print("Connected to:", conn:getAddress())
print("Connection ID:", conn:getId())
end
-- Close connection
conn:close()
-- Get active connection count
local count = bluetooth.getConnectionCount()
-- Disconnect all
bluetooth.disconnectAll()
```
### 5. Error Codes
```lua
-- Error values passed to error callbacks
"PERMISSION_DENIED" -- No bluetooth permission
"USER_CONSENT_REQUIRED" -- User gesture required for discovery
"DISCOVERY_FAILED" -- Discovery could not start
"CONNECTION_FAILED" -- Could not connect to device
"NOT_CONNECTED" -- Connection not established
"CONNECTION_LIMIT" -- Max connections reached (5)
"TIMEOUT" -- Operation timed out
"ALREADY_DISCOVERING" -- Discovery already in progress
```
---
## Test Cases
### Test 1: Requires Permission
```cpp
bool Test_BluetoothRequiresPermission(std::string& error_msg) {
mosis::BluetoothInterface bluetooth("test.app");
// No permission granted
std::string err;
bool started = bluetooth.StartDiscovery(
{},
[](const std::vector<mosis::BluetoothDevice>&) {},
[](mosis::BluetoothError, const std::string&) {},
err
);
EXPECT_TRUE(!started);
EXPECT_TRUE(err.find("permission") != std::string::npos);
return true;
}
```
### Test 2: Requires User Consent
```cpp
bool Test_BluetoothRequiresUserConsent(std::string& error_msg) {
mosis::BluetoothInterface bluetooth("test.app");
bluetooth.SetBluetoothPermission(true);
// User consent NOT granted
std::string err;
bool started = bluetooth.StartDiscovery(
{},
[](const std::vector<mosis::BluetoothDevice>&) {},
[](mosis::BluetoothError, const std::string&) {},
err
);
EXPECT_TRUE(!started);
EXPECT_TRUE(err.find("consent") != std::string::npos ||
err.find("gesture") != std::string::npos);
return true;
}
```
### Test 3: Discovery Works With Permission and Consent
```cpp
bool Test_BluetoothDiscoveryWorks(std::string& error_msg) {
mosis::BluetoothInterface bluetooth("test.app");
bluetooth.SetBluetoothPermission(true);
bluetooth.SetUserConsent(true);
// Set mock devices
std::vector<mosis::BluetoothDevice> devices = {
{"Device A", "AA:BB:CC:DD:EE:FF"},
{"Device B", "11:22:33:44:55:66"}
};
bluetooth.SetMockDevices(devices);
std::vector<mosis::BluetoothDevice> discovered;
bool got_devices = false;
std::string err;
bool started = bluetooth.StartDiscovery(
{},
[&](const std::vector<mosis::BluetoothDevice>& devs) {
got_devices = true;
discovered = devs;
},
[](mosis::BluetoothError, const std::string&) {},
err
);
EXPECT_TRUE(started);
EXPECT_TRUE(got_devices);
EXPECT_TRUE(discovered.size() == 2);
EXPECT_TRUE(discovered[0].name == "Device A");
EXPECT_TRUE(discovered[0].address == "AA:BB:CC:DD:EE:FF");
return true;
}
```
### Test 4: Connection Limit
```cpp
bool Test_BluetoothConnectionLimit(std::string& error_msg) {
mosis::BluetoothInterface bluetooth("test.app");
bluetooth.SetBluetoothPermission(true);
std::vector<std::shared_ptr<mosis::BluetoothConnection>> connections;
std::string err;
// Create MAX_CONNECTIONS (5)
for (int i = 0; i < 5; i++) {
char addr[20];
snprintf(addr, sizeof(addr), "AA:BB:CC:DD:EE:%02X", i);
auto conn = bluetooth.Connect(addr, {}, err);
EXPECT_TRUE(conn != nullptr);
connections.push_back(conn);
}
// 6th should fail
auto extra = bluetooth.Connect("AA:BB:CC:DD:EE:FF", {}, err);
EXPECT_TRUE(extra == nullptr);
EXPECT_TRUE(err.find("limit") != std::string::npos);
return true;
}
```
### Test 5: Connection Send/Receive
```cpp
bool Test_BluetoothSendReceive(std::string& error_msg) {
mosis::BluetoothInterface bluetooth("test.app");
bluetooth.SetBluetoothPermission(true);
std::string err;
auto conn = bluetooth.Connect("AA:BB:CC:DD:EE:FF", {}, err);
EXPECT_TRUE(conn != nullptr);
EXPECT_TRUE(conn->IsConnected());
// Test send
std::vector<uint8_t> data = {0x01, 0x02, 0x03};
bool sent = conn->Send(data, err);
EXPECT_TRUE(sent);
// Test receive via simulation
std::vector<uint8_t> received;
bool got_data = false;
conn->SetOnData([&](const std::vector<uint8_t>& d) {
got_data = true;
received = d;
});
std::vector<uint8_t> incoming = {0xAA, 0xBB};
conn->SimulateData(incoming);
EXPECT_TRUE(got_data);
EXPECT_TRUE(received.size() == 2);
EXPECT_TRUE(received[0] == 0xAA);
return true;
}
```
### Test 6: Cleanup on Shutdown
```cpp
bool Test_BluetoothCleansUpOnShutdown(std::string& error_msg) {
mosis::BluetoothInterface bluetooth("test.app");
bluetooth.SetBluetoothPermission(true);
std::string err;
auto conn1 = bluetooth.Connect("AA:BB:CC:DD:EE:01", {}, err);
auto conn2 = bluetooth.Connect("AA:BB:CC:DD:EE:02", {}, err);
EXPECT_TRUE(bluetooth.GetActiveConnectionCount() == 2);
bluetooth.Shutdown();
EXPECT_TRUE(bluetooth.GetActiveConnectionCount() == 0);
EXPECT_TRUE(!conn1->IsConnected());
EXPECT_TRUE(!conn2->IsConnected());
return true;
}
```
### Test 7: Lua Integration
```cpp
bool Test_BluetoothLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::BluetoothInterface bluetooth("test.app");
mosis::RegisterBluetoothAPI(sandbox.GetState(), &bluetooth);
std::string script = R"lua(
-- Test that bluetooth global exists
if not bluetooth then
error("bluetooth global not found")
end
if not bluetooth.startDiscovery then
error("bluetooth.startDiscovery not found")
end
if not bluetooth.stopDiscovery then
error("bluetooth.stopDiscovery not found")
end
if not bluetooth.connect then
error("bluetooth.connect not found")
end
if not bluetooth.disconnectAll then
error("bluetooth.disconnectAll not found")
end
if not bluetooth.getConnectionCount then
error("bluetooth.getConnectionCount not found")
end
if not bluetooth.isDiscovering then
error("bluetooth.isDiscovering not found")
end
-- Connection count should be 0 initially
if bluetooth.getConnectionCount() ~= 0 then
error("should have no active connections initially")
end
-- Should not be discovering initially
if bluetooth.isDiscovering() then
error("should not be discovering initially")
end
)lua";
bool ok = sandbox.LoadString(script, "bluetooth_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_BluetoothRequiresPermission` - Permission required
- [x] `Test_BluetoothRequiresUserConsent` - User consent required for discovery
- [x] `Test_BluetoothDiscoveryWorks` - Discovery returns devices
- [x] `Test_BluetoothConnectionLimit` - Max 5 connections enforced
- [x] `Test_BluetoothSendReceive` - Send/receive data works
- [x] `Test_BluetoothCleansUpOnShutdown` - Cleanup on shutdown
- [x] `Test_BluetoothLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate + user gesture tracking)
---
## Notes
### Desktop vs Android Implementation
For desktop testing, BluetoothInterface operates in mock mode:
- Discovery returns mock device list
- Connections track state but don't communicate
- Send/receive can be simulated for testing
On Android, the real implementation would:
1. Use BluetoothAdapter via JNI
2. Handle BLUETOOTH_SCAN and BLUETOOTH_CONNECT permissions
3. Use BluetoothSocket for RFCOMM connections
4. Support BLE via BluetoothGatt
### Security Considerations
1. **Permission required**: `bluetooth` permission needed for all operations
2. **User consent**: Discovery requires active user gesture
3. **Limited device info**: Only name and address exposed (no UUIDs)
4. **Connection limit**: Max 5 connections per app
5. **Cleanup**: All connections closed when app stops
### Privacy Features
1. **No service UUIDs**: Prevents detailed device fingerprinting
2. **No RSSI**: Prevents proximity tracking
3. **User consent**: Discovery requires explicit user action
4. **Audit logging**: All Bluetooth operations logged
---
## Next Steps
After Milestone 16 passes:
1. Milestone 17: Virtual Hardware - Contacts

472
docs/SANDBOX_MILESTONE_17.md Normal file
View File

@@ -0,0 +1,472 @@
# Milestone 17: Virtual Hardware - Contacts
**Status**: Complete
**Goal**: Contact access with granular read/write permissions.
---
## Overview
This milestone implements secure Contacts access for Lua apps:
- Separate permissions: `contacts.read` and `contacts.write`
- Limited contact fields: name, phone, email (no raw account data)
- Query with search/pagination support
- Mock mode for desktop testing
- Maximum 100 results per query
### Key Deliverables
1. **ContactsInterface class** - Contact query and management
2. **Contact struct** - Limited contact fields (privacy-preserving)
3. **Lua contacts API** - `contacts.query()`, `contacts.add()`, `contacts.update()`, `contacts.delete()`
4. **Privacy protection** - Limited fields, permission enforcement
---
## File Structure
```
src/main/cpp/sandbox/
├── contacts_interface.h # NEW - Contacts API header
└── contacts_interface.cpp # NEW - Contacts implementation
```
---
## Implementation Details
### 1. ContactsInterface Class
```cpp
// contacts_interface.h
#pragma once
#include <string>
#include <vector>
#include <optional>
#include <functional>
#include <mutex>
#include <unordered_map>
struct lua_State;
namespace mosis {
struct Contact {
int id = 0; // Internal ID
std::string name; // Display name
std::string phone; // Primary phone number
std::string email; // Primary email
};
struct QueryOptions {
std::string search; // Search string (matches name, phone, email)
int limit = 50; // Max results (capped at 100)
int offset = 0; // Pagination offset
};
enum class ContactsError {
None,
PermissionDenied,
NotFound,
InvalidData,
LimitExceeded
};
class ContactsInterface {
public:
ContactsInterface(const std::string& app_id);
~ContactsInterface();
// Permission checks
bool HasReadPermission() const { return m_has_read_permission; }
void SetReadPermission(bool granted) { m_has_read_permission = granted; }
bool HasWritePermission() const { return m_has_write_permission; }
void SetWritePermission(bool granted) { m_has_write_permission = granted; }
// Query contacts (requires read permission)
std::vector<Contact> Query(const QueryOptions& options, std::string& error);
// Get single contact by ID (requires read permission)
std::optional<Contact> GetById(int id, std::string& error);
// Add contact (requires write permission)
int Add(const Contact& contact, std::string& error);
// Update contact (requires write permission)
bool Update(const Contact& contact, std::string& error);
// Delete contact (requires write permission)
bool Delete(int id, std::string& error);
// Get total contact count (requires read permission)
int GetCount(std::string& error);
// For testing
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
bool IsMockMode() const { return m_mock_mode; }
// Mock mode: set contacts directly
void SetMockContacts(const std::vector<Contact>& contacts);
// Clear all mock contacts
void ClearMockContacts();
private:
std::string m_app_id;
bool m_mock_mode = true;
bool m_has_read_permission = false;
bool m_has_write_permission = false;
std::vector<Contact> m_mock_contacts;
int m_next_id = 1;
mutable std::mutex m_mutex;
static constexpr int MAX_RESULTS = 100;
bool MatchesSearch(const Contact& contact, const std::string& search) const;
};
// Register contacts.* APIs as globals
void RegisterContactsAPI(lua_State* L, ContactsInterface* contacts);
} // namespace mosis
```
### 2. Permission Requirements
| Action | Permission Required |
|--------|---------------------|
| Query contacts | `contacts.read` |
| Get contact by ID | `contacts.read` |
| Get contact count | `contacts.read` |
| Add contact | `contacts.write` |
| Update contact | `contacts.write` |
| Delete contact | `contacts.write` |
### 3. Privacy Protection
Contact fields exposed:
- `id` - Internal identifier
- `name` - Display name
- `phone` - Primary phone number
- `email` - Primary email address
NOT exposed:
- Account type/name (Google, Exchange, etc.)
- Contact photo
- Multiple phone numbers/emails
- Address, notes, organization
- Social profiles
- Custom fields
- Sync metadata
### 4. Lua API
```lua
-- Query contacts (requires contacts.read)
local results = contacts.query({
search = "John", -- optional: search name/phone/email
limit = 20, -- optional: max results (default 50, max 100)
offset = 0 -- optional: pagination offset
})
for _, c in ipairs(results) do
print(c.id, c.name, c.phone, c.email)
end
-- Get by ID (requires contacts.read)
local contact = contacts.getById(123)
if contact then
print(contact.name)
end
-- Get total count (requires contacts.read)
local count = contacts.getCount()
-- Add contact (requires contacts.write)
local newId = contacts.add({
name = "John Doe",
phone = "555-1234",
email = "john@example.com"
})
-- Update contact (requires contacts.write)
local success = contacts.update({
id = 123,
name = "John Smith",
phone = "555-5678",
email = "john.smith@example.com"
})
-- Delete contact (requires contacts.write)
local success = contacts.delete(123)
```
### 5. Error Handling
```lua
-- Errors are raised via Lua errors
-- Use pcall for safe error handling
local ok, result = pcall(function()
return contacts.query({search = "test"})
end)
if not ok then
print("Error:", result) -- e.g., "permission denied: contacts.read required"
end
```
---
## Test Cases
### Test 1: Query Requires Read Permission
```cpp
bool Test_ContactsRequiresReadPermission(std::string& error_msg) {
mosis::ContactsInterface contacts("test.app");
// No permission granted
std::string err;
auto results = contacts.Query({}, err);
EXPECT_TRUE(results.empty());
EXPECT_TRUE(err.find("permission") != std::string::npos);
return true;
}
```
### Test 2: Add Requires Write Permission
```cpp
bool Test_ContactsRequiresWritePermission(std::string& error_msg) {
mosis::ContactsInterface contacts("test.app");
contacts.SetReadPermission(true); // Read OK, but not write
std::string err;
mosis::Contact contact{"John Doe", "555-1234", "john@test.com"};
int id = contacts.Add(contact, err);
EXPECT_TRUE(id == 0); // Failed
EXPECT_TRUE(err.find("permission") != std::string::npos);
return true;
}
```
### Test 3: Query Returns Matching Contacts
```cpp
bool Test_ContactsQueryWorks(std::string& error_msg) {
mosis::ContactsInterface contacts("test.app");
contacts.SetReadPermission(true);
// Set mock contacts
std::vector<mosis::Contact> mockContacts = {
{1, "John Doe", "555-1234", "john@test.com"},
{2, "Jane Doe", "555-5678", "jane@test.com"},
{3, "Bob Smith", "555-9999", "bob@test.com"}
};
contacts.SetMockContacts(mockContacts);
std::string err;
auto results = contacts.Query({.search = "Doe"}, err);
EXPECT_TRUE(err.empty());
EXPECT_TRUE(results.size() == 2); // John and Jane
EXPECT_TRUE(results[0].name == "John Doe" || results[0].name == "Jane Doe");
return true;
}
```
### Test 4: Query Respects Limit
```cpp
bool Test_ContactsQueryLimit(std::string& error_msg) {
mosis::ContactsInterface contacts("test.app");
contacts.SetReadPermission(true);
// Set 50 mock contacts
std::vector<mosis::Contact> mockContacts;
for (int i = 0; i < 50; i++) {
mockContacts.push_back({i + 1, "Contact " + std::to_string(i), "555-" + std::to_string(i), ""});
}
contacts.SetMockContacts(mockContacts);
std::string err;
// Query with limit
auto results = contacts.Query({.limit = 10}, err);
EXPECT_TRUE(results.size() == 10);
// Query with excessive limit (capped at MAX_RESULTS)
results = contacts.Query({.limit = 200}, err);
EXPECT_TRUE(results.size() == 50); // All contacts (< MAX_RESULTS)
return true;
}
```
### Test 5: Add/Update/Delete Works
```cpp
bool Test_ContactsCRUD(std::string& error_msg) {
mosis::ContactsInterface contacts("test.app");
contacts.SetReadPermission(true);
contacts.SetWritePermission(true);
std::string err;
// Add
mosis::Contact newContact{0, "Test User", "555-1111", "test@test.com"};
int id = contacts.Add(newContact, err);
EXPECT_TRUE(id > 0);
EXPECT_TRUE(err.empty());
// Read back
auto contact = contacts.GetById(id, err);
EXPECT_TRUE(contact.has_value());
EXPECT_TRUE(contact->name == "Test User");
// Update
contact->name = "Updated User";
contact->phone = "555-2222";
bool updated = contacts.Update(*contact, err);
EXPECT_TRUE(updated);
// Verify update
contact = contacts.GetById(id, err);
EXPECT_TRUE(contact->name == "Updated User");
EXPECT_TRUE(contact->phone == "555-2222");
// Delete
bool deleted = contacts.Delete(id, err);
EXPECT_TRUE(deleted);
// Verify deleted
contact = contacts.GetById(id, err);
EXPECT_TRUE(!contact.has_value());
return true;
}
```
### Test 6: Delete Nonexistent Fails
```cpp
bool Test_ContactsDeleteNotFound(std::string& error_msg) {
mosis::ContactsInterface contacts("test.app");
contacts.SetWritePermission(true);
std::string err;
bool deleted = contacts.Delete(99999, err);
EXPECT_TRUE(!deleted);
EXPECT_TRUE(err.find("not found") != std::string::npos);
return true;
}
```
### Test 7: Lua Integration
```cpp
bool Test_ContactsLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::ContactsInterface contacts("test.app");
mosis::RegisterContactsAPI(sandbox.GetState(), &contacts);
std::string script = R"lua(
-- Test that contacts global exists
if not contacts then
error("contacts global not found")
end
if not contacts.query then
error("contacts.query not found")
end
if not contacts.getById then
error("contacts.getById not found")
end
if not contacts.getCount then
error("contacts.getCount not found")
end
if not contacts.add then
error("contacts.add not found")
end
if not contacts.update then
error("contacts.update not found")
end
if not contacts.delete then
error("contacts.delete not found")
end
)lua";
bool ok = sandbox.LoadString(script, "contacts_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_ContactsRequiresReadPermission` - Read permission required for query
- [x] `Test_ContactsRequiresWritePermission` - Write permission required for add
- [x] `Test_ContactsQueryWorks` - Query returns matching contacts
- [x] `Test_ContactsQueryLimit` - Query respects limit/pagination
- [x] `Test_ContactsCRUD` - Add/update/delete works
- [x] `Test_ContactsDeleteNotFound` - Delete nonexistent contact fails gracefully
- [x] `Test_ContactsLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate)
---
## Notes
### Desktop vs Android Implementation
For desktop testing, ContactsInterface operates in mock mode:
- Stores contacts in memory
- Full CRUD support for testing
- Search matches name, phone, or email
On Android, the real implementation would:
1. Use ContentResolver to query ContactsContract
2. Handle READ_CONTACTS and WRITE_CONTACTS permissions
3. Use proper contact lookup/insert/update/delete URIs
### Security Considerations
1. **Separate permissions**: Read and write are independent
2. **Limited fields**: Only name, phone, email exposed
3. **Query limits**: Maximum 100 results per query
4. **Audit logging**: All contact operations logged
### Privacy Features
1. **No account info**: Account type/name hidden
2. **No photos**: Contact photos not accessible
3. **Single values**: Only primary phone/email (no full list)
4. **No sync metadata**: Internal sync state hidden
---
## Next Steps
After Milestone 17 passes:
1. Milestone 18: Inter-App Communication

463
docs/SANDBOX_MILESTONE_18.md Normal file
View File

@@ -0,0 +1,463 @@
# Milestone 18: Inter-App Communication
**Status**: Complete
**Goal**: Kernel-mediated message passing between apps.
---
## Overview
This milestone implements secure inter-app communication via an intent system:
- Apps register to receive specific intent actions
- Senders must have required permissions
- All messages go through kernel mediation (no direct app-to-app)
- Message size limits prevent resource exhaustion
- Audit logging for all inter-app communication
### Key Deliverables
1. **MessageBus class** - Central message routing and validation
2. **Intent struct** - Message format with action, type, and data
3. **Lua intents API** - `intents.send()`, `intents.on()`, `intents.broadcast()`
4. **Security validation** - Permission checks, sender/receiver verification
---
## File Structure
```
src/main/cpp/sandbox/
├── message_bus.h # NEW - Message bus API header
└── message_bus.cpp # NEW - Message bus implementation
```
---
## Implementation Details
### 1. MessageBus Class
```cpp
// message_bus.h
#pragma once
#include <string>
#include <vector>
#include <functional>
#include <mutex>
#include <unordered_map>
#include <unordered_set>
struct lua_State;
namespace mosis {
struct Intent {
std::string action; // Intent action (e.g., "share", "view", "edit")
std::string type; // MIME type (e.g., "text/plain", "image/png")
std::string data; // Intent data/payload
std::string from_app; // Sender app ID (set by kernel)
};
struct IntentFilter {
std::string action; // Required action
std::vector<std::string> types; // Accepted MIME types (empty = all)
};
enum class MessageError {
None,
NoReceivers,
PermissionDenied,
InvalidIntent,
DataTooLarge,
ReceiverNotFound,
SenderBlocked
};
class MessageBus {
public:
using IntentHandler = std::function<void(const Intent&)>;
MessageBus();
~MessageBus();
// Register an app to receive intents
void RegisterReceiver(
const std::string& app_id,
const IntentFilter& filter,
IntentHandler handler
);
// Unregister all handlers for an app
void UnregisterApp(const std::string& app_id);
// Send intent to specific app (requires permission)
MessageError Send(
const std::string& sender_app,
const std::string& target_app,
const Intent& intent,
std::string& error
);
// Broadcast intent to all registered receivers
MessageError Broadcast(
const std::string& sender_app,
const Intent& intent,
std::string& error
);
// Check if any app handles this action
bool HasReceiverFor(const std::string& action) const;
// Get list of apps that handle an action
std::vector<std::string> GetReceiversFor(const std::string& action) const;
// Permission management
void SetAppPermission(const std::string& app_id, const std::string& permission, bool granted);
bool HasPermission(const std::string& app_id, const std::string& permission) const;
// Block specific app from sending/receiving
void BlockApp(const std::string& app_id);
void UnblockApp(const std::string& app_id);
bool IsBlocked(const std::string& app_id) const;
// Clear all registrations (for testing)
void Clear();
// Statistics
size_t GetReceiverCount() const;
size_t GetMessageCount() const { return m_message_count; }
private:
struct ReceiverEntry {
std::string app_id;
IntentFilter filter;
IntentHandler handler;
};
std::vector<ReceiverEntry> m_receivers;
std::unordered_map<std::string, std::unordered_set<std::string>> m_permissions;
std::unordered_set<std::string> m_blocked_apps;
mutable std::mutex m_mutex;
size_t m_message_count = 0;
static constexpr size_t MAX_DATA_SIZE = 1024 * 1024; // 1MB max
bool MatchesFilter(const Intent& intent, const IntentFilter& filter) const;
bool ValidateIntent(const Intent& intent, std::string& error) const;
};
// Register intents.* APIs as globals
void RegisterIntentsAPI(lua_State* L, MessageBus* bus, const std::string& app_id);
} // namespace mosis
```
### 2. Standard Intent Actions
| Action | Description | Required Permission |
|--------|-------------|---------------------|
| `share` | Share content with another app | None |
| `view` | Request app to view content | None |
| `edit` | Request app to edit content | None |
| `pick` | Request user to pick item | None |
| `call` | Initiate phone call | `phone.call` |
| `message` | Send SMS/message | `sms.send` |
| `email` | Compose email | None |
### 3. Permission Requirements
| Operation | Permission Required |
|-----------|---------------------|
| Send intent | Action-specific (see above) |
| Broadcast intent | Action-specific |
| Register receiver | None |
| Receive intent | None |
### 4. Lua API
```lua
-- Register to receive intents
intents.on("share", function(intent)
print("Received share from:", intent.from)
print("Type:", intent.type)
print("Data:", intent.data)
end)
-- Register with MIME type filter
intents.on("view", function(intent)
-- Handle view intent
end, {types = {"image/png", "image/jpeg"}})
-- Send intent to specific app
local success, err = intents.send("target.app.id", {
action = "share",
type = "text/plain",
data = "Hello world"
})
-- Broadcast to all handlers
local count = intents.broadcast({
action = "share",
type = "text/plain",
data = "Hello everyone"
})
-- Check if any app handles an action
local hasHandler = intents.hasReceiver("share")
-- Get list of apps that handle an action
local apps = intents.getReceivers("share")
-- Unregister all handlers for this app
intents.unregisterAll()
```
### 5. Error Handling
```lua
-- Errors raised via Lua errors
local ok, err = pcall(function()
intents.send("target.app", {action = "share", data = "test"})
end)
if not ok then
print("Error:", err) -- e.g., "no receivers for action: share"
end
```
---
## Test Cases
### Test 1: Send to Registered Receiver
```cpp
bool Test_MessageBusSendToReceiver(std::string& error_msg) {
mosis::MessageBus bus;
bool received = false;
mosis::Intent receivedIntent;
// Register receiver
bus.RegisterReceiver("receiver.app", {"share", {}}, [&](const mosis::Intent& i) {
received = true;
receivedIntent = i;
});
// Send intent
mosis::Intent intent{"share", "text/plain", "Hello", "sender.app"};
std::string err;
auto result = bus.Send("sender.app", "receiver.app", intent, err);
EXPECT_TRUE(result == mosis::MessageError::None);
EXPECT_TRUE(received);
EXPECT_TRUE(receivedIntent.action == "share");
EXPECT_TRUE(receivedIntent.data == "Hello");
EXPECT_TRUE(receivedIntent.from_app == "sender.app");
return true;
}
```
### Test 2: Block Unregistered Action
```cpp
bool Test_MessageBusBlockUnregistered(std::string& error_msg) {
mosis::MessageBus bus;
// No receivers registered
mosis::Intent intent{"share", "text/plain", "Hello", "sender.app"};
std::string err;
auto result = bus.Send("sender.app", "receiver.app", intent, err);
EXPECT_TRUE(result == mosis::MessageError::ReceiverNotFound);
EXPECT_TRUE(err.find("not found") != std::string::npos ||
err.find("no receiver") != std::string::npos);
return true;
}
```
### Test 3: Broadcast to Multiple Receivers
```cpp
bool Test_MessageBusBroadcast(std::string& error_msg) {
mosis::MessageBus bus;
int receiveCount = 0;
// Register multiple receivers for same action
bus.RegisterReceiver("app1", {"share", {}}, [&](const mosis::Intent&) { receiveCount++; });
bus.RegisterReceiver("app2", {"share", {}}, [&](const mosis::Intent&) { receiveCount++; });
bus.RegisterReceiver("app3", {"other", {}}, [&](const mosis::Intent&) { receiveCount++; });
// Broadcast share intent
mosis::Intent intent{"share", "text/plain", "Hello", "sender.app"};
std::string err;
auto result = bus.Broadcast("sender.app", intent, err);
EXPECT_TRUE(result == mosis::MessageError::None);
EXPECT_TRUE(receiveCount == 2); // Only app1 and app2
return true;
}
```
### Test 4: MIME Type Filtering
```cpp
bool Test_MessageBusMimeFilter(std::string& error_msg) {
mosis::MessageBus bus;
bool imageReceived = false;
bool textReceived = false;
// Register with MIME filter
bus.RegisterReceiver("image.viewer", {"view", {"image/png", "image/jpeg"}},
[&](const mosis::Intent&) { imageReceived = true; });
bus.RegisterReceiver("text.viewer", {"view", {"text/plain"}},
[&](const mosis::Intent&) { textReceived = true; });
// Send image intent
mosis::Intent imgIntent{"view", "image/png", "data", "sender"};
std::string err;
bus.Broadcast("sender", imgIntent, err);
EXPECT_TRUE(imageReceived);
EXPECT_TRUE(!textReceived);
return true;
}
```
### Test 5: Data Size Limit
```cpp
bool Test_MessageBusDataLimit(std::string& error_msg) {
mosis::MessageBus bus;
bus.RegisterReceiver("receiver", {"share", {}}, [](const mosis::Intent&) {});
// Create oversized data (> 1MB)
std::string largeData(2 * 1024 * 1024, 'x');
mosis::Intent intent{"share", "text/plain", largeData, "sender"};
std::string err;
auto result = bus.Send("sender", "receiver", intent, err);
EXPECT_TRUE(result == mosis::MessageError::DataTooLarge);
return true;
}
```
### Test 6: Blocked App Cannot Send
```cpp
bool Test_MessageBusBlockedApp(std::string& error_msg) {
mosis::MessageBus bus;
bus.RegisterReceiver("receiver", {"share", {}}, [](const mosis::Intent&) {});
bus.BlockApp("bad.app");
mosis::Intent intent{"share", "text/plain", "Hello", "bad.app"};
std::string err;
auto result = bus.Send("bad.app", "receiver", intent, err);
EXPECT_TRUE(result == mosis::MessageError::SenderBlocked);
return true;
}
```
### Test 7: Lua Integration
```cpp
bool Test_MessageBusLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::MessageBus bus;
mosis::RegisterIntentsAPI(sandbox.GetState(), &bus, "test.app");
std::string script = R"lua(
-- Test that intents global exists
if not intents then
error("intents global not found")
end
if not intents.on then
error("intents.on not found")
end
if not intents.send then
error("intents.send not found")
end
if not intents.broadcast then
error("intents.broadcast not found")
end
if not intents.hasReceiver then
error("intents.hasReceiver not found")
end
if not intents.getReceivers then
error("intents.getReceivers not found")
end
if not intents.unregisterAll then
error("intents.unregisterAll not found")
end
)lua";
bool ok = sandbox.LoadString(script, "intents_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_MessageBusSendToReceiver` - Send reaches registered receiver
- [x] `Test_MessageBusBlockUnregistered` - Unregistered action fails
- [x] `Test_MessageBusBroadcast` - Broadcast reaches all matching receivers
- [x] `Test_MessageBusMimeFilter` - MIME type filtering works
- [x] `Test_MessageBusDataLimit` - Data size limit enforced
- [x] `Test_MessageBusBlockedApp` - Blocked apps cannot send
- [x] `Test_MessageBusLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate)
- Milestone 3 (AuditLog)
---
## Notes
### Security Considerations
1. **Kernel mediation**: All messages route through MessageBus, no direct app-to-app
2. **Sender verification**: `from_app` is set by kernel, cannot be spoofed
3. **Permission checks**: Sensitive actions require permissions
4. **Size limits**: 1MB max to prevent resource exhaustion
5. **App blocking**: Misbehaving apps can be blocked
6. **Audit logging**: All inter-app communication is logged
### Privacy Features
1. **Opt-in receiving**: Apps must explicitly register for actions
2. **MIME filtering**: Receivers can filter by content type
3. **No discovery**: Apps cannot enumerate other apps (only check receivers)
---
## Next Steps
After Milestone 18 passes:
1. Milestone 19: Security Testing Suite

336
docs/SANDBOX_MILESTONE_19.md Normal file
View File

@@ -0,0 +1,336 @@
# Milestone 19: Security Testing Suite
**Status**: Complete
**Goal**: Comprehensive security test coverage with fuzzing.
---
## Overview
This milestone formalizes the security testing infrastructure:
- Unit tests for all sandbox components (already implemented in Milestones 1-18)
- Integration tests for full app lifecycle
- Fuzzer for random Lua code testing
- Security audit checklist verification
### Key Deliverables
1. **LuaFuzzer class** - Generates random Lua code and verifies sandbox integrity
2. **Integration tests** - Full lifecycle tests
3. **Security audit tests** - Verify all SANDBOX.md security requirements
---
## File Structure
```
sandbox-test/
├── src/
│ ├── main.cpp # Existing - all unit tests (135+)
│ ├── lua_fuzzer.h # NEW - Fuzzer header
│ └── lua_fuzzer.cpp # NEW - Fuzzer implementation
```
---
## Implementation Details
### 1. LuaFuzzer Class
```cpp
// lua_fuzzer.h
#pragma once
#include <string>
#include <vector>
#include <random>
#include <functional>
namespace mosis {
struct FuzzResult {
bool crashed = false;
bool sandbox_intact = true;
std::string error;
size_t iterations = 0;
size_t errors_caught = 0;
};
class LuaFuzzer {
public:
LuaFuzzer(uint32_t seed = 0);
~LuaFuzzer();
// Run fuzzing for N iterations
FuzzResult Run(size_t iterations);
// Configuration
void SetMaxCodeLength(size_t len) { m_max_code_length = len; }
void SetMaxNesting(size_t depth) { m_max_nesting = depth; }
// Statistics
size_t GetTotalRuns() const { return m_total_runs; }
size_t GetCrashes() const { return m_crashes; }
size_t GetErrorsCaught() const { return m_errors_caught; }
private:
std::mt19937 m_rng;
size_t m_max_code_length = 1000;
size_t m_max_nesting = 10;
size_t m_total_runs = 0;
size_t m_crashes = 0;
size_t m_errors_caught = 0;
// Code generators
std::string GenerateRandomCode();
std::string GenerateExpression(int depth);
std::string GenerateStatement(int depth);
std::string GenerateIdentifier();
std::string GenerateLiteral();
// Sandbox integrity verification
bool VerifySandboxIntegrity();
};
} // namespace mosis
```
### 2. Fuzzer Code Generation
The fuzzer generates random Lua code including:
- Valid expressions (arithmetic, string, table)
- Control flow (if, while, for, repeat)
- Function definitions and calls
- Table operations
- Error-inducing patterns (intentional)
- Boundary conditions
### 3. Security Audit Tests
| Test | Description | Verifies |
|------|-------------|----------|
| `AuditNoOsAccess` | os.* blocked | SANDBOX.md §1 |
| `AuditNoIoAccess` | io.* blocked | SANDBOX.md §1 |
| `AuditNoLoadfile` | loadfile blocked | SANDBOX.md §1 |
| `AuditNoDofile` | dofile blocked | SANDBOX.md §1 |
| `AuditNoBytecode` | Bytecode rejected | SANDBOX.md §2 |
| `AuditMemoryLimit` | Memory limited | SANDBOX.md §3 |
| `AuditCPULimit` | CPU limited | SANDBOX.md §3 |
| `AuditMetatableProtected` | Metatables protected | SANDBOX.md §4 |
| `AuditNoStringDump` | string.dump removed | SANDBOX.md §5 |
| `AuditPathTraversal` | Path traversal blocked | SANDBOX.md §6 |
| `AuditPermissionEnforced` | Permissions checked | SANDBOX.md §7 |
| `AuditRateLimiting` | Rate limits work | SANDBOX.md §8 |
---
## Test Cases
### Test 1: Fuzzer Runs Without Crashes
```cpp
bool Test_FuzzerNoCrashes(std::string& error_msg) {
mosis::LuaFuzzer fuzzer(12345); // Deterministic seed
auto result = fuzzer.Run(1000); // 1000 iterations
EXPECT_TRUE(!result.crashed);
EXPECT_TRUE(result.sandbox_intact);
EXPECT_TRUE(result.iterations == 1000);
return true;
}
```
### Test 2: Fuzzer Catches Errors Gracefully
```cpp
bool Test_FuzzerCatchesErrors(std::string& error_msg) {
mosis::LuaFuzzer fuzzer(54321);
auto result = fuzzer.Run(500);
// Some generated code should produce errors (caught gracefully)
EXPECT_TRUE(result.errors_caught > 0);
EXPECT_TRUE(!result.crashed);
return true;
}
```
### Test 3: Sandbox Integrity After Fuzzing
```cpp
bool Test_FuzzerSandboxIntegrity(std::string& error_msg) {
mosis::LuaFuzzer fuzzer;
// Run many iterations
auto result = fuzzer.Run(2000);
// Sandbox must still be intact
EXPECT_TRUE(result.sandbox_intact);
// Verify by running a normal script
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
EXPECT_TRUE(sandbox.LoadString("return 1 + 1", "verify"));
return true;
}
```
### Test 4: Audit - Dangerous Globals Blocked
```cpp
bool Test_AuditDangerousGlobalsBlocked(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
// All these must fail
std::vector<std::string> dangerous = {
"os.execute('ls')",
"io.open('test.txt')",
"loadfile('test.lua')",
"dofile('test.lua')",
"require('os')",
"package.loadlib('test', 'func')"
};
for (const auto& code : dangerous) {
bool ok = sandbox.LoadString(code, "audit");
if (ok) {
error_msg = "Dangerous code executed: " + code;
return false;
}
}
return true;
}
```
### Test 5: Audit - Resource Limits
```cpp
bool Test_AuditResourceLimits(std::string& error_msg) {
SandboxContext ctx = TestContext();
// Memory limit test
{
LuaSandbox sandbox(ctx);
sandbox.SetMemoryLimit(1024 * 1024); // 1MB
bool ok = sandbox.LoadString(R"lua(
local t = {}
for i = 1, 10000000 do
t[i] = string.rep("x", 1000)
end
)lua", "mem_test");
EXPECT_FALSE(ok); // Should fail due to memory limit
}
// CPU limit test
{
LuaSandbox sandbox(ctx);
sandbox.SetInstructionLimit(10000);
bool ok = sandbox.LoadString("while true do end", "cpu_test");
EXPECT_FALSE(ok); // Should fail due to CPU limit
}
return true;
}
```
### Test 6: Integration - Full App Lifecycle
```cpp
bool Test_IntegrationAppLifecycle(std::string& error_msg) {
// Create app context
SandboxContext ctx{
.app_id = "lifecycle.test.app",
.app_path = ".",
.permissions = {"storage"},
.is_system_app = false
};
// Create sandbox
LuaSandbox sandbox(ctx);
// Register all APIs
mosis::PermissionGate gate("lifecycle.test.app");
mosis::AuditLog audit;
mosis::VirtualFS vfs("lifecycle.test.app", ".");
mosis::TimerManager timers;
// Load app script
std::string script = R"lua(
-- App initialization
local data = json.encode({started = true})
-- Use storage (has permission)
storage.write("state.json", data)
-- Read back
local content = storage.read("state.json")
local state = json.decode(content)
-- Return success
return state.started == true
)lua";
bool ok = sandbox.LoadString(script, "lifecycle_test");
EXPECT_TRUE(ok);
// Cleanup
vfs.Delete("state.json");
return true;
}
```
---
## Acceptance Criteria
All tests pass:
- [x] `Test_FuzzerNoCrashes` - Fuzzer runs 100 iterations without crash
- [x] `Test_FuzzerCatchesErrors` - Fuzzer catches Lua errors gracefully
- [x] `Test_FuzzerSandboxIntegrity` - Sandbox intact after fuzzing
- [x] `Test_AuditDangerousGlobalsBlocked` - All dangerous globals blocked
- [x] `Test_AuditResourceLimits` - Memory and CPU limits enforced
- [x] `Test_IntegrationAppLifecycle` - Full app lifecycle works
---
## Dependencies
- All previous milestones (1-18)
---
## Notes
### Fuzzer Strategy
1. **Seed-based**: Deterministic with seed for reproducibility
2. **Incremental complexity**: Start simple, increase nesting
3. **Boundary testing**: Test edge cases (empty strings, huge numbers)
4. **Error injection**: Intentionally generate invalid code
### Security Audit Coverage
The audit tests verify all requirements from SANDBOX.md:
1. Dangerous standard library functions removed
2. Bytecode loading disabled
3. Memory limits enforced
4. CPU/instruction limits enforced
5. Metatables protected
6. Path traversal blocked
7. Permissions enforced
8. Rate limiting works
---
## Next Steps
After Milestone 19 passes:
1. Milestone 20: Final Integration

488
docs/SANDBOX_MILESTONE_2.md Normal file
View File

@@ -0,0 +1,488 @@
# Milestone 2: Permission System
**Status**: ✅ Complete
**Goal**: Gate API access based on app permissions.
---
## Overview
This milestone implements the permission system that controls which APIs an app can access. It defines permission categories (Normal, Dangerous, Signature) and provides mechanisms for checking permissions at runtime.
### Key Deliverables
1. **PermissionGate class** - Permission checking and enforcement
2. **Permission categories** - Normal/Dangerous/Signature classification
3. **User gesture tracking** - Detect recent user interactions
4. **Manifest parsing** - Read app permissions from manifest.json
---
## File Structure
```
src/main/cpp/sandbox/
├── lua_sandbox.h # (existing)
├── lua_sandbox.cpp # (existing)
├── permission_gate.h # NEW - Permission checking
└── permission_gate.cpp # NEW - Implementation
sandbox-test/
├── scripts/
│ ├── test_permission_normal.lua # NEW
│ ├── test_permission_dangerous.lua # NEW
│ └── test_permission_signature.lua # NEW
└── src/
└── main.cpp # Add new tests
```
---
## Implementation Details
### 1. Permission Categories
```cpp
// permission_gate.h
enum class PermissionCategory {
Normal, // Auto-granted (e.g., vibrate, internet)
Dangerous, // Requires user consent (e.g., camera, location)
Signature // System apps only (e.g., system settings, install apps)
};
```
### 2. Permission Definitions
```cpp
// permission_gate.cpp
static const std::unordered_map<std::string, PermissionInfo> PERMISSIONS = {
// Normal permissions (auto-granted)
{"internet", {PermissionCategory::Normal, "Access the internet"}},
{"vibrate", {PermissionCategory::Normal, "Vibrate the device"}},
{"wake_lock", {PermissionCategory::Normal, "Keep device awake"}},
{"notifications", {PermissionCategory::Normal, "Show notifications"}},
// Dangerous permissions (require user consent)
{"camera", {PermissionCategory::Dangerous, "Access the camera"}},
{"microphone", {PermissionCategory::Dangerous, "Record audio"}},
{"location.fine", {PermissionCategory::Dangerous, "Access precise location"}},
{"location.coarse", {PermissionCategory::Dangerous, "Access approximate location"}},
{"contacts.read", {PermissionCategory::Dangerous, "Read contacts"}},
{"contacts.write", {PermissionCategory::Dangerous, "Modify contacts"}},
{"storage.external", {PermissionCategory::Dangerous, "Access external storage"}},
{"sensors.motion", {PermissionCategory::Dangerous, "Access motion sensors"}},
{"bluetooth", {PermissionCategory::Dangerous, "Use Bluetooth"}},
{"calendar.read", {PermissionCategory::Dangerous, "Read calendar"}},
{"calendar.write", {PermissionCategory::Dangerous, "Modify calendar"}},
// Signature permissions (system apps only)
{"system.settings", {PermissionCategory::Signature, "Modify system settings"}},
{"system.install", {PermissionCategory::Signature, "Install apps"}},
{"system.uninstall", {PermissionCategory::Signature, "Uninstall apps"}},
{"system.admin", {PermissionCategory::Signature, "Device administrator"}},
};
```
### 3. PermissionGate Class
```cpp
// permission_gate.h
#pragma once
#include <string>
#include <vector>
#include <unordered_set>
#include <chrono>
struct lua_State;
namespace mosis {
struct SandboxContext; // Forward declaration
enum class PermissionCategory {
Normal,
Dangerous,
Signature
};
struct PermissionInfo {
PermissionCategory category;
std::string description;
};
class PermissionGate {
public:
explicit PermissionGate(const SandboxContext& context);
// Check if app has permission (throws Lua error if not)
bool Check(lua_State* L, const std::string& permission);
// Check without throwing (returns false if denied)
bool HasPermission(const std::string& permission) const;
// Get permission category
static PermissionCategory GetCategory(const std::string& permission);
// User gesture tracking
void RecordUserGesture();
bool HasRecentUserGesture(int ms = 5000) const;
// Runtime permission grant (called after user consent)
void GrantPermission(const std::string& permission);
void RevokePermission(const std::string& permission);
// Get all declared permissions
const std::vector<std::string>& GetDeclaredPermissions() const;
// Get all granted permissions
std::vector<std::string> GetGrantedPermissions() const;
private:
const SandboxContext& m_context;
std::unordered_set<std::string> m_runtime_grants; // Runtime-granted dangerous perms
std::chrono::steady_clock::time_point m_last_gesture;
bool CheckNormalPermission(const std::string& permission) const;
bool CheckDangerousPermission(const std::string& permission) const;
bool CheckSignaturePermission(const std::string& permission) const;
};
// Lua helper - throws error if permission denied
int RequirePermission(lua_State* L, const char* permission);
} // namespace mosis
```
### 4. Implementation
```cpp
// permission_gate.cpp
#include "permission_gate.h"
#include "lua_sandbox.h"
#include <lua.hpp>
#include <algorithm>
namespace mosis {
// Permission database
static const std::unordered_map<std::string, PermissionInfo> PERMISSIONS = {
// Normal
{"internet", {PermissionCategory::Normal, "Access the internet"}},
{"vibrate", {PermissionCategory::Normal, "Vibrate the device"}},
{"wake_lock", {PermissionCategory::Normal, "Keep device awake"}},
{"notifications", {PermissionCategory::Normal, "Show notifications"}},
// Dangerous
{"camera", {PermissionCategory::Dangerous, "Access the camera"}},
{"microphone", {PermissionCategory::Dangerous, "Record audio"}},
{"location.fine", {PermissionCategory::Dangerous, "Access precise location"}},
{"location.coarse", {PermissionCategory::Dangerous, "Access approximate location"}},
{"contacts.read", {PermissionCategory::Dangerous, "Read contacts"}},
{"contacts.write", {PermissionCategory::Dangerous, "Modify contacts"}},
{"storage.external", {PermissionCategory::Dangerous, "Access external storage"}},
{"sensors.motion", {PermissionCategory::Dangerous, "Access motion sensors"}},
{"bluetooth", {PermissionCategory::Dangerous, "Use Bluetooth"}},
// Signature
{"system.settings", {PermissionCategory::Signature, "Modify system settings"}},
{"system.install", {PermissionCategory::Signature, "Install apps"}},
{"system.admin", {PermissionCategory::Signature, "Device administrator"}},
};
PermissionGate::PermissionGate(const SandboxContext& context)
: m_context(context)
, m_last_gesture(std::chrono::steady_clock::time_point::min())
{
}
PermissionCategory PermissionGate::GetCategory(const std::string& permission) {
auto it = PERMISSIONS.find(permission);
if (it != PERMISSIONS.end()) {
return it->second.category;
}
// Unknown permissions default to Dangerous
return PermissionCategory::Dangerous;
}
bool PermissionGate::HasPermission(const std::string& permission) const {
auto category = GetCategory(permission);
switch (category) {
case PermissionCategory::Normal:
return CheckNormalPermission(permission);
case PermissionCategory::Dangerous:
return CheckDangerousPermission(permission);
case PermissionCategory::Signature:
return CheckSignaturePermission(permission);
}
return false;
}
bool PermissionGate::Check(lua_State* L, const std::string& permission) {
if (!HasPermission(permission)) {
luaL_error(L, "permission denied: %s", permission.c_str());
return false;
}
return true;
}
bool PermissionGate::CheckNormalPermission(const std::string& permission) const {
// Normal permissions are auto-granted if declared in manifest
const auto& declared = m_context.permissions;
return std::find(declared.begin(), declared.end(), permission) != declared.end();
}
bool PermissionGate::CheckDangerousPermission(const std::string& permission) const {
// Must be declared in manifest
const auto& declared = m_context.permissions;
if (std::find(declared.begin(), declared.end(), permission) == declared.end()) {
return false;
}
// Must be granted at runtime (or be a system app)
if (m_context.is_system_app) {
return true;
}
return m_runtime_grants.count(permission) > 0;
}
bool PermissionGate::CheckSignaturePermission(const std::string& permission) const {
// Only system apps get signature permissions
if (!m_context.is_system_app) {
return false;
}
// Must still be declared
const auto& declared = m_context.permissions;
return std::find(declared.begin(), declared.end(), permission) != declared.end();
}
void PermissionGate::RecordUserGesture() {
m_last_gesture = std::chrono::steady_clock::now();
}
bool PermissionGate::HasRecentUserGesture(int ms) const {
auto now = std::chrono::steady_clock::now();
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(now - m_last_gesture);
return elapsed.count() < ms;
}
void PermissionGate::GrantPermission(const std::string& permission) {
m_runtime_grants.insert(permission);
}
void PermissionGate::RevokePermission(const std::string& permission) {
m_runtime_grants.erase(permission);
}
const std::vector<std::string>& PermissionGate::GetDeclaredPermissions() const {
return m_context.permissions;
}
std::vector<std::string> PermissionGate::GetGrantedPermissions() const {
std::vector<std::string> granted;
for (const auto& perm : m_context.permissions) {
if (HasPermission(perm)) {
granted.push_back(perm);
}
}
return granted;
}
int RequirePermission(lua_State* L, const char* permission) {
// Get sandbox from registry
lua_getfield(L, LUA_REGISTRYINDEX, "__mosis_sandbox");
auto* sandbox = static_cast<LuaSandbox*>(lua_touserdata(L, -1));
lua_pop(L, 1);
if (!sandbox) {
return luaL_error(L, "sandbox not initialized");
}
// TODO: Get PermissionGate from sandbox once integrated
// For now, this is a placeholder
return 0;
}
} // namespace mosis
```
---
## Test Cases
### Test 1: Normal Permission Auto-Granted
**Script**: `scripts/test_permission_normal.lua`
```lua
-- Test that normal permissions are auto-granted when declared
-- This is called from C++ which sets up context with "internet" permission
-- If we get here, permission check passed
print("PASS: Normal permission granted")
```
**C++ Test**:
```cpp
bool Test_NormalPermissionAutoGranted(std::string& error_msg) {
SandboxContext ctx = TestContext();
ctx.permissions = {"internet"}; // Declare normal permission
LuaSandbox sandbox(ctx);
PermissionGate gate(ctx);
// Normal permissions should be auto-granted
EXPECT_TRUE(gate.HasPermission("internet"));
return true;
}
```
### Test 2: Dangerous Permission Requires Grant
**C++ Test**:
```cpp
bool Test_DangerousPermissionRequiresGrant(std::string& error_msg) {
SandboxContext ctx = TestContext();
ctx.permissions = {"camera"}; // Declare dangerous permission
PermissionGate gate(ctx);
// Not granted yet
EXPECT_FALSE(gate.HasPermission("camera"));
// Grant at runtime
gate.GrantPermission("camera");
// Now should have it
EXPECT_TRUE(gate.HasPermission("camera"));
// Revoke
gate.RevokePermission("camera");
EXPECT_FALSE(gate.HasPermission("camera"));
return true;
}
```
### Test 3: Signature Permission System Only
**C++ Test**:
```cpp
bool Test_SignaturePermissionSystemOnly(std::string& error_msg) {
// Non-system app
SandboxContext ctx = TestContext();
ctx.permissions = {"system.settings"};
ctx.is_system_app = false;
PermissionGate gate(ctx);
EXPECT_FALSE(gate.HasPermission("system.settings"));
// System app
SandboxContext sys_ctx = TestContext();
sys_ctx.permissions = {"system.settings"};
sys_ctx.is_system_app = true;
PermissionGate sys_gate(sys_ctx);
EXPECT_TRUE(sys_gate.HasPermission("system.settings"));
return true;
}
```
### Test 4: User Gesture Tracking
**C++ Test**:
```cpp
bool Test_UserGestureRequired(std::string& error_msg) {
SandboxContext ctx = TestContext();
PermissionGate gate(ctx);
// No recent gesture
EXPECT_FALSE(gate.HasRecentUserGesture(5000));
// Record gesture
gate.RecordUserGesture();
// Should have recent gesture
EXPECT_TRUE(gate.HasRecentUserGesture(5000));
return true;
}
```
### Test 5: Undeclared Permission Denied
**C++ Test**:
```cpp
bool Test_UndeclaredPermissionDenied(std::string& error_msg) {
SandboxContext ctx = TestContext();
ctx.permissions = {}; // No permissions declared
PermissionGate gate(ctx);
// Even normal permissions need to be declared
EXPECT_FALSE(gate.HasPermission("internet"));
return true;
}
```
---
## Build & Test Commands
### Build
```bash
cd sandbox-test
cmake -B build -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake
cmake --build build --config Debug
```
### Run All Tests
```bash
./build/Debug/sandbox-test.exe
```
### Run Permission Tests Only
```bash
./build/Debug/sandbox-test.exe --test Permission
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_NormalPermissionAutoGranted` - Normal perms auto-granted when declared
- [x] `Test_DangerousPermissionRequiresGrant` - Dangerous perms need runtime grant
- [x] `Test_SignaturePermissionSystemOnly` - Signature perms only for system apps
- [x] `Test_UserGestureTracking` - User gesture tracking works
- [x] `Test_UndeclaredPermissionDenied` - Undeclared perms always denied
- [x] `Test_SystemAppGetsDangerousAuto` - System apps get dangerous perms auto
- [x] `Test_PermissionCategoryCheck` - Permission categories are correct
---
## Integration Notes
After Milestone 2:
1. Add `PermissionGate` to `LuaSandbox` class
2. Call `RequirePermission()` before sensitive operations
3. Wire up user gesture recording from touch events
---
## Next Steps
After Milestone 2 passes:
1. Milestone 3: Audit Logging & Rate Limiting
2. Use permission gate in all subsequent API implementations

204
docs/SANDBOX_MILESTONE_20.md Normal file
View File

@@ -0,0 +1,204 @@
# Milestone 20: Kernel Integration
**Status**: Complete
**Goal**: Multi-app sandbox orchestrator for kernel integration.
---
## Overview
This milestone creates the LuaSandboxManager class that orchestrates multiple isolated Lua app sandboxes, providing the final integration point for the kernel.
### Key Deliverables
1. **LuaSandboxManager class** - Manages multiple concurrent app sandboxes
2. **AppSandbox struct** - Per-app container with all components
3. **Lifecycle management** - Start/stop apps with full resource cleanup
4. **Shared components** - Cross-app services (AuditLog, RateLimiter, MessageBus, TimerManager)
---
## File Structure
```
src/main/cpp/sandbox/
├── sandbox_manager.h # NEW - Multi-app orchestrator header
├── sandbox_manager.cpp # NEW - Orchestrator implementation
└── [all previous M1-19 files]
```
---
## Implementation Details
### 1. AppSandbox Struct
Per-app container holding all isolated components:
```cpp
struct AppSandbox {
std::unique_ptr<LuaSandbox> lua;
std::unique_ptr<PermissionGate> permissions;
std::unique_ptr<VirtualFS> filesystem;
std::unique_ptr<DatabaseManager> database;
std::unique_ptr<NetworkManager> network;
std::unique_ptr<WebSocketManager> websocket;
std::unique_ptr<CameraInterface> camera;
std::unique_ptr<MicrophoneInterface> microphone;
std::unique_ptr<AudioOutputInterface> audio;
std::unique_ptr<LocationInterface> location;
std::unique_ptr<SensorInterface> sensors;
std::unique_ptr<BluetoothInterface> bluetooth;
std::unique_ptr<ContactsInterface> contacts;
SandboxContext context;
bool is_running = false;
};
```
### 2. LuaSandboxManager Class
```cpp
class LuaSandboxManager {
public:
explicit LuaSandboxManager(const std::string& data_root = ".");
~LuaSandboxManager();
// App lifecycle
bool StartApp(const std::string& app_id, const std::string& app_path,
const std::vector<std::string>& permissions,
bool is_system_app = false);
bool StopApp(const std::string& app_id);
bool IsAppRunning(const std::string& app_id) const;
// Get app sandbox for direct access
AppSandbox* GetApp(const std::string& app_id);
const AppSandbox* GetApp(const std::string& app_id) const;
// Execute Lua code in app context
bool ExecuteCode(const std::string& app_id, const std::string& code,
const std::string& source_name = "script");
bool LoadFile(const std::string& app_id, const std::string& path);
// User gesture tracking
void RecordUserGesture(const std::string& app_id);
// Timer management
void UpdateTimers();
// Get all running app IDs
std::vector<std::string> GetRunningApps() const;
size_t GetRunningAppCount() const;
// Shared components
AuditLog& GetAuditLog();
RateLimiter& GetRateLimiter();
MessageBus& GetMessageBus();
TimerManager& GetTimerManager();
// Configuration
void SetDefaultLimits(const SandboxLimits& limits);
const SandboxLimits& GetDefaultLimits() const;
};
```
### 3. App Lifecycle
**StartApp Flow:**
1. Create SandboxContext from parameters
2. Create AppSandbox with all components
3. Register JSON and Crypto APIs
4. Store manager/app references in Lua registry
5. Mark as running and store in map
**StopApp Flow:**
1. Clear app timers
2. Close WebSocket connections
3. Shutdown camera/microphone/audio
4. Shutdown location/sensors/bluetooth
5. Clear temp files in VirtualFS
6. Close database connections
7. Unregister from message bus
8. Remove from map
### 4. Resource Isolation
Each app gets:
- Isolated Lua state
- Isolated permissions
- Isolated filesystem (under `data_root/apps/{app_id}/`)
- Isolated database directory
- Isolated network manager
- Isolated hardware interfaces
Shared across apps:
- AuditLog (thread-safe)
- RateLimiter (app-keyed)
- MessageBus (for inter-app communication)
- TimerManager (timers keyed by app_id)
---
## Test Cases
| Test | Description |
|------|-------------|
| `Test_ManagerStartStopApp` | Start and stop app lifecycle |
| `Test_ManagerMultipleApps` | Run multiple apps concurrently |
| `Test_ManagerAppIsolation` | Verify separate Lua states |
| `Test_ManagerExecuteCode` | Execute code in app context |
| `Test_ManagerResourceCleanup` | Verify cleanup on stop |
| `Test_ManagerUserGesture` | User gesture forwarding |
| `Test_ManagerDoubleStartStop` | Idempotent start/stop |
| `Test_ManagerSharedComponents` | Access shared components |
---
## Acceptance Criteria
All tests pass (149 total):
- [x] All M1-19 tests (141 tests)
- [x] `Test_ManagerStartStopApp` - App lifecycle works
- [x] `Test_ManagerMultipleApps` - Multiple concurrent apps
- [x] `Test_ManagerAppIsolation` - Lua states are isolated
- [x] `Test_ManagerExecuteCode` - Code execution works
- [x] `Test_ManagerResourceCleanup` - Resources cleaned on stop
- [x] `Test_ManagerUserGesture` - Gesture forwarding works
- [x] `Test_ManagerDoubleStartStop` - Idempotent operations
- [x] `Test_ManagerSharedComponents` - Shared components accessible
---
## Dependencies
- All previous milestones (1-19)
---
## Notes
### Thread Safety
LuaSandboxManager is thread-safe via mutex protection on all public methods that access the app map.
### Memory Management
All components use unique_ptr for automatic cleanup. When StopApp is called:
1. CleanupApp() releases resources gracefully
2. unique_ptr destructors clean up remaining state
3. App is removed from map
### Integration with Kernel
The kernel can integrate with LuaSandboxManager by:
1. Creating a single manager instance at startup
2. Calling StartApp() for each installed app
3. Calling UpdateTimers() from the main loop
4. Calling RecordUserGesture() on UI interactions
5. Using GetApp() to access specific app components
6. Calling StopApp() when apps are closed
---
## Milestone Complete
All 149 tests pass. The sandbox system is fully integrated and ready for kernel use.

357
docs/SANDBOX_MILESTONE_3.md Normal file
View File

@@ -0,0 +1,357 @@
# Milestone 3: Audit Logging & Rate Limiting
**Status**: Complete ✓
**Goal**: Track security events and prevent API abuse.
---
## Overview
This milestone adds security event logging and rate limiting to prevent abuse. The audit log tracks permission checks, sandbox violations, and resource usage. The rate limiter uses a token bucket algorithm to limit API call frequency.
### Key Deliverables
1. **AuditLog class** - Security event logging with ring buffer
2. **RateLimiter class** - Token bucket rate limiting
3. **Thread safety** - Concurrent access support
---
## File Structure
```
src/main/cpp/sandbox/
├── lua_sandbox.h # (existing)
├── lua_sandbox.cpp # (existing)
├── permission_gate.h # (existing)
├── permission_gate.cpp # (existing)
├── audit_log.h # NEW - Audit logging
├── audit_log.cpp # NEW - Implementation
├── rate_limiter.h # NEW - Rate limiting
└── rate_limiter.cpp # NEW - Implementation
```
---
## Implementation Details
### 1. AuditEvent Enum
```cpp
// audit_log.h
enum class AuditEvent {
// Lifecycle
AppStart,
AppStop,
// Permissions
PermissionCheck,
PermissionGranted,
PermissionDenied,
// Network
NetworkRequest,
NetworkBlocked,
// Storage
FileAccess,
FileBlocked,
DatabaseAccess,
// Hardware
CameraAccess,
MicrophoneAccess,
LocationAccess,
// Security
SandboxViolation,
ResourceLimitHit,
RateLimitHit,
// Other
Custom
};
```
### 2. AuditLog Class
```cpp
// audit_log.h
#pragma once
#include <string>
#include <vector>
#include <mutex>
#include <chrono>
namespace mosis {
enum class AuditEvent { /* ... */ };
struct AuditEntry {
std::chrono::system_clock::time_point timestamp;
AuditEvent event;
std::string app_id;
std::string details;
bool success;
};
class AuditLog {
public:
explicit AuditLog(size_t max_entries = 10000);
// Log an event
void Log(AuditEvent event, const std::string& app_id,
const std::string& details = "", bool success = true);
// Query entries
std::vector<AuditEntry> GetEntries(size_t count = 100) const;
std::vector<AuditEntry> GetEntriesForApp(const std::string& app_id,
size_t count = 100) const;
std::vector<AuditEntry> GetEntriesByEvent(AuditEvent event,
size_t count = 100) const;
// Statistics
size_t GetTotalEntries() const;
size_t CountEvents(AuditEvent event, const std::string& app_id = "") const;
// Clear
void Clear();
private:
mutable std::mutex m_mutex;
std::vector<AuditEntry> m_entries;
size_t m_max_entries;
size_t m_write_index = 0;
size_t m_total_logged = 0;
};
// Global audit log (singleton pattern)
AuditLog& GetAuditLog();
} // namespace mosis
```
### 3. RateLimiter Class (Token Bucket)
```cpp
// rate_limiter.h
#pragma once
#include <string>
#include <unordered_map>
#include <mutex>
#include <chrono>
namespace mosis {
struct RateLimitConfig {
double tokens_per_second; // Refill rate
double max_tokens; // Bucket capacity
};
class RateLimiter {
public:
// Default limits for common operations
RateLimiter();
// Check if operation is allowed (consumes token if yes)
bool Check(const std::string& app_id, const std::string& operation);
// Check without consuming
bool CanProceed(const std::string& app_id, const std::string& operation) const;
// Configure limits for an operation
void SetLimit(const std::string& operation, const RateLimitConfig& config);
// Get current token count
double GetTokens(const std::string& app_id, const std::string& operation) const;
// Reset an app's tokens (e.g., on app restart)
void ResetApp(const std::string& app_id);
private:
struct Bucket {
double tokens;
std::chrono::steady_clock::time_point last_refill;
};
void Refill(Bucket& bucket, const RateLimitConfig& config) const;
Bucket& GetBucket(const std::string& app_id, const std::string& operation);
mutable std::mutex m_mutex;
std::unordered_map<std::string, RateLimitConfig> m_configs;
std::unordered_map<std::string, Bucket> m_buckets; // Key: app_id:operation
};
// Global rate limiter
RateLimiter& GetRateLimiter();
} // namespace mosis
```
### 4. Default Rate Limits
```cpp
// rate_limiter.cpp
RateLimiter::RateLimiter() {
// Network
SetLimit("network.request", {10.0, 100.0}); // 10/sec, burst 100
SetLimit("network.websocket", {2.0, 10.0}); // 2/sec, burst 10
// Storage
SetLimit("storage.read", {100.0, 500.0}); // 100/sec, burst 500
SetLimit("storage.write", {20.0, 100.0}); // 20/sec, burst 100
SetLimit("database.query", {50.0, 200.0}); // 50/sec, burst 200
// Hardware
SetLimit("camera.capture", {30.0, 30.0}); // 30 fps max
SetLimit("microphone.record", {1.0, 1.0}); // 1 session at a time
SetLimit("location.request", {1.0, 5.0}); // 1/sec, burst 5
// Timers
SetLimit("timer.create", {10.0, 100.0}); // 10/sec, burst 100
}
```
---
## Test Cases
### Test 1: Audit Log Basic
```cpp
bool Test_AuditLogBasic(std::string& error_msg) {
mosis::AuditLog log(1000);
log.Log(mosis::AuditEvent::AppStart, "test.app", "App started");
log.Log(mosis::AuditEvent::PermissionCheck, "test.app", "camera", true);
log.Log(mosis::AuditEvent::PermissionDenied, "test.app", "microphone", false);
auto entries = log.GetEntries(10);
EXPECT_TRUE(entries.size() == 3);
auto app_entries = log.GetEntriesForApp("test.app", 10);
EXPECT_TRUE(app_entries.size() == 3);
return true;
}
```
### Test 2: Audit Log Ring Buffer
```cpp
bool Test_AuditLogRingBuffer(std::string& error_msg) {
mosis::AuditLog log(100); // Small buffer
// Log more than capacity
for (int i = 0; i < 200; i++) {
log.Log(mosis::AuditEvent::Custom, "test.app", std::to_string(i));
}
// Should only have latest 100
auto entries = log.GetEntries(200);
EXPECT_TRUE(entries.size() == 100);
// Total logged should be 200
EXPECT_TRUE(log.GetTotalEntries() == 200);
return true;
}
```
### Test 3: Rate Limiter Basic
```cpp
bool Test_RateLimiterBasic(std::string& error_msg) {
mosis::RateLimiter limiter;
// Should succeed initially (has tokens)
EXPECT_TRUE(limiter.Check("test.app", "network.request"));
return true;
}
```
### Test 4: Rate Limiter Exhaustion
```cpp
bool Test_RateLimiterExhaustion(std::string& error_msg) {
mosis::RateLimiter limiter;
limiter.SetLimit("test.op", {0.0, 5.0}); // 5 tokens, no refill
// Use all tokens
for (int i = 0; i < 5; i++) {
EXPECT_TRUE(limiter.Check("test.app", "test.op"));
}
// Should be denied now
EXPECT_FALSE(limiter.Check("test.app", "test.op"));
return true;
}
```
### Test 5: Rate Limiter Refill
```cpp
bool Test_RateLimiterRefill(std::string& error_msg) {
mosis::RateLimiter limiter;
limiter.SetLimit("test.op", {1000.0, 1.0}); // 1000/sec, max 1
// Use the token
EXPECT_TRUE(limiter.Check("test.app", "test.op"));
EXPECT_FALSE(limiter.Check("test.app", "test.op"));
// Wait a bit for refill (1ms = 1 token at 1000/sec)
std::this_thread::sleep_for(std::chrono::milliseconds(2));
// Should have token again
EXPECT_TRUE(limiter.Check("test.app", "test.op"));
return true;
}
```
### Test 6: App Isolation
```cpp
bool Test_RateLimiterAppIsolation(std::string& error_msg) {
mosis::RateLimiter limiter;
limiter.SetLimit("test.op", {0.0, 1.0}); // 1 token, no refill
// App 1 uses its token
EXPECT_TRUE(limiter.Check("app1", "test.op"));
EXPECT_FALSE(limiter.Check("app1", "test.op"));
// App 2 should still have its token
EXPECT_TRUE(limiter.Check("app2", "test.op"));
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_AuditLogBasic` - Logging and retrieval work
- [x] `Test_AuditLogRingBuffer` - Ring buffer wraps correctly
- [x] `Test_AuditLogThreadSafe` - Thread safety verified
- [x] `Test_RateLimiterBasic` - Basic rate limiting works
- [x] `Test_RateLimiterExhaustion` - Tokens exhaust correctly
- [x] `Test_RateLimiterRefill` - Tokens refill over time
- [x] `Test_RateLimiterAppIsolation` - Apps have separate buckets
- [x] `Test_RateLimiterReset` - App reset clears buckets
- [x] `Test_RateLimiterNoConfig` - Unconfigured ops allowed
---
## Next Steps
After Milestone 3 passes:
1. Integrate AuditLog into LuaSandbox and PermissionGate
2. Integrate RateLimiter into API implementations
3. Milestone 4: Safe Path & Require

241
docs/SANDBOX_MILESTONE_4.md Normal file
View File

@@ -0,0 +1,241 @@
# Milestone 4: Safe Path & Require
**Status**: Complete ✓
**Goal**: Secure file access within app sandbox.
---
## Overview
This milestone implements path validation to prevent directory traversal attacks and a safe `require()` function that loads Lua modules only from the app's scripts directory.
### Key Deliverables
1. **PathSandbox class** - Path validation and canonicalization
2. **SafeRequire function** - Secure module loader
3. **Module caching** - Registry-based cache for loaded modules
---
## File Structure
```
src/main/cpp/sandbox/
├── lua_sandbox.h # (existing)
├── lua_sandbox.cpp # (existing)
├── permission_gate.h # (existing)
├── permission_gate.cpp # (existing)
├── audit_log.h # (existing)
├── audit_log.cpp # (existing)
├── rate_limiter.h # (existing)
├── rate_limiter.cpp # (existing)
├── path_sandbox.h # NEW - Path validation
└── path_sandbox.cpp # NEW - Implementation
```
---
## Implementation Details
### 1. PathSandbox Class
```cpp
// path_sandbox.h
#pragma once
#include <string>
#include <filesystem>
namespace mosis {
class PathSandbox {
public:
explicit PathSandbox(const std::string& app_path);
// Validate a path is within the sandbox
bool ValidatePath(const std::string& path, std::string& out_canonical);
// Check if path contains traversal attempts
static bool ContainsTraversal(const std::string& path);
// Check if path is absolute
static bool IsAbsolutePath(const std::string& path);
// Normalize path separators and remove redundant components
static std::string NormalizePath(const std::string& path);
// Get the app's base path
const std::string& GetAppPath() const { return m_app_path; }
// Resolve a relative path to full path within sandbox
std::string ResolvePath(const std::string& relative_path);
private:
std::string m_app_path;
};
} // namespace mosis
```
### 2. SafeRequire Function
```cpp
// path_sandbox.cpp
// Safe require implementation for Lua
int SafeRequire(lua_State* L);
// Register safe require as global
void RegisterSafeRequire(lua_State* L, PathSandbox* sandbox);
```
### 3. Module Name Validation
Valid module names contain only:
- Alphanumeric characters (a-z, A-Z, 0-9)
- Underscores (_)
- Dots (.) for submodules
Examples:
- `utils` → loads `scripts/utils.lua`
- `ui.button` → loads `scripts/ui/button.lua`
---
## Test Cases
### Test 1: Rejects Directory Traversal
```cpp
bool Test_PathRejectsTraversal(std::string& error_msg) {
mosis::PathSandbox sandbox("D:/test/app");
EXPECT_TRUE(mosis::PathSandbox::ContainsTraversal("../etc/passwd"));
EXPECT_TRUE(mosis::PathSandbox::ContainsTraversal("foo/../../../bar"));
EXPECT_TRUE(mosis::PathSandbox::ContainsTraversal("..\\windows\\system32"));
std::string canonical;
EXPECT_FALSE(sandbox.ValidatePath("../etc/passwd", canonical));
EXPECT_FALSE(sandbox.ValidatePath("data/../../../etc/passwd", canonical));
return true;
}
```
### Test 2: Rejects Absolute Paths
```cpp
bool Test_PathRejectsAbsolute(std::string& error_msg) {
EXPECT_TRUE(mosis::PathSandbox::IsAbsolutePath("/etc/passwd"));
EXPECT_TRUE(mosis::PathSandbox::IsAbsolutePath("C:\\Windows\\System32"));
EXPECT_TRUE(mosis::PathSandbox::IsAbsolutePath("D:/test/file.txt"));
EXPECT_FALSE(mosis::PathSandbox::IsAbsolutePath("scripts/utils.lua"));
EXPECT_FALSE(mosis::PathSandbox::IsAbsolutePath("./data/file.txt"));
return true;
}
```
### Test 3: Accepts Valid Paths
```cpp
bool Test_PathAcceptsValid(std::string& error_msg) {
mosis::PathSandbox sandbox("D:/test/app");
std::string canonical;
EXPECT_TRUE(sandbox.ValidatePath("scripts/utils.lua", canonical));
EXPECT_TRUE(sandbox.ValidatePath("data/config.json", canonical));
EXPECT_TRUE(sandbox.ValidatePath("./scripts/ui/button.lua", canonical));
return true;
}
```
### Test 4: Safe Require Loads Modules
```cpp
bool Test_SafeRequireLoads(std::string& error_msg) {
// Create sandbox with test scripts directory
SandboxContext ctx = TestContext();
ctx.app_path = GetScriptsDir(); // Use scripts/ as app path
LuaSandbox sandbox(ctx);
// Should be able to require a test module
std::string script = R"(
local m = require("test_module")
return m.value == 42
)";
EXPECT_TRUE(sandbox.LoadString(script, "require_test"));
return true;
}
```
### Test 5: Safe Require Caches Modules
```cpp
bool Test_SafeRequireCaches(std::string& error_msg) {
SandboxContext ctx = TestContext();
ctx.app_path = GetScriptsDir();
LuaSandbox sandbox(ctx);
std::string script = R"(
local m1 = require("test_module")
local m2 = require("test_module")
return m1 == m2 -- Should be same table (cached)
)";
EXPECT_TRUE(sandbox.LoadString(script, "cache_test"));
return true;
}
```
### Test 6: Safe Require Rejects Invalid Names
```cpp
bool Test_SafeRequireRejectsInvalid(std::string& error_msg) {
SandboxContext ctx = TestContext();
ctx.app_path = GetScriptsDir();
LuaSandbox sandbox(ctx);
// Should reject path traversal in module name
EXPECT_FALSE(sandbox.LoadString("require('../evil')", "evil_require"));
// Should reject absolute paths
EXPECT_FALSE(sandbox.LoadString("require('/etc/passwd')", "abs_require"));
// Should reject special characters
EXPECT_FALSE(sandbox.LoadString("require('foo;bar')", "special_require"));
return true;
}
```
---
## Acceptance Criteria
All tests pass:
- [x] `Test_PathRejectsTraversal` - Block .. traversal
- [x] `Test_PathRejectsAbsolute` - Block absolute paths
- [x] `Test_PathAcceptsValid` - Allow valid relative paths
- [x] `Test_ModuleNameValidation` - Validate module names
- [x] `Test_ModuleToPath` - Convert module names to paths
- [x] `Test_SafeRequireLoads` - Load modules from scripts/
- [x] `Test_SafeRequireCaches` - Cache loaded modules
- [x] `Test_SafeRequireRejectsInvalid` - Reject malicious require calls
---
## Next Steps
After Milestone 4 passes:
1. Milestone 5: Timer & Callback System
2. Milestone 6: JSON & Crypto APIs

333
docs/SANDBOX_MILESTONE_5.md Normal file
View File

@@ -0,0 +1,333 @@
# Milestone 5: Timer & Callback System
**Status**: Complete
**Goal**: Safe timer APIs managed by kernel.
---
## Overview
This milestone implements JavaScript-style timer APIs (`setTimeout`, `setInterval`) that are:
- Managed by the kernel (not Lua coroutines)
- Subject to per-app limits
- Properly cleaned up when apps stop
- Integrated with sandbox instruction counting
### Key Deliverables
1. **TimerManager class** - Central timer management
2. **Lua timer APIs** - setTimeout, setInterval, clearTimeout, clearInterval
3. **Kernel integration** - Fire timers from main loop
---
## File Structure
```
src/main/cpp/sandbox/
├── lua_sandbox.h # (existing)
├── lua_sandbox.cpp # (existing)
├── timer_manager.h # NEW - Timer management
└── timer_manager.cpp # NEW - Implementation
```
---
## Implementation Details
### 1. TimerManager Class
```cpp
// timer_manager.h
#pragma once
#include <string>
#include <functional>
#include <queue>
#include <unordered_map>
#include <chrono>
#include <mutex>
struct lua_State;
namespace mosis {
using TimerId = uint64_t;
using TimePoint = std::chrono::steady_clock::time_point;
using Duration = std::chrono::milliseconds;
struct Timer {
TimerId id;
std::string app_id;
TimePoint fire_time;
Duration interval; // 0 for setTimeout, >0 for setInterval
int callback_ref; // Lua registry reference
lua_State* L; // Lua state that owns the callback
bool cancelled = false;
};
class TimerManager {
public:
TimerManager();
~TimerManager();
// Create timers (returns timer ID)
TimerId SetTimeout(lua_State* L, const std::string& app_id,
int callback_ref, int delay_ms);
TimerId SetInterval(lua_State* L, const std::string& app_id,
int callback_ref, int interval_ms);
// Cancel timers
bool ClearTimer(const std::string& app_id, TimerId id);
// Cancel all timers for an app
void ClearAppTimers(const std::string& app_id);
// Process timers (call from main loop)
// Returns number of timers fired
int ProcessTimers();
// Get timer count for an app
size_t GetTimerCount(const std::string& app_id) const;
// Configuration
static constexpr size_t MAX_TIMERS_PER_APP = 100;
static constexpr int MIN_INTERVAL_MS = 10;
static constexpr int MIN_TIMEOUT_MS = 0;
private:
struct TimerCompare {
bool operator()(const Timer& a, const Timer& b) const {
return a.fire_time > b.fire_time; // Min-heap
}
};
TimerId m_next_id = 1;
std::priority_queue<Timer, std::vector<Timer>, TimerCompare> m_timers;
std::unordered_map<std::string, size_t> m_app_timer_counts;
mutable std::mutex m_mutex;
void FireTimer(Timer& timer);
};
// Lua API registration
void RegisterTimerAPI(lua_State* L, TimerManager* manager, const std::string& app_id);
} // namespace mosis
```
### 2. Lua Timer APIs
```lua
-- setTimeout: fire callback once after delay
local id = setTimeout(function()
print("Fired after 1000ms")
end, 1000)
-- clearTimeout: cancel a pending timeout
clearTimeout(id)
-- setInterval: fire callback repeatedly
local id = setInterval(function()
print("Fires every 500ms")
end, 500)
-- clearInterval: stop an interval
clearInterval(id)
```
### 3. Timer Limits
| Limit | Value | Reason |
|-------|-------|--------|
| Max timers per app | 100 | Prevent resource exhaustion |
| Min interval | 10ms | Prevent CPU spinning |
| Min timeout | 0ms | Allow immediate callbacks |
---
## Test Cases
### Test 1: SetTimeout Fires
```cpp
bool Test_SetTimeoutFires(std::string& error_msg) {
mosis::TimerManager manager;
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
// Register timer API
mosis::RegisterTimerAPI(sandbox.GetState(), &manager, ctx.app_id);
// Set a timeout
sandbox.LoadString(R"(
fired = false
setTimeout(function() fired = true end, 50)
)", "timeout_test");
// Process timers after delay
std::this_thread::sleep_for(std::chrono::milliseconds(100));
manager.ProcessTimers();
// Check if callback fired
sandbox.LoadString("assert(fired == true)", "check");
return true;
}
```
### Test 2: SetInterval Fires Multiple Times
```cpp
bool Test_SetIntervalFires(std::string& error_msg) {
mosis::TimerManager manager;
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterTimerAPI(sandbox.GetState(), &manager, ctx.app_id);
sandbox.LoadString(R"(
count = 0
setInterval(function() count = count + 1 end, 30)
)", "interval_test");
// Process multiple times
for (int i = 0; i < 5; i++) {
std::this_thread::sleep_for(std::chrono::milliseconds(40));
manager.ProcessTimers();
}
// Should have fired multiple times
sandbox.LoadString("assert(count >= 3)", "check");
return true;
}
```
### Test 3: ClearTimeout Cancels
```cpp
bool Test_ClearTimeoutCancels(std::string& error_msg) {
mosis::TimerManager manager;
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterTimerAPI(sandbox.GetState(), &manager, ctx.app_id);
sandbox.LoadString(R"(
fired = false
local id = setTimeout(function() fired = true end, 100)
clearTimeout(id)
)", "clear_test");
std::this_thread::sleep_for(std::chrono::milliseconds(150));
manager.ProcessTimers();
// Should NOT have fired
sandbox.LoadString("assert(fired == false)", "check");
return true;
}
```
### Test 4: Timer Limit Enforced
```cpp
bool Test_TimerLimitEnforced(std::string& error_msg) {
mosis::TimerManager manager;
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterTimerAPI(sandbox.GetState(), &manager, ctx.app_id);
// Try to create too many timers
sandbox.LoadString(R"(
for i = 1, 150 do
setTimeout(function() end, 1000000)
end
)", "limit_test");
// Should be capped at MAX_TIMERS_PER_APP
EXPECT_TRUE(manager.GetTimerCount(ctx.app_id) <= 100);
return true;
}
```
### Test 5: ClearAppTimers Cleanup
```cpp
bool Test_ClearAppTimersCleanup(std::string& error_msg) {
mosis::TimerManager manager;
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterTimerAPI(sandbox.GetState(), &manager, ctx.app_id);
sandbox.LoadString(R"(
for i = 1, 10 do
setTimeout(function() end, 1000000)
end
)", "cleanup_test");
EXPECT_TRUE(manager.GetTimerCount(ctx.app_id) == 10);
// Clear all timers for app
manager.ClearAppTimers(ctx.app_id);
EXPECT_TRUE(manager.GetTimerCount(ctx.app_id) == 0);
return true;
}
```
### Test 6: Minimum Interval Enforced
```cpp
bool Test_MinIntervalEnforced(std::string& error_msg) {
mosis::TimerManager manager;
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterTimerAPI(sandbox.GetState(), &manager, ctx.app_id);
// Try to set interval less than minimum
sandbox.LoadString(R"(
count = 0
setInterval(function() count = count + 1 end, 1) -- 1ms, should be clamped to 10ms
)", "min_interval_test");
// With 1ms interval, in 50ms we'd get 50 callbacks
// With 10ms minimum, we should get ~5
std::this_thread::sleep_for(std::chrono::milliseconds(55));
for (int i = 0; i < 10; i++) {
manager.ProcessTimers();
}
sandbox.LoadString("assert(count <= 10)", "check");
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_SetTimeoutFires` - Timeout fires after delay
- [x] `Test_SetIntervalFires` - Interval fires repeatedly
- [x] `Test_ClearTimeoutCancels` - Cancelled timeout doesn't fire
- [x] `Test_ClearIntervalCancels` - Cancelled interval stops
- [x] `Test_TimerLimitEnforced` - Max 100 timers per app
- [x] `Test_ClearAppTimersCleanup` - All app timers cleared on stop
- [x] `Test_MinIntervalEnforced` - Interval clamped to 10ms minimum
---
## Next Steps
After Milestone 5 passes:
1. Milestone 6: JSON & Crypto APIs
2. Milestone 7: Virtual Filesystem

411
docs/SANDBOX_MILESTONE_6.md Normal file
View File

@@ -0,0 +1,411 @@
# Milestone 6: JSON & Crypto APIs
**Status**: ✅ Complete
**Goal**: Safe data parsing and cryptographic primitives.
---
## Overview
This milestone implements secure JSON encoding/decoding and cryptographic primitives for Lua apps:
- JSON API with depth/size limits to prevent DoS attacks
- Crypto API with secure random, hashing, and HMAC
- Replace insecure `math.random` with per-app CSPRNG
### Key Deliverables
1. **JSON API** - Safe encode/decode with limits
2. **Crypto API** - Hash, HMAC, random bytes
3. **Secure math.random replacement**
---
## File Structure
```
src/main/cpp/sandbox/
├── json_api.h # NEW - JSON API header
├── json_api.cpp # NEW - JSON implementation
├── crypto_api.h # NEW - Crypto API header
└── crypto_api.cpp # NEW - Crypto implementation
```
---
## Implementation Details
### 1. JSON API
```cpp
// json_api.h
#pragma once
#include <string>
struct lua_State;
namespace mosis {
// Configuration limits
struct JsonLimits {
int max_depth = 32; // Maximum nesting depth
size_t max_string_length = 1 * 1024 * 1024; // 1 MB per string
size_t max_output_size = 10 * 1024 * 1024; // 10 MB total output
size_t max_array_size = 100000; // Max elements in array
size_t max_object_size = 10000; // Max keys in object
};
// Register json.encode() and json.decode() as globals
void RegisterJsonAPI(lua_State* L, const JsonLimits& limits = JsonLimits{});
} // namespace mosis
```
#### json.decode(str) -> table
- Parse JSON string to Lua table
- Enforce depth limit (32 levels)
- Enforce string length limit (1 MB)
- Enforce array/object size limits
- Return `nil, error_message` on failure
#### json.encode(table) -> string
- Convert Lua table to JSON string
- Detect cycles (error on circular references)
- Enforce output size limit (10 MB)
- Handle Lua types: nil, boolean, number, string, table
- Error on unsupported types (functions, userdata, threads)
### 2. Crypto API
```cpp
// crypto_api.h
#pragma once
#include <string>
#include <cstdint>
#include <random>
struct lua_State;
namespace mosis {
// Per-app cryptographically secure RNG
class SecureRandom {
public:
SecureRandom();
// Get random bytes
std::string GetBytes(size_t count);
// Get random integer in range [min, max]
int64_t GetInt(int64_t min, int64_t max);
// Get random double in range [0.0, 1.0)
double GetDouble();
private:
std::random_device m_rd;
std::mt19937_64 m_gen;
};
// Register crypto.* APIs as globals
void RegisterCryptoAPI(lua_State* L);
// Register secure math.random replacement
void RegisterSecureMathRandom(lua_State* L, SecureRandom* rng);
} // namespace mosis
```
#### crypto.randomBytes(n) -> string
- Generate `n` cryptographically secure random bytes
- Limit: max 1024 bytes per call
- Use system CSPRNG (std::random_device or platform-specific)
#### crypto.hash(algorithm, data) -> string
- Supported algorithms: "sha256", "sha512", "sha1", "md5"
- Returns hex-encoded hash
- Input size limit: 10 MB
#### crypto.hmac(algorithm, key, data) -> string
- HMAC using specified algorithm
- Returns hex-encoded result
- Key and data limits same as hash
### 3. Secure math.random
Replace Lua's `math.random` and remove `math.randomseed`:
```lua
-- After registration:
math.random() -- Returns double in [0.0, 1.0) using CSPRNG
math.random(n) -- Returns integer in [1, n]
math.random(m, n) -- Returns integer in [m, n]
math.randomseed -- Removed (nil)
```
---
## Test Cases
### Test 1: JSON Decode Valid
```cpp
bool Test_JsonDecodeValid(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterJsonAPI(sandbox.GetState());
std::string script = R"(
local obj = json.decode('{"name":"test","value":42,"arr":[1,2,3]}')
assert(obj.name == "test")
assert(obj.value == 42)
assert(#obj.arr == 3)
)";
if (!sandbox.LoadString(script, "decode_test")) {
error_msg = sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 2: JSON Decode Rejects Deep Nesting
```cpp
bool Test_JsonDecodeRejectsDeep(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::JsonLimits limits;
limits.max_depth = 5;
mosis::RegisterJsonAPI(sandbox.GetState(), limits);
// Create deeply nested JSON (10 levels)
std::string deep_json = "[[[[[[[[[[1]]]]]]]]]]";
std::string script =
"local result, err = json.decode('" + deep_json + "')\n"
"assert(result == nil, 'should fail')\n"
"assert(err:find('depth'), 'should mention depth')";
if (!sandbox.LoadString(script, "deep_test")) {
error_msg = sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 3: JSON Encode Detects Cycles
```cpp
bool Test_JsonEncodeDetectsCycles(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterJsonAPI(sandbox.GetState());
std::string script = R"(
local t = {a = 1}
t.self = t -- Create cycle
local result, err = json.encode(t)
assert(result == nil, 'should fail on cycle')
assert(err:find('cycle') or err:find('circular'), 'should mention cycle')
)";
if (!sandbox.LoadString(script, "cycle_test")) {
error_msg = sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 4: JSON Encode Valid
```cpp
bool Test_JsonEncodeValid(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterJsonAPI(sandbox.GetState());
std::string script = R"(
local str = json.encode({name = "test", value = 42, arr = {1, 2, 3}})
assert(type(str) == "string")
-- Decode back to verify
local obj = json.decode(str)
assert(obj.name == "test")
)";
if (!sandbox.LoadString(script, "encode_test")) {
error_msg = sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 5: Crypto RandomBytes
```cpp
bool Test_CryptoRandomBytes(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterCryptoAPI(sandbox.GetState());
std::string script = R"(
local bytes = crypto.randomBytes(16)
assert(#bytes == 16, 'should be 16 bytes')
-- Should be different each time
local bytes2 = crypto.randomBytes(16)
assert(bytes ~= bytes2, 'should be random')
)";
if (!sandbox.LoadString(script, "random_test")) {
error_msg = sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 6: Crypto Hash SHA256
```cpp
bool Test_CryptoHashSHA256(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterCryptoAPI(sandbox.GetState());
std::string script = R"(
local hash = crypto.hash("sha256", "hello")
-- Known SHA256 of "hello"
assert(hash == "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824")
)";
if (!sandbox.LoadString(script, "hash_test")) {
error_msg = sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 7: Crypto HMAC
```cpp
bool Test_CryptoHMAC(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::RegisterCryptoAPI(sandbox.GetState());
std::string script = R"(
local hmac = crypto.hmac("sha256", "key", "message")
-- Known HMAC-SHA256 of "message" with key "key"
assert(hmac == "6e9ef29b75fffc5b7abae527d58fdadb2fe42e7219011976917343065f58ed4a")
)";
if (!sandbox.LoadString(script, "hmac_test")) {
error_msg = sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 8: Secure Math.Random
```cpp
bool Test_SecureMathRandom(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::SecureRandom rng;
mosis::RegisterSecureMathRandom(sandbox.GetState(), &rng);
std::string script = R"(
-- math.randomseed should be removed
assert(math.randomseed == nil, 'randomseed should be removed')
-- math.random should work
local r1 = math.random()
assert(r1 >= 0 and r1 < 1, 'random() should return [0,1)')
local r2 = math.random(10)
assert(r2 >= 1 and r2 <= 10, 'random(n) should return [1,n]')
local r3 = math.random(5, 15)
assert(r3 >= 5 and r3 <= 15, 'random(m,n) should return [m,n]')
)";
if (!sandbox.LoadString(script, "math_random_test")) {
error_msg = sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 9: JSON Rejects Too Large
```cpp
bool Test_JsonRejectsTooLarge(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::JsonLimits limits;
limits.max_array_size = 10;
mosis::RegisterJsonAPI(sandbox.GetState(), limits);
std::string script = R"(
-- Try to decode array with 20 elements
local result, err = json.decode('[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]')
assert(result == nil, 'should fail')
assert(err:find('size') or err:find('limit'), 'should mention size limit')
)";
if (!sandbox.LoadString(script, "size_test")) {
error_msg = sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests pass:
- [x] `Test_JsonDecodeValid` - Decodes valid JSON to Lua table
- [x] `Test_JsonDecodeRejectsDeep` - Rejects deeply nested JSON
- [x] `Test_JsonEncodeValid` - Encodes Lua table to JSON string
- [x] `Test_JsonEncodeDetectsCycles` - Detects circular references
- [x] `Test_JsonRejectsTooLarge` - Enforces size limits
- [x] `Test_CryptoRandomBytes` - Generates secure random bytes
- [x] `Test_CryptoHashSHA256` - Computes correct SHA256 hash
- [x] `Test_CryptoHMAC` - Computes correct HMAC
- [x] `Test_SecureMathRandom` - Replaces math.random securely
---
## Dependencies
- nlohmann-json (already in vcpkg for sandbox-test)
- OpenSSL or platform crypto APIs for SHA256/HMAC (or header-only implementation)
---
## Next Steps
After Milestone 6 passes:
1. Milestone 7: Virtual Filesystem
2. Milestone 8: SQLite Database

440
docs/SANDBOX_MILESTONE_7.md Normal file
View File

@@ -0,0 +1,440 @@
# Milestone 7: Virtual Filesystem
**Status**: ✅ Complete
**Goal**: Per-app isolated storage with quotas.
---
## Overview
This milestone implements a sandboxed virtual filesystem for Lua apps:
- Per-app isolated storage directories
- Path validation to prevent escapes
- Quota enforcement to limit disk usage
- Session-only temp storage that cleans up automatically
### Key Deliverables
1. **VirtualFS class** - Path mapping and validation
2. **Lua fs API** - read, write, append, delete, exists, list, mkdir, stat
3. **Quota tracking** - Per-app disk usage limits
4. **Temp cleanup** - Clear temp files on app stop
---
## File Structure
```
src/main/cpp/sandbox/
├── virtual_fs.h # NEW - VirtualFS header
└── virtual_fs.cpp # NEW - VirtualFS implementation
```
---
## Implementation Details
### 1. Virtual Path Mapping
```
Virtual Path → Physical Path
─────────────────────────────────────────────────────────────
/data/ → <app_root>/data/ (persistent app data)
/cache/ → <app_root>/cache/ (clearable cache)
/temp/ → <app_root>/temp/ (session-only, auto-cleared)
/shared/ → <shared_root>/ (requires storage.shared permission)
```
For testing, `app_root` is configurable (e.g., `./test_apps/<app_id>/`).
### 2. VirtualFS Class
```cpp
// virtual_fs.h
#pragma once
#include <string>
#include <vector>
#include <cstdint>
#include <optional>
#include <functional>
struct lua_State;
namespace mosis {
struct FileStat {
size_t size;
int64_t modified; // Unix timestamp
bool is_dir;
};
struct VirtualFSLimits {
size_t max_quota_bytes = 50 * 1024 * 1024; // 50 MB per app
size_t max_file_size = 10 * 1024 * 1024; // 10 MB per file
int max_path_depth = 10; // Max directory depth
size_t max_path_length = 256; // Max path string length
};
class VirtualFS {
public:
VirtualFS(const std::string& app_id,
const std::string& app_root,
const VirtualFSLimits& limits = VirtualFSLimits{});
~VirtualFS();
// Path operations
bool ValidatePath(const std::string& virtual_path, std::string& error);
std::string ResolvePath(const std::string& virtual_path);
// File operations
std::optional<std::string> Read(const std::string& path, std::string& error);
bool Write(const std::string& path, const std::string& data, std::string& error);
bool Append(const std::string& path, const std::string& data, std::string& error);
bool Delete(const std::string& path, std::string& error);
bool Exists(const std::string& path);
std::optional<std::vector<std::string>> List(const std::string& path, std::string& error);
bool MakeDir(const std::string& path, std::string& error);
std::optional<FileStat> Stat(const std::string& path, std::string& error);
// Quota management
size_t GetUsedBytes() const { return m_used_bytes; }
size_t GetQuotaBytes() const { return m_limits.max_quota_bytes; }
void RecalculateUsage();
// Cleanup
void ClearTemp();
void ClearAll(); // For testing
// Permission check callback (set by sandbox)
std::function<bool(const std::string&)> CheckPermission;
private:
std::string m_app_id;
std::string m_app_root;
VirtualFSLimits m_limits;
size_t m_used_bytes = 0;
bool EnsureParentDir(const std::string& path);
void UpdateUsage(int64_t delta);
bool CheckQuota(size_t additional_bytes, std::string& error);
int GetPathDepth(const std::string& path);
};
// Register fs.* APIs as globals
void RegisterVirtualFS(lua_State* L, VirtualFS* vfs);
} // namespace mosis
```
### 3. Path Validation Rules
1. **Must start with virtual root**: `/data/`, `/cache/`, `/temp/`, or `/shared/`
2. **No path traversal**: Reject `..` components
3. **No absolute escapes**: After prefix, must stay within sandbox
4. **Max depth**: Limit directory nesting (default 10)
5. **Max length**: Limit path string length (default 256 chars)
6. **Valid characters**: Alphanumeric, `-`, `_`, `.`, `/`
### 4. Lua API
```lua
-- Read file contents
local content, err = fs.read("/data/config.json")
if not content then
print("Error: " .. err)
end
-- Write file (creates parent dirs)
local ok, err = fs.write("/data/config.json", '{"key": "value"}')
-- Append to file
fs.append("/data/log.txt", "New line\n")
-- Delete file or empty directory
fs.delete("/data/old_file.txt")
-- Check existence
if fs.exists("/data/config.json") then
-- file exists
end
-- List directory contents
local files, err = fs.list("/data/")
for _, name in ipairs(files) do
print(name)
end
-- Create directory
fs.mkdir("/data/subdir")
-- Get file info
local stat, err = fs.stat("/data/config.json")
if stat then
print("Size: " .. stat.size)
print("Modified: " .. stat.modified)
print("Is dir: " .. tostring(stat.isDir))
end
```
### 5. Quota Enforcement
- Track total bytes used per app
- Check before each write/append
- Return error if quota would be exceeded
- Recalculate on startup (in case of crashes)
### 6. Permission Requirements
| Operation | Permission Required |
|-----------|---------------------|
| `/data/*` | None (auto-granted) |
| `/cache/*` | None (auto-granted) |
| `/temp/*` | None (auto-granted) |
| `/shared/*` | `storage.shared` |
---
## Test Cases
### Test 1: Read/Write in App Dir
```cpp
bool Test_VirtualFSReadWrite(std::string& error_msg) {
std::string test_root = "test_vfs_app";
mosis::VirtualFS vfs("test.app", test_root);
std::string err;
// Write a file
EXPECT_TRUE(vfs.Write("/data/test.txt", "Hello World", err));
// Read it back
auto content = vfs.Read("/data/test.txt", err);
EXPECT_TRUE(content.has_value());
EXPECT_TRUE(*content == "Hello World");
// Cleanup
vfs.ClearAll();
return true;
}
```
### Test 2: Blocks Path Traversal
```cpp
bool Test_VirtualFSBlocksTraversal(std::string& error_msg) {
std::string test_root = "test_vfs_app";
mosis::VirtualFS vfs("test.app", test_root);
std::string err;
// Should reject traversal
EXPECT_FALSE(vfs.Write("/data/../../../etc/passwd", "hack", err));
EXPECT_TRUE(err.find("traversal") != std::string::npos ||
err.find("invalid") != std::string::npos);
// Should reject absolute paths
EXPECT_FALSE(vfs.Write("/etc/passwd", "hack", err));
vfs.ClearAll();
return true;
}
```
### Test 3: Enforces Quota
```cpp
bool Test_VirtualFSEnforcesQuota(std::string& error_msg) {
std::string test_root = "test_vfs_app";
mosis::VirtualFSLimits limits;
limits.max_quota_bytes = 1024; // 1 KB quota for testing
mosis::VirtualFS vfs("test.app", test_root, limits);
std::string err;
// Write should succeed
std::string small_data(500, 'a');
EXPECT_TRUE(vfs.Write("/data/file1.txt", small_data, err));
// Second write should fail (exceeds quota)
std::string large_data(600, 'b');
EXPECT_FALSE(vfs.Write("/data/file2.txt", large_data, err));
EXPECT_TRUE(err.find("quota") != std::string::npos);
vfs.ClearAll();
return true;
}
```
### Test 4: Cleans Up Temp
```cpp
bool Test_VirtualFSCleansUpTemp(std::string& error_msg) {
std::string test_root = "test_vfs_app";
mosis::VirtualFS vfs("test.app", test_root);
std::string err;
// Write to temp
EXPECT_TRUE(vfs.Write("/temp/session.txt", "temp data", err));
EXPECT_TRUE(vfs.Exists("/temp/session.txt"));
// Clear temp
vfs.ClearTemp();
// Should be gone
EXPECT_FALSE(vfs.Exists("/temp/session.txt"));
vfs.ClearAll();
return true;
}
```
### Test 5: List Directory
```cpp
bool Test_VirtualFSList(std::string& error_msg) {
std::string test_root = "test_vfs_app";
mosis::VirtualFS vfs("test.app", test_root);
std::string err;
// Create some files
vfs.Write("/data/file1.txt", "content1", err);
vfs.Write("/data/file2.txt", "content2", err);
vfs.MakeDir("/data/subdir", err);
// List directory
auto files = vfs.List("/data/", err);
EXPECT_TRUE(files.has_value());
EXPECT_TRUE(files->size() == 3);
vfs.ClearAll();
return true;
}
```
### Test 6: File Stat
```cpp
bool Test_VirtualFSStat(std::string& error_msg) {
std::string test_root = "test_vfs_app";
mosis::VirtualFS vfs("test.app", test_root);
std::string err;
// Write a file
vfs.Write("/data/test.txt", "Hello", err);
// Get stat
auto stat = vfs.Stat("/data/test.txt", err);
EXPECT_TRUE(stat.has_value());
EXPECT_TRUE(stat->size == 5);
EXPECT_FALSE(stat->is_dir);
// Directory stat
vfs.MakeDir("/data/subdir", err);
auto dir_stat = vfs.Stat("/data/subdir", err);
EXPECT_TRUE(dir_stat.has_value());
EXPECT_TRUE(dir_stat->is_dir);
vfs.ClearAll();
return true;
}
```
### Test 7: Lua Integration
```cpp
bool Test_VirtualFSLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
std::string test_root = "test_vfs_lua";
mosis::VirtualFS vfs("test.app", test_root);
mosis::RegisterVirtualFS(sandbox.GetState(), &vfs);
std::string script = R"(
-- Write and read
local ok, err = fs.write("/data/test.txt", "Hello from Lua")
assert(ok, "write failed: " .. (err or ""))
local content, err = fs.read("/data/test.txt")
assert(content == "Hello from Lua", "content mismatch")
-- Check exists
assert(fs.exists("/data/test.txt"), "file should exist")
assert(not fs.exists("/data/nonexistent.txt"), "file should not exist")
-- Stat
local stat = fs.stat("/data/test.txt")
assert(stat.size == 14, "size should be 14")
assert(not stat.isDir, "should not be dir")
)";
bool ok = sandbox.LoadString(script, "vfs_test");
vfs.ClearAll();
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 8: Max File Size
```cpp
bool Test_VirtualFSMaxFileSize(std::string& error_msg) {
std::string test_root = "test_vfs_app";
mosis::VirtualFSLimits limits;
limits.max_file_size = 100; // 100 bytes max
limits.max_quota_bytes = 10000; // Large quota
mosis::VirtualFS vfs("test.app", test_root, limits);
std::string err;
// Small file should succeed
EXPECT_TRUE(vfs.Write("/data/small.txt", std::string(50, 'a'), err));
// Large file should fail
EXPECT_FALSE(vfs.Write("/data/large.txt", std::string(200, 'b'), err));
EXPECT_TRUE(err.find("size") != std::string::npos);
vfs.ClearAll();
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_VirtualFSReadWrite` - Basic read/write operations
- [x] `Test_VirtualFSBlocksTraversal` - Path traversal prevention
- [x] `Test_VirtualFSEnforcesQuota` - Quota enforcement
- [x] `Test_VirtualFSCleansUpTemp` - Temp directory cleanup
- [x] `Test_VirtualFSList` - Directory listing
- [x] `Test_VirtualFSStat` - File stat information
- [x] `Test_VirtualFSLuaIntegration` - Lua API integration
- [x] `Test_VirtualFSMaxFileSize` - File size limit enforcement
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 4 (Path validation - reuse ValidatePath logic)
- C++ filesystem library (`<filesystem>`)
---
## Next Steps
After Milestone 7 passes:
1. Milestone 8: SQLite Database
2. Milestone 9: Network - HTTP

476
docs/SANDBOX_MILESTONE_8.md Normal file
View File

@@ -0,0 +1,476 @@
# Milestone 8: SQLite Database
**Status**: Complete ✅
**Goal**: Per-app SQLite with injection prevention.
---
## Overview
This milestone implements a sandboxed SQLite database for Lua apps:
- Per-app isolated database files
- SQL injection prevention via prepared statements
- SQLite authorizer to block dangerous operations
- Database size limits
### Key Deliverables
1. **DatabaseManager class** - SQLite wrapper with security
2. **Lua db API** - open, execute, query, close
3. **SQL authorizer** - Block ATTACH, DETACH, dangerous PRAGMAs
4. **Prepared statements** - Parameterized queries only
---
## File Structure
```
src/main/cpp/sandbox/
├── database_manager.h # NEW - DatabaseManager header
└── database_manager.cpp # NEW - DatabaseManager implementation
```
---
## Implementation Details
### 1. Database Path Mapping
```
Database Name → Physical Path
─────────────────────────────────────────────────────────────
"mydata" → <app_root>/db/mydata.db
"cache" → <app_root>/db/cache.db
```
For testing, `app_root` is configurable (e.g., `./test_apps/<app_id>/`).
### 2. DatabaseManager Class
```cpp
// database_manager.h
#pragma once
#include <string>
#include <vector>
#include <variant>
#include <optional>
#include <memory>
struct sqlite3;
struct lua_State;
namespace mosis {
// SQL value types
using SqlValue = std::variant<std::nullptr_t, int64_t, double, std::string, std::vector<uint8_t>>;
using SqlRow = std::vector<SqlValue>;
using SqlResult = std::vector<SqlRow>;
struct DatabaseLimits {
size_t max_database_size = 50 * 1024 * 1024; // 50 MB per database
int max_databases_per_app = 5; // Max open databases
int max_query_time_ms = 5000; // 5 second query timeout
int max_result_rows = 10000; // Max rows returned
};
class DatabaseHandle;
class DatabaseManager {
public:
DatabaseManager(const std::string& app_id,
const std::string& app_root,
const DatabaseLimits& limits = DatabaseLimits{});
~DatabaseManager();
// Database operations
std::shared_ptr<DatabaseHandle> Open(const std::string& name, std::string& error);
void CloseAll();
// Stats
size_t GetOpenDatabaseCount() const;
private:
std::string m_app_id;
std::string m_app_root;
DatabaseLimits m_limits;
std::vector<std::shared_ptr<DatabaseHandle>> m_databases;
std::string ResolvePath(const std::string& name);
bool ValidateName(const std::string& name, std::string& error);
};
class DatabaseHandle {
public:
DatabaseHandle(sqlite3* db, const std::string& path, const DatabaseLimits& limits);
~DatabaseHandle();
// Execute (INSERT, UPDATE, DELETE, CREATE, etc.)
bool Execute(const std::string& sql, const std::vector<SqlValue>& params, std::string& error);
// Query (SELECT)
std::optional<SqlResult> Query(const std::string& sql, const std::vector<SqlValue>& params,
std::string& error);
// Get last insert rowid
int64_t GetLastInsertRowId() const;
// Get affected rows
int GetChanges() const;
bool IsOpen() const { return m_db != nullptr; }
void Close();
private:
sqlite3* m_db;
std::string m_path;
DatabaseLimits m_limits;
static int Authorizer(void* user_data, int action, const char* arg1,
const char* arg2, const char* arg3, const char* arg4);
};
// Register db.* APIs as globals
void RegisterDatabaseAPI(lua_State* L, DatabaseManager* manager);
} // namespace mosis
```
### 3. SQLite Authorizer Rules
Block dangerous operations:
| Action | Decision | Reason |
|--------|----------|--------|
| ATTACH | DENY | Prevents accessing other databases |
| DETACH | DENY | Prevents detaching |
| PRAGMA (most) | DENY | Many PRAGMAs are dangerous |
| PRAGMA table_info | ALLOW | Safe introspection |
| PRAGMA index_list | ALLOW | Safe introspection |
| PRAGMA foreign_keys | ALLOW | Safe setting |
| Function (load_extension) | DENY | Prevents loading native code |
### 4. Lua API
```lua
-- Open database (creates if not exists)
local db, err = database.open("mydata")
if not db then
print("Error: " .. err)
return
end
-- Create table
local ok, err = db:execute([[
CREATE TABLE IF NOT EXISTS items (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
price REAL
)
]])
-- Insert with parameters (prevents SQL injection)
db:execute("INSERT INTO items (name, price) VALUES (?, ?)", {"Widget", 19.99})
-- Query with parameters
local rows, err = db:query("SELECT * FROM items WHERE price > ?", {10.0})
if rows then
for _, row in ipairs(rows) do
print(row[1], row[2], row[3]) -- id, name, price
end
end
-- Get last insert ID
local id = db:lastInsertId()
-- Get affected rows
local changes = db:changes()
-- Close database
db:close()
```
### 5. Security Features
1. **Prepared Statements**: All queries use parameterized binding
2. **Authorizer**: Block dangerous SQL operations
3. **Path Sandboxing**: Database files only in app's db/ directory
4. **Size Limits**: Enforce max database size
5. **Query Timeout**: Prevent long-running queries
6. **Result Limits**: Cap number of returned rows
---
## Test Cases
### Test 1: Create and Query Tables
```cpp
bool Test_DatabaseCreatesTables(std::string& error_msg) {
std::string test_root = "test_db_app";
mosis::DatabaseManager manager("test.app", test_root);
std::string err;
auto db = manager.Open("test", err);
EXPECT_TRUE(db != nullptr);
// Create table
EXPECT_TRUE(db->Execute(
"CREATE TABLE IF NOT EXISTS items (id INTEGER PRIMARY KEY, name TEXT)",
{}, err));
// Insert
EXPECT_TRUE(db->Execute(
"INSERT INTO items (name) VALUES (?)",
{std::string("Test Item")}, err));
// Query
auto rows = db->Query("SELECT * FROM items", {}, err);
EXPECT_TRUE(rows.has_value());
EXPECT_TRUE(rows->size() == 1);
db->Close();
manager.CloseAll();
// Cleanup
std::filesystem::remove_all(test_root);
return true;
}
```
### Test 2: Prepared Statements Prevent Injection
```cpp
bool Test_DatabasePreparedStatements(std::string& error_msg) {
std::string test_root = "test_db_app";
mosis::DatabaseManager manager("test.app", test_root);
std::string err;
auto db = manager.Open("test", err);
EXPECT_TRUE(db != nullptr);
db->Execute("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)", {}, err);
db->Execute("INSERT INTO users (name) VALUES (?)", {std::string("Alice")}, err);
// Attempt SQL injection via parameter - should be safely escaped
std::string malicious = "'; DROP TABLE users; --";
auto rows = db->Query("SELECT * FROM users WHERE name = ?", {malicious}, err);
// Query should succeed (finding nothing) and table should still exist
EXPECT_TRUE(rows.has_value());
EXPECT_TRUE(rows->size() == 0);
// Table should still exist
auto check = db->Query("SELECT * FROM users", {}, err);
EXPECT_TRUE(check.has_value());
EXPECT_TRUE(check->size() == 1);
db->Close();
std::filesystem::remove_all(test_root);
return true;
}
```
### Test 3: Blocks ATTACH
```cpp
bool Test_DatabaseBlocksAttach(std::string& error_msg) {
std::string test_root = "test_db_app";
mosis::DatabaseManager manager("test.app", test_root);
std::string err;
auto db = manager.Open("test", err);
EXPECT_TRUE(db != nullptr);
// Try to attach another database - should fail
EXPECT_FALSE(db->Execute("ATTACH DATABASE '/etc/passwd' AS evil", {}, err));
EXPECT_TRUE(err.find("authorized") != std::string::npos ||
err.find("denied") != std::string::npos);
db->Close();
std::filesystem::remove_all(test_root);
return true;
}
```
### Test 4: Blocks Dangerous PRAGMA
```cpp
bool Test_DatabaseBlocksDangerousPragma(std::string& error_msg) {
std::string test_root = "test_db_app";
mosis::DatabaseManager manager("test.app", test_root);
std::string err;
auto db = manager.Open("test", err);
EXPECT_TRUE(db != nullptr);
// Try dangerous PRAGMAs - should fail
EXPECT_FALSE(db->Execute("PRAGMA journal_mode = OFF", {}, err));
EXPECT_FALSE(db->Execute("PRAGMA synchronous = OFF", {}, err));
// Safe PRAGMAs should work
auto rows = db->Query("PRAGMA table_info(sqlite_master)", {}, err);
EXPECT_TRUE(rows.has_value());
db->Close();
std::filesystem::remove_all(test_root);
return true;
}
```
### Test 5: Multiple Databases
```cpp
bool Test_DatabaseMultiple(std::string& error_msg) {
std::string test_root = "test_db_app";
mosis::DatabaseManager manager("test.app", test_root);
std::string err;
auto db1 = manager.Open("db1", err);
auto db2 = manager.Open("db2", err);
EXPECT_TRUE(db1 != nullptr);
EXPECT_TRUE(db2 != nullptr);
EXPECT_TRUE(manager.GetOpenDatabaseCount() == 2);
// They should be independent
db1->Execute("CREATE TABLE t1 (x INTEGER)", {}, err);
db2->Execute("CREATE TABLE t2 (y INTEGER)", {}, err);
// t1 shouldn't exist in db2
auto rows = db2->Query("SELECT * FROM t1", {}, err);
EXPECT_FALSE(rows.has_value()); // Should fail - table doesn't exist
manager.CloseAll();
std::filesystem::remove_all(test_root);
return true;
}
```
### Test 6: Lua Integration
```cpp
bool Test_DatabaseLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
std::string test_root = "test_db_lua";
mosis::DatabaseManager manager("test.app", test_root);
mosis::RegisterDatabaseAPI(sandbox.GetState(), &manager);
std::string script = R"(
local db, err = database.open("test")
assert(db, "failed to open: " .. (err or ""))
-- Create table
local ok, err = db:execute("CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)")
assert(ok, "create failed: " .. (err or ""))
-- Insert
ok, err = db:execute("INSERT INTO items (name) VALUES (?)", {"Test"})
assert(ok, "insert failed: " .. (err or ""))
-- Query
local rows, err = db:query("SELECT * FROM items")
assert(rows, "query failed: " .. (err or ""))
assert(#rows == 1, "expected 1 row")
db:close()
)";
bool ok = sandbox.LoadString(script, "db_test");
manager.CloseAll();
std::filesystem::remove_all(test_root);
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
### Test 7: Invalid Database Names
```cpp
bool Test_DatabaseInvalidNames(std::string& error_msg) {
std::string test_root = "test_db_app";
mosis::DatabaseManager manager("test.app", test_root);
std::string err;
// Path traversal
auto db1 = manager.Open("../evil", err);
EXPECT_TRUE(db1 == nullptr);
// Absolute path
auto db2 = manager.Open("/etc/passwd", err);
EXPECT_TRUE(db2 == nullptr);
// Special characters
auto db3 = manager.Open("test;drop", err);
EXPECT_TRUE(db3 == nullptr);
std::filesystem::remove_all(test_root);
return true;
}
```
### Test 8: Last Insert ID and Changes
```cpp
bool Test_DatabaseLastInsertAndChanges(std::string& error_msg) {
std::string test_root = "test_db_app";
mosis::DatabaseManager manager("test.app", test_root);
std::string err;
auto db = manager.Open("test", err);
EXPECT_TRUE(db != nullptr);
db->Execute("CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)", {}, err);
db->Execute("INSERT INTO items (name) VALUES (?)", {std::string("Item 1")}, err);
EXPECT_TRUE(db->GetLastInsertRowId() == 1);
EXPECT_TRUE(db->GetChanges() == 1);
db->Execute("INSERT INTO items (name) VALUES (?)", {std::string("Item 2")}, err);
EXPECT_TRUE(db->GetLastInsertRowId() == 2);
db->Execute("UPDATE items SET name = ?", {std::string("Updated")}, err);
EXPECT_TRUE(db->GetChanges() == 2); // Updated 2 rows
db->Close();
std::filesystem::remove_all(test_root);
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_DatabaseCreatesTables` - Basic create/insert/query
- [x] `Test_DatabasePreparedStatements` - SQL injection prevention
- [x] `Test_DatabaseBlocksAttach` - ATTACH blocked
- [x] `Test_DatabaseBlocksDangerousPragma` - Dangerous PRAGMAs blocked
- [x] `Test_DatabaseMultiple` - Multiple isolated databases
- [x] `Test_DatabaseLuaIntegration` - Lua API works
- [x] `Test_DatabaseInvalidNames` - Path validation
- [x] `Test_DatabaseLastInsertAndChanges` - Metadata functions
---
## Dependencies
- Milestone 1 (LuaSandbox)
- SQLite3 library (add to vcpkg.json)
---
## Next Steps
After Milestone 8 passes:
1. Milestone 9: Network - HTTP
2. Milestone 10: Network - WebSocket

521
docs/SANDBOX_MILESTONE_9.md Normal file
View File

@@ -0,0 +1,521 @@
# Milestone 9: Network - HTTP
**Status**: Complete
**Goal**: Secure HTTP requests with domain filtering and SSRF prevention.
---
## Overview
This milestone implements secure HTTP networking for Lua apps:
- HTTPS required (no plain HTTP)
- SSRF prevention (block private IPs, localhost, metadata endpoints)
- Domain whitelist from app manifest
- Request/response size limits
- Concurrent request limits
### Key Deliverables
1. **HttpValidator class** - URL parsing, domain validation, IP blocking
2. **NetworkManager class** - HTTP client with limits
3. **Lua network API** - `network.request()` function
4. **Desktop mock server** - For testing without network
---
## File Structure
```
src/main/cpp/sandbox/
├── http_validator.h # NEW - URL validation
├── http_validator.cpp # NEW - SSRF prevention
├── network_manager.h # NEW - HTTP client wrapper
└── network_manager.cpp # NEW - Request execution
```
---
## Implementation Details
### 1. URL Validation Rules
```
Allowed:
https://api.example.com/path
https://192.0.2.1/api (public IP)
Blocked:
http://example.com/... # No plain HTTP
https://127.0.0.1/... # Localhost
https://localhost/... # Localhost name
https://10.0.0.1/... # Private IP (10.x.x.x)
https://192.168.1.1/... # Private IP (192.168.x.x)
https://172.16.0.1/... # Private IP (172.16-31.x.x)
https://169.254.169.254/... # AWS metadata
https://[::1]/... # IPv6 localhost
https://[fc00::1]/... # IPv6 private
https://0.0.0.0/... # All interfaces
file:///etc/passwd # File scheme
ftp://ftp.example.com/... # Non-HTTP schemes
```
### 2. HttpValidator Class
```cpp
// http_validator.h
#pragma once
#include <string>
#include <vector>
#include <optional>
namespace mosis {
struct ParsedUrl {
std::string scheme; // "https"
std::string host; // "api.example.com" or "192.0.2.1"
uint16_t port; // 443
std::string path; // "/api/data"
std::string query; // "?key=value"
bool is_ip_address; // true if host is IP
};
class HttpValidator {
public:
HttpValidator();
// Set allowed domains (from app manifest)
void SetAllowedDomains(const std::vector<std::string>& domains);
// Clear domain restrictions (for testing)
void ClearDomainRestrictions();
// Validate URL
// Returns parsed URL on success, sets error on failure
std::optional<ParsedUrl> Validate(const std::string& url, std::string& error);
private:
std::vector<std::string> m_allowed_domains;
bool m_domain_restrictions_enabled;
bool IsPrivateIP(const std::string& ip);
bool IsLocalhostIP(const std::string& ip);
bool IsMetadataIP(const std::string& ip);
bool IsDomainAllowed(const std::string& host);
std::optional<ParsedUrl> ParseUrl(const std::string& url);
};
} // namespace mosis
```
### 3. NetworkManager Class
```cpp
// network_manager.h
#pragma once
#include <string>
#include <vector>
#include <map>
#include <functional>
#include <mutex>
#include <atomic>
#include "http_validator.h"
struct lua_State;
namespace mosis {
struct HttpRequest {
std::string url;
std::string method = "GET";
std::map<std::string, std::string> headers;
std::string body;
int timeout_ms = 30000;
};
struct HttpResponse {
int status_code = 0;
std::map<std::string, std::string> headers;
std::string body;
std::string error;
};
struct NetworkLimits {
size_t max_request_body = 10 * 1024 * 1024; // 10 MB
size_t max_response_body = 50 * 1024 * 1024; // 50 MB
int max_timeout_ms = 60000; // 60 seconds
int max_concurrent_requests = 6;
int default_timeout_ms = 30000;
};
class NetworkManager {
public:
NetworkManager(const std::string& app_id, const NetworkLimits& limits = NetworkLimits{});
~NetworkManager();
// Configure domain restrictions
void SetAllowedDomains(const std::vector<std::string>& domains);
void ClearDomainRestrictions();
// Synchronous request (for testing)
HttpResponse Request(const HttpRequest& request, std::string& error);
// Stats
int GetActiveRequestCount() const;
// Get validator for testing
HttpValidator& GetValidator() { return m_validator; }
private:
std::string m_app_id;
NetworkLimits m_limits;
HttpValidator m_validator;
std::atomic<int> m_active_requests{0};
std::mutex m_mutex;
HttpResponse ExecuteRequest(const ParsedUrl& parsed, const HttpRequest& request);
};
// Register network.* APIs as globals
void RegisterNetworkAPI(lua_State* L, NetworkManager* manager);
} // namespace mosis
```
### 4. Private IP Ranges (SSRF Prevention)
| Range | Description |
|-------|-------------|
| `127.0.0.0/8` | Loopback |
| `10.0.0.0/8` | Private Class A |
| `172.16.0.0/12` | Private Class B |
| `192.168.0.0/16` | Private Class C |
| `169.254.0.0/16` | Link-local |
| `169.254.169.254` | Cloud metadata |
| `0.0.0.0` | All interfaces |
| `::1` | IPv6 loopback |
| `fc00::/7` | IPv6 private |
| `fe80::/10` | IPv6 link-local |
### 5. Lua API
```lua
-- Simple GET request
local response, err = network.request({
url = "https://api.example.com/data"
})
if not response then
print("Error: " .. err)
return
end
print("Status:", response.status)
print("Body:", response.body)
-- POST with headers and body
local response = network.request({
url = "https://api.example.com/submit",
method = "POST",
headers = {
["Content-Type"] = "application/json",
["Authorization"] = "Bearer token123"
},
body = '{"key": "value"}',
timeout = 30000 -- 30 seconds
})
-- Response structure
-- response.status - HTTP status code (200, 404, etc.)
-- response.headers - Response headers table
-- response.body - Response body string
-- response.error - Error message (if request failed)
```
### 6. Desktop Implementation
For desktop testing, we use cpp-httplib (header-only HTTP client).
On Android, we would use HttpURLConnection through JNI, but for the sandbox tests,
we implement a mock HTTP server that returns predefined responses.
### 7. Mock Server for Testing
```cpp
// Test helper - mock HTTP responses
class MockHttpServer {
public:
void AddResponse(const std::string& url, const HttpResponse& response);
std::optional<HttpResponse> GetResponse(const std::string& url);
void Clear();
private:
std::map<std::string, HttpResponse> m_responses;
};
```
---
## Test Cases
### Test 1: Block Private IPs
```cpp
bool Test_NetworkBlocksPrivateIP(std::string& error_msg) {
mosis::NetworkManager manager("test.app");
manager.ClearDomainRestrictions();
std::string err;
// All these should be blocked
std::vector<std::string> private_urls = {
"https://127.0.0.1/api",
"https://10.0.0.1/api",
"https://192.168.1.1/api",
"https://172.16.0.1/api",
"https://169.254.169.254/latest/meta-data/",
"https://localhost/api",
"https://0.0.0.0/api"
};
for (const auto& url : private_urls) {
mosis::HttpRequest req;
req.url = url;
auto response = manager.Request(req, err);
EXPECT_TRUE(response.status_code == 0); // Request should fail
EXPECT_TRUE(err.find("blocked") != std::string::npos ||
err.find("private") != std::string::npos ||
err.find("localhost") != std::string::npos);
}
return true;
}
```
### Test 2: Block Plain HTTP
```cpp
bool Test_NetworkBlocksPlainHttp(std::string& error_msg) {
mosis::NetworkManager manager("test.app");
manager.ClearDomainRestrictions();
std::string err;
mosis::HttpRequest req;
req.url = "http://example.com/api"; // No HTTPS
auto response = manager.Request(req, err);
EXPECT_TRUE(response.status_code == 0);
EXPECT_TRUE(err.find("HTTPS") != std::string::npos);
return true;
}
```
### Test 3: Require HTTPS
```cpp
bool Test_NetworkRequiresHttps(std::string& error_msg) {
mosis::NetworkManager manager("test.app");
manager.ClearDomainRestrictions();
// HTTPS URL with mock validator should parse successfully
std::string err;
auto parsed = manager.GetValidator().Validate("https://example.com/api", err);
EXPECT_TRUE(parsed.has_value());
EXPECT_TRUE(parsed->scheme == "https");
// HTTP should fail validation
parsed = manager.GetValidator().Validate("http://example.com/api", err);
EXPECT_FALSE(parsed.has_value());
return true;
}
```
### Test 4: Domain Whitelist
```cpp
bool Test_NetworkEnforcesDomainWhitelist(std::string& error_msg) {
mosis::NetworkManager manager("test.app");
// Set allowed domains
manager.SetAllowedDomains({"api.example.com", "cdn.example.com"});
std::string err;
// Allowed domain should validate
auto parsed = manager.GetValidator().Validate("https://api.example.com/data", err);
EXPECT_TRUE(parsed.has_value());
// Disallowed domain should fail
parsed = manager.GetValidator().Validate("https://evil.com/steal", err);
EXPECT_FALSE(parsed.has_value());
EXPECT_TRUE(err.find("allowed") != std::string::npos ||
err.find("whitelist") != std::string::npos);
return true;
}
```
### Test 5: URL Parsing
```cpp
bool Test_NetworkUrlParsing(std::string& error_msg) {
mosis::HttpValidator validator;
std::string err;
// Full URL
auto parsed = validator.Validate("https://api.example.com:8443/path/to/resource?key=value", err);
EXPECT_TRUE(parsed.has_value());
EXPECT_TRUE(parsed->scheme == "https");
EXPECT_TRUE(parsed->host == "api.example.com");
EXPECT_TRUE(parsed->port == 8443);
EXPECT_TRUE(parsed->path == "/path/to/resource");
EXPECT_TRUE(parsed->query == "?key=value");
// Default port
parsed = validator.Validate("https://example.com/api", err);
EXPECT_TRUE(parsed.has_value());
EXPECT_TRUE(parsed->port == 443);
return true;
}
```
### Test 6: Block Metadata Endpoints
```cpp
bool Test_NetworkBlocksMetadata(std::string& error_msg) {
mosis::NetworkManager manager("test.app");
manager.ClearDomainRestrictions();
std::string err;
// AWS metadata
mosis::HttpRequest req;
req.url = "https://169.254.169.254/latest/meta-data/";
auto response = manager.Request(req, err);
EXPECT_TRUE(response.status_code == 0);
// GCP metadata
req.url = "https://metadata.google.internal/computeMetadata/v1/";
response = manager.Request(req, err);
EXPECT_TRUE(response.status_code == 0);
return true;
}
```
### Test 7: Request Limits
```cpp
bool Test_NetworkRequestLimits(std::string& error_msg) {
mosis::NetworkLimits limits;
limits.max_request_body = 1024; // 1 KB for testing
mosis::NetworkManager manager("test.app", limits);
manager.ClearDomainRestrictions();
std::string err;
mosis::HttpRequest req;
req.url = "https://example.com/api";
req.method = "POST";
req.body = std::string(2048, 'X'); // 2 KB - exceeds limit
auto response = manager.Request(req, err);
EXPECT_TRUE(response.status_code == 0);
EXPECT_TRUE(err.find("size") != std::string::npos ||
err.find("limit") != std::string::npos ||
err.find("large") != std::string::npos);
return true;
}
```
### Test 8: Lua Integration
```cpp
bool Test_NetworkLuaIntegration(std::string& error_msg) {
SandboxContext ctx = TestContext();
LuaSandbox sandbox(ctx);
mosis::NetworkManager manager("test.app");
manager.ClearDomainRestrictions();
mosis::RegisterNetworkAPI(sandbox.GetState(), &manager);
std::string script = R"lua(
-- Test that network global exists
if not network then
error("network global not found")
end
if not network.request then
error("network.request not found")
end
-- Test validation rejection (private IP)
local response, err = network.request({
url = "https://127.0.0.1/api"
})
if response then
error("expected private IP to be blocked")
end
)lua";
bool ok = sandbox.LoadString(script, "network_test");
if (!ok) {
error_msg = "Lua test failed: " + sandbox.GetLastError();
return false;
}
return true;
}
```
---
## Acceptance Criteria
All tests must pass:
- [x] `Test_NetworkBlocksPrivateIP` - Private IPs blocked
- [x] `Test_NetworkBlocksPlainHttp` - Plain HTTP rejected
- [x] `Test_NetworkRequiresHttps` - HTTPS required
- [x] `Test_NetworkEnforcesDomainWhitelist` - Domain restrictions work
- [x] `Test_NetworkUrlParsing` - URL parsing correct
- [x] `Test_NetworkBlocksMetadata` - Cloud metadata blocked
- [x] `Test_NetworkRequestLimits` - Size limits enforced
- [x] `Test_NetworkLuaIntegration` - Lua API works
---
## Dependencies
- Milestone 1 (LuaSandbox)
- Milestone 2 (PermissionGate)
- Milestone 3 (RateLimiter, AuditLog)
---
## Notes
### Desktop vs Android Implementation
For the sandbox-test project (desktop), we focus on URL validation and request
structure validation. Actual network requests would go through cpp-httplib or
a mock server.
On Android, the real implementation would:
1. Call through JNI to Java HttpURLConnection
2. Use Android's network security config
3. Integrate with the permission system
### Security Considerations
1. **DNS Rebinding**: In production, re-validate IP after DNS resolution
2. **Redirects**: Follow redirects only to allowed domains, max 5 redirects
3. **Content-Type**: Validate response content-type matches expected
4. **Timeouts**: Both connection and read timeouts
---
## Next Steps
After Milestone 9 passes:
1. Milestone 10: Network - WebSocket

60
docs/TESTING-FRAMEWORK.md Normal file
View File

@@ -0,0 +1,60 @@
# Automated Testing Framework
The designer-test (`designer-test/`) provides automated UI testing.
## Test Architecture
1. **WindowController**: Finds designer window, sends mouse/keyboard events via Windows SendInput API
2. **HierarchyReader**: Parses UI hierarchy JSON to find elements by ID/class (with retry logic and exponential backoff)
3. **LogParser**: Monitors log file for navigation events
4. **TestRunner**: Orchestrates test execution, reports results
5. **UIInspector**: Dumps UI hierarchy with atomic writes (temp file + rename pattern)
## Key Implementation Details
- **Path Normalization**: RmlUi uses `|` instead of `:` in Windows paths (e.g., `D|\Dev\...`). The UIInspector normalizes paths for correct document matching.
- **Atomic File Writes**: Hierarchy files are written to `.tmp` then renamed to prevent partial reads.
- **Retry with Backoff**: HierarchyReader retries up to 10 times with exponential backoff (30ms base) and validates JSON completeness.
- **Dynamic Back Button**: `GoHome()` finds back buttons from hierarchy by class (`app-bar-nav` or `browser-nav-btn`) instead of fixed coordinates.
## Writing Tests
```cpp
// Find element by ID and click it
bool ClickById(TestContext& ctx, const std::string& id) {
ctx.hierarchy.Reload();
auto element = ctx.hierarchy.FindById(id);
if (!element) return false;
int x = element->bounds.centerX();
int y = element->bounds.centerY();
ScaleToPhysical(ctx, x, y); // Convert logical to physical coords
ctx.window.SendClick(x, y);
return true;
}
// Test example
bool TestNavigateToDialer(TestContext& ctx) {
GoHome(ctx);
ctx.log.Clear();
if (!ClickById(ctx, "dock-phone")) return false;
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
ctx.log.Reload();
return ctx.log.Contains("Loaded screen: apps/dialer/dialer.rml");
}
```
## Test Output
Tests produce JSON results at `test_results.json`:
```json
{
"name": "Mosis Designer UI Tests",
"summary": {"passed": 5, "failed": 0, "total": 5},
"tests": [
{"name": "Navigate to Dialer", "status": "passed", "duration_ms": 3500}
]
}
```

534
docs/TESTING.md Normal file
View File

@@ -0,0 +1,534 @@
# Mosis UI Testing Framework
This document describes the automated UI testing framework for Mosis virtual smartphone.
## Overview
The testing framework enables automated validation of UI behavior through:
1. **UI Hierarchy Inspection**: JSON dump of all UI elements with bounds
2. **Input Simulation**: Mouse clicks via Windows SendInput API
3. **Log Verification**: Check navigation events via log file parsing
4. **Test Results**: JSON output compatible with CI/CD pipelines
5. **Action Recording/Playback**: Record and replay user interactions
6. **Visual Regression**: Screenshot comparison with pixel-level diff
## Components
### Designer (mosis-designer.exe)
The desktop designer serves as the test target. When launched with testing options:
```bash
mosis-designer.exe home.rml --log test.log --hierarchy hierarchy.json
```
**Testing Options**:
| Option | Description |
|--------|-------------|
| `--log FILE` | Write all RmlUi INFO logs to file (navigation events, errors) |
| `--hierarchy FILE` | Dump UI element tree to JSON each frame |
| `--record FILE` | Enable action recording mode (F5 to start/stop) |
| `--playback FILE` | Play back recorded actions from JSON file |
**Keyboard Controls**:
| Key | Function |
|-----|----------|
| F5 | Start/stop recording (when `--record` is enabled) |
| F6 | Pause/resume playback (when `--playback` is enabled) |
| F12 | Take screenshot (saves to current directory) |
### Test Runner (designer-test.exe)
Automated test executor that:
1. Launches designer with testing options
2. Waits for window to appear
3. Reads UI hierarchy to find elements
4. Sends input events to simulate user interaction
5. Verifies results via log file
6. Outputs JSON test results
## UI Hierarchy Format
The hierarchy JSON contains all visible UI elements:
```json
{
"timestamp": 1705312200,
"screen": "apps/home/home.rml",
"resolution": {"width": 677, "height": 1202},
"elements": {
"tag": "body",
"id": "",
"classes": ["home-screen"],
"bounds": {"x": 0, "y": 0, "width": 677, "height": 1202},
"visible": true,
"text": null,
"children": [
{
"tag": "div",
"id": "dock-phone",
"classes": ["dock-item"],
"bounds": {"x": 85, "y": 1138, "width": 56, "height": 56},
"visible": true,
"text": null,
"children": [...]
}
]
}
}
```
### Key Fields
| Field | Description |
|-------|-------------|
| `tag` | HTML element tag (div, span, img, input) |
| `id` | Element ID attribute (empty if none) |
| `classes` | Array of CSS class names |
| `bounds` | Position and size in logical pixels |
| `visible` | Whether element is displayed |
| `text` | Text content (for leaf text nodes) |
| `children` | Nested child elements |
### Resolution Note
The `resolution` field contains RmlUi's logical dimensions, which may differ from physical window size due to DPI scaling. Tests must scale coordinates:
```cpp
// Scale from hierarchy (logical) to window (physical)
int physicalX = logicalX * windowWidth / hierarchyWidth;
int physicalY = logicalY * windowHeight / hierarchyHeight;
```
## Writing Tests
### Test Function Signature
```cpp
bool MyTest(TestContext& ctx);
```
TestContext provides:
- `ctx.window` - WindowController for input
- `ctx.log` - LogParser for log verification
- `ctx.hierarchy` - HierarchyReader for element lookup
### Finding Elements
```cpp
// By ID
auto element = ctx.hierarchy.FindById("dock-phone");
// By class (returns all matches)
auto buttons = ctx.hierarchy.FindByClass("btn-icon");
// By tag
auto divs = ctx.hierarchy.FindByTag("div");
```
### Clicking Elements
```cpp
bool ClickById(TestContext& ctx, const std::string& id) {
ctx.hierarchy.Reload(); // Get fresh hierarchy
auto element = ctx.hierarchy.FindById(id);
if (!element || !element->visible) {
return false;
}
// Get center point
int x = element->bounds.centerX();
int y = element->bounds.centerY();
// Scale to physical coordinates
ScaleToPhysical(ctx, x, y);
// Send click
ctx.window.SendClick(x, y);
return true;
}
```
### Verifying Navigation
```cpp
bool TestNavigateToDialer(TestContext& ctx) {
GoHome(ctx); // Return to home screen
ctx.log.Clear(); // Clear previous logs
if (!ClickById(ctx, "dock-phone")) {
return false;
}
// Wait for navigation animation
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
// Check log for navigation event
ctx.log.Reload();
return ctx.log.Contains("Loaded screen: apps/dialer/dialer.rml");
}
```
### Navigation Helpers
```cpp
// Go back to home screen by clicking back buttons
void GoHome(TestContext& ctx) {
for (int i = 0; i < 5; ++i) {
ctx.hierarchy.Reload();
auto elements = ctx.hierarchy.FindByClass("app-bar-nav");
if (!elements.empty() && elements[0].visible) {
int x = elements[0].bounds.centerX();
int y = elements[0].bounds.centerY();
ScaleToPhysical(ctx, x, y);
ctx.window.SendClick(x, y);
}
std::this_thread::sleep_for(std::chrono::milliseconds(400));
}
std::this_thread::sleep_for(std::chrono::milliseconds(800));
}
```
## Registering Tests
In `main.cpp`:
```cpp
TestRunner runner;
runner.SetDesignerPath(designerPath);
runner.SetDocumentPath(documentPath);
runner.SetLogPath(logPath);
runner.SetHierarchyPath(hierarchyPath);
// Register tests
runner.AddTest("Navigate to Dialer", TestNavigateToDialer);
runner.AddTest("Navigate to Messages", TestNavigateToMessages);
runner.AddTest("Navigate to Contacts", TestNavigateToContacts);
runner.AddTest("Navigate to Browser", TestNavigateToBrowser);
runner.AddTest("Navigation Sequence", TestNavigationSequence);
// Run all tests
auto results = runner.RunAll();
results.SaveToFile(resultsPath);
```
## Test Results Format
```json
{
"name": "Mosis Designer UI Tests",
"summary": {
"passed": 5,
"failed": 0,
"skipped": 0,
"errors": 0,
"total": 5,
"duration_ms": 31162.3
},
"tests": [
{
"name": "Navigate to Dialer",
"status": "passed",
"message": "Test passed",
"duration_ms": 4216.88
},
{
"name": "Navigate to Browser",
"status": "passed",
"message": "Test passed",
"duration_ms": 4067.34
}
]
}
```
## Action Recording and Playback
The designer supports recording user interactions and playing them back for automated testing.
### Recording Actions
```bash
# Start designer with recording enabled
mosis-designer.exe home.rml --record my-test.json
# Press F5 to start recording
# Interact with the UI (clicks, swipes, etc.)
# Press F5 again to stop and save
```
Recording is automatically saved when you close the window.
### Playing Back Actions
```bash
# Play back a recorded test
mosis-designer.exe home.rml --playback my-test.json
```
Use F6 to pause/resume playback.
### Action Recording Format
```json
{
"name": "Navigate to contacts",
"description": "Test navigation flow",
"screen_width": 540,
"screen_height": 960,
"initial_screen": "apps/home/home.rml",
"actions": [
{"type": "tap", "x": 413, "y": 1174, "timestamp": 0},
{"type": "wait", "duration": 1000, "timestamp": 100},
{"type": "tap", "x": 40, "y": 28, "timestamp": 1100},
{"type": "swipe", "x1": 100, "y1": 500, "x2": 100, "y2": 200, "duration": 300, "timestamp": 2000}
]
}
```
### Supported Action Types
| Type | Fields | Description |
|------|--------|-------------|
| `tap` | x, y, timestamp | Single tap at coordinates |
| `swipe` | x1, y1, x2, y2, duration, timestamp | Swipe gesture |
| `long_press` | x, y, duration, timestamp | Long press gesture |
| `button` | button, timestamp | Hardware button ("back", "home") |
| `wait` | duration, timestamp | Pause between actions |
| `key` | key_code, pressed, timestamp | Keyboard input |
### Creating Test Files Manually
You can also create test files manually using the UI hierarchy to find element coordinates:
```bash
# Get element coordinates from hierarchy
mosis-designer.exe home.rml --hierarchy hierarchy.json
# Read hierarchy.json to find element bounds
# Write action JSON with those coordinates
```
## Screenshot Comparison
The testing framework includes pixel-level screenshot comparison for visual regression testing.
### Using Screenshot Comparison
```cpp
#include "testing/visual_capture.h"
// Capture a screenshot
mosis::testing::VisualCapture capture(540, 960);
capture.CaptureScreenshot("current.png");
// Compare two screenshots
float diff = mosis::testing::VisualCapture::CompareImages("baseline.png", "current.png");
// diff = 0.0 means identical
// diff = 1.0 means completely different
// Typical threshold: diff < 0.01 (less than 1% different)
```
### Comparison Details
- Compares RGBA pixels with a tolerance of 2 per channel
- Returns ratio of differing pixels (0.0 to 1.0)
- Different dimensions = 1.0 (completely different)
- Missing files = 1.0 (comparison failed)
### Visual Regression Test Example
```cpp
bool TestVisualRegression(TestContext& ctx) {
// Navigate to screen
GoHome(ctx);
ClickById(ctx, "dock-phone");
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
// Capture screenshot
mosis::testing::VisualCapture capture(ctx.width, ctx.height);
capture.CaptureScreenshot("dialer-current.png");
// Compare with baseline
float diff = mosis::testing::VisualCapture::CompareImages(
"baselines/dialer-expected.png",
"dialer-current.png"
);
// Allow up to 1% difference
return diff < 0.01f;
}
```
## Running Tests
### Command Line
```bash
# Run with defaults (auto-detects paths)
designer-test.exe
# Override paths
designer-test.exe --designer /path/to/mosis-designer.exe \
--document /path/to/home.rml \
--log /path/to/output.log \
--hierarchy /path/to/hierarchy.json \
--results /path/to/results.json
```
### Exit Codes
- `0`: All tests passed
- `1`: One or more tests failed
## Best Practices
### Element Identification
1. **Use IDs** for clickable elements that tests need to find
2. **Use semantic classes** like `app-bar-nav` for navigation buttons
3. **Avoid relying on position** - use hierarchy lookup instead
### Timing
1. **Wait after navigation** (1000ms) for animations to complete
2. **Reload hierarchy** before each element lookup
3. **Use retry logic** for file reads (hierarchy may be mid-write)
### Test Independence
1. **Start from known state** - call GoHome() at start of each test
2. **Clear logs** before verification
3. **Don't depend on previous test state**
## Adding Test IDs to RML
When creating new screens, add IDs to interactive elements:
```html
<!-- Good: Has ID for testing -->
<div id="dock-phone" class="dock-item" onclick="navigateTo('dialer')">
<img src="../../icons/phone.tga"/>
</div>
<!-- Good: Has class for back button detection -->
<div class="app-bar-nav btn-icon" onclick="goBack()">
<img src="../../icons/back.tga"/>
</div>
```
## Troubleshooting
### Element Not Found
- Verify the screen has loaded (check hierarchy `screen` field)
- Check if element has the expected ID/class
- Ensure element is visible (`visible: true`)
### Click Not Registering
- Check coordinate scaling (hierarchy vs window size)
- Increase wait time for animations
- Verify window is in foreground
### Timing Issues
- Increase delays after GoHome() and navigation
- Add retries for hierarchy reload
- Check for race conditions in file I/O
## Android Testing
The Android app supports event injection for automated testing.
### Event Injection Methods
The `MainActivity` provides these methods for programmatic testing:
```kotlin
// Inject touch events with normalized coordinates (0.0-1.0)
MainActivity.instance?.injectTouchDown(x, y)
MainActivity.instance?.injectTouchMove(x, y)
MainActivity.instance?.injectTouchUp(x, y)
MainActivity.instance?.injectClick(x, y) // Down + delay + Up
```
### Broadcast-Based Injection
For external test frameworks or ADB, send broadcasts:
```bash
# Inject a click at center (0.5, 0.5)
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "click" \
--ef x 0.5 \
--ef y 0.5
# Inject touch down
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "down" \
--ef x 0.2 \
--ef y 0.9
# Inject touch up
adb shell am broadcast -a com.omixlab.mosis.INJECT_TOUCH \
--es touch_type "up" \
--ef x 0.2 \
--ef y 0.9
```
### UI Element Coordinates
Dock items are positioned at the bottom of the screen. Approximate normalized positions:
| Element | X | Y |
|---------|---|---|
| dock-phone | 0.16 | 0.97 |
| dock-messages | 0.39 | 0.97 |
| dock-contacts | 0.61 | 0.97 |
| dock-browser | 0.84 | 0.97 |
### Instrumentation Tests
For full integration tests, use Android's instrumentation framework:
```kotlin
@RunWith(AndroidJUnit4::class)
class NavigationTest {
@get:Rule
val activityRule = ActivityScenarioRule(MainActivity::class.java)
@Test
fun testNavigateToDialer() {
// Wait for service to connect
Thread.sleep(2000)
// Click on phone dock icon (normalized coords)
activityRule.scenario.onActivity { activity ->
activity.injectClick(0.16f, 0.97f)
}
// Wait for navigation
Thread.sleep(1000)
// Verify navigation occurred (check logs or UI state)
}
}
```
## Future Improvements
- [x] Action recording (capture user interactions) - *CLI and infrastructure complete*
- [x] Screenshot comparison (visual regression testing) - *Pixel-level diff implemented*
- [x] Action playback with timing - *Fully functional*
- [x] GLFW input hooks for automatic mouse recording - *Complete via forked backend*
- [ ] Android instrumentation test suite
- [ ] Parallel test execution
- [ ] Test coverage reporting
- [ ] Cross-platform test runner (desktop + Android)
- [ ] Visual diff output (highlight changed pixels)

67
docs/UI-ASSETS.md Normal file
View File

@@ -0,0 +1,67 @@
# UI Assets Structure
All UI assets are in `src/main/assets/`:
```
assets/
├── apps/ # System apps (RML documents)
│ ├── home/home.rml # Home screen launcher
│ ├── dialer/dialer.rml # Phone dialer
│ ├── messages/ # Messages app
│ ├── contacts/ # Contacts app
│ ├── settings/ # Settings app
│ └── browser/ # Web browser
├── ui/ # Shared stylesheets
│ ├── html.rcss # Base HTML element styles
│ ├── theme.rcss # Design tokens (colors, typography)
│ └── components.rcss # Reusable UI components
├── scripts/ # Lua scripts
│ └── navigation.lua # Navigation system
├── icons/ # TGA icon files (24x24, 32x32)
└── fonts/ # TTF fonts (LatoLatin, Roboto)
```
## Navigation System
Navigation is handled by `scripts/navigation.lua`:
```lua
-- Navigate to a screen
navigateTo('dialer') -- Push to history, animate forward
-- Go back
goBack() -- Pop from history, animate back
-- Go home
goHome() -- Clear history, return to home
```
## Element IDs for Testing
Key elements have IDs for automated testing:
| ID | Location | Purpose |
|----|----------|---------|
| `dock-phone` | home.rml | Phone dock icon |
| `dock-messages` | home.rml | Messages dock icon |
| `dock-contacts` | home.rml | Contacts dock icon |
| `dock-browser` | home.rml | Browser dock icon |
| `app-settings` | home.rml | Settings app icon |
## CSS Classes for Navigation
Back buttons use `app-bar-nav` class for automated GoHome:
```html
<div class="app-bar-nav btn-icon" onclick="goBack()">
<img src="../../icons/back.tga"/>
</div>
```
Browser uses `browser-nav-btn` class for its toolbar back button:
```html
<div class="app-bar-nav browser-nav-btn" onclick="goBack()">
<img src="../../icons/back.tga"/>
</div>
```
The test framework's `FindBackButton()` searches for both classes to handle all screen layouts.