MosisService/docs/MILESTONE-3.md

Milestone 3: Virtual Hardware

Status: Not Started Goal: Hardware-like APIs backed by game engine or real devices.

---

Overview

Mosis needs to expose smartphone-like hardware interfaces that can be:

• Provided by game engines (Unity/Unreal) in VR mode
• Mocked for desktop testing
• Connected to real device hardware on Android

---

Components

3.1 Camera Interface

Header: src/main/kernel/include/camera.h

namespace mosis {

struct CameraFrame {
    int width;
    int height;
    std::vector<uint8_t> rgba_data;  // RGBA8 format
    int64_t timestamp_ms;
};

using CameraCallback = std::function<void(const CameraFrame& frame)>;

class ICamera {
public:
    virtual ~ICamera() = default;

    // Start/stop frame capture
    virtual void StartCapture(CameraCallback callback) = 0;
    virtual void StopCapture() = 0;

    // Configuration
    virtual void SetResolution(int width, int height) = 0;
    virtual bool IsAvailable() const = 0;

    // Get current state
    virtual int GetWidth() const = 0;
    virtual int GetHeight() const = 0;
};

} // namespace mosis

Implementations:

Implementation	Description	File
GameCamera	Receives texture from Unity/Unreal	game_camera.cpp
DesktopCamera	System webcam via OpenCV (optional)	desktop_camera.cpp
AndroidCamera	Camera2 API integration	android_camera.cpp
MockCamera	Test patterns (checkerboard, gradient)	mock_camera.cpp

3.2 Microphone Interface

Header: src/main/kernel/include/microphone.h

namespace mosis {

struct AudioBuffer {
    std::vector<int16_t> samples;  // PCM 16-bit
    int sample_rate;               // Typically 44100 or 48000
    int channels;                  // 1 = mono, 2 = stereo
    int64_t timestamp_ms;
};

using AudioCallback = std::function<void(const AudioBuffer& buffer)>;

class IMicrophone {
public:
    virtual ~IMicrophone() = default;

    virtual void StartCapture(AudioCallback callback) = 0;
    virtual void StopCapture() = 0;

    virtual void SetSampleRate(int rate) = 0;
    virtual bool IsAvailable() const = 0;
    virtual bool IsCapturing() const = 0;
};

} // namespace mosis

Implementations:

Implementation	Description
GameMicrophone	Audio from Unity/Unreal AudioSource
DesktopMicrophone	System mic via PortAudio
AndroidMicrophone	AudioRecord API
MockMicrophone	Silence or test tones

3.3 Speaker Interface

Header: src/main/kernel/include/speaker.h

namespace mosis {

class ISpeaker {
public:
    virtual ~ISpeaker() = default;

    virtual void PlayAudio(const AudioBuffer& buffer) = 0;
    virtual void SetVolume(float volume) = 0;  // 0.0 - 1.0
    virtual float GetVolume() const = 0;

    virtual bool IsAvailable() const = 0;
};

} // namespace mosis

3.4 Filesystem Interface

Header: src/main/kernel/include/filesystem.h

namespace mosis {

enum class FileMode { Read, Write, Append };

struct FileInfo {
    std::string name;
    bool is_directory;
    size_t size;
    int64_t modified_time;
};

class IFileSystem {
public:
    virtual ~IFileSystem() = default;

    // File operations
    virtual std::vector<uint8_t> ReadFile(const std::string& path) = 0;
    virtual bool WriteFile(const std::string& path, const std::vector<uint8_t>& data) = 0;
    virtual bool DeleteFile(const std::string& path) = 0;
    virtual bool FileExists(const std::string& path) = 0;

    // Directory operations
    virtual std::vector<FileInfo> ListDirectory(const std::string& path) = 0;
    virtual bool CreateDirectory(const std::string& path) = 0;
    virtual bool DeleteDirectory(const std::string& path) = 0;

    // Sandboxed paths
    virtual std::string GetAppDataPath(const std::string& app_id) = 0;
    virtual std::string GetSharedMediaPath() = 0;
};

} // namespace mosis

App Storage Structure:

/data/
├── apps/
│   ├── com.example.app1/
│   │   ├── files/
│   │   ├── cache/
│   │   └── databases/
│   └── com.example.app2/
├── shared/
│   ├── photos/
│   ├── downloads/
│   └── music/

3.5 Network Interface

Header: src/main/kernel/include/network.h

namespace mosis {

struct HttpRequest {
    std::string method;  // GET, POST, PUT, DELETE
    std::string url;
    std::map<std::string, std::string> headers;
    std::vector<uint8_t> body;
};

struct HttpResponse {
    int status_code;
    std::map<std::string, std::string> headers;
    std::vector<uint8_t> body;
};

class INetwork {
public:
    virtual ~INetwork() = default;

    // HTTP
    using HttpCallback = std::function<void(const HttpResponse&)>;
    virtual void Fetch(const HttpRequest& request, HttpCallback callback) = 0;

    // Connectivity
    virtual bool IsOnline() const = 0;
    virtual std::string GetConnectionType() const = 0;  // "wifi", "cellular", "none"
};

} // namespace mosis

---

Platform Integration

Platform Interface Extension

File: src/main/kernel/include/platform.h

class IPlatform {
public:
    // ... existing methods ...

    // Hardware providers
    virtual ICamera* GetCamera() = 0;
    virtual IMicrophone* GetMicrophone() = 0;
    virtual ISpeaker* GetSpeaker() = 0;
    virtual IFileSystem* GetFileSystem() = 0;
    virtual INetwork* GetNetwork() = 0;
};

Desktop Implementation

File: designer/src/desktop_platform.cpp

class DesktopPlatform : public IPlatform {
    std::unique_ptr<MockCamera> m_camera;
    std::unique_ptr<MockMicrophone> m_microphone;
    std::unique_ptr<DesktopSpeaker> m_speaker;
    std::unique_ptr<DesktopFileSystem> m_filesystem;
    std::unique_ptr<DesktopNetwork> m_network;

public:
    ICamera* GetCamera() override { return m_camera.get(); }
    // ...
};

Android Implementation

File: src/main/cpp/android_platform.cpp

class AndroidPlatform : public IPlatform {
    // Use JNI to access Android APIs
    std::unique_ptr<AndroidCamera> m_camera;
    // ...
};

Game Engine Implementation

File: Unity plugin or Unreal plugin

class GamePlatform : public IPlatform {
    // Receives textures/audio from game engine
    std::unique_ptr<GameCamera> m_camera;
    // ...
};

---

Implementation Plan

Phase 1: Interfaces Only

• Define all interface headers
• Add to platform abstraction
• Create mock implementations for testing

Phase 2: Desktop Implementation

• MockCamera (test patterns)
• PortAudio for speaker output
• Standard filesystem access
• libcurl for HTTP

Phase 3: Android Implementation

• Camera2 API wrapper (JNI)
• AudioRecord/AudioTrack wrappers
• Android filesystem with proper sandboxing
• OkHttp or native networking

Phase 4: Game Engine Integration

• Unity RenderTexture → ICamera
• Unity AudioSource → IMicrophone
• Unity AudioListener → ISpeaker
• Unreal equivalents

---

Dependencies

Dependency	Purpose	vcpkg Package
PortAudio	Desktop audio I/O	portaudio
OpenCV	Desktop webcam (optional)	opencv4
libcurl	HTTP client	curl

---

Test Cases

1. MockCamera: Renders test pattern, verify frame callback
2. FileSystem: Create, read, write, delete operations
3. Network: Mock HTTP responses, verify request/response
4. Audio: Verify sample rates, buffer formats

---

Acceptance Criteria

• All interfaces defined in kernel/include/
• Mock implementations work on desktop
• Camera app can display camera frames
• Browser app can make HTTP requests
• Apps can persist data to filesystem