# Milestone 3: Virtual Hardware **Status**: Not Started **Goal**: Hardware-like APIs backed by game engine or real devices. --- ## Overview Mosis needs to expose smartphone-like hardware interfaces that can be: - Provided by game engines (Unity/Unreal) in VR mode - Mocked for desktop testing - Connected to real device hardware on Android --- ## Components ### 3.1 Camera Interface **Header**: `src/main/kernel/include/camera.h` ```cpp namespace mosis { struct CameraFrame { int width; int height; std::vector rgba_data; // RGBA8 format int64_t timestamp_ms; }; using CameraCallback = std::function; class ICamera { public: virtual ~ICamera() = default; // Start/stop frame capture virtual void StartCapture(CameraCallback callback) = 0; virtual void StopCapture() = 0; // Configuration virtual void SetResolution(int width, int height) = 0; virtual bool IsAvailable() const = 0; // Get current state virtual int GetWidth() const = 0; virtual int GetHeight() const = 0; }; } // namespace mosis ``` **Implementations**: | Implementation | Description | File | |----------------|-------------|------| | `GameCamera` | Receives texture from Unity/Unreal | `game_camera.cpp` | | `DesktopCamera` | System webcam via OpenCV (optional) | `desktop_camera.cpp` | | `AndroidCamera` | Camera2 API integration | `android_camera.cpp` | | `MockCamera` | Test patterns (checkerboard, gradient) | `mock_camera.cpp` | ### 3.2 Microphone Interface **Header**: `src/main/kernel/include/microphone.h` ```cpp namespace mosis { struct AudioBuffer { std::vector samples; // PCM 16-bit int sample_rate; // Typically 44100 or 48000 int channels; // 1 = mono, 2 = stereo int64_t timestamp_ms; }; using AudioCallback = std::function; class IMicrophone { public: virtual ~IMicrophone() = default; virtual void StartCapture(AudioCallback callback) = 0; virtual void StopCapture() = 0; virtual void SetSampleRate(int rate) = 0; virtual bool IsAvailable() const = 0; virtual bool IsCapturing() const = 0; }; } // namespace mosis ``` **Implementations**: | Implementation | Description | |----------------|-------------| | `GameMicrophone` | Audio from Unity/Unreal AudioSource | | `DesktopMicrophone` | System mic via PortAudio | | `AndroidMicrophone` | AudioRecord API | | `MockMicrophone` | Silence or test tones | ### 3.3 Speaker Interface **Header**: `src/main/kernel/include/speaker.h` ```cpp namespace mosis { class ISpeaker { public: virtual ~ISpeaker() = default; virtual void PlayAudio(const AudioBuffer& buffer) = 0; virtual void SetVolume(float volume) = 0; // 0.0 - 1.0 virtual float GetVolume() const = 0; virtual bool IsAvailable() const = 0; }; } // namespace mosis ``` ### 3.4 Filesystem Interface **Header**: `src/main/kernel/include/filesystem.h` ```cpp namespace mosis { enum class FileMode { Read, Write, Append }; struct FileInfo { std::string name; bool is_directory; size_t size; int64_t modified_time; }; class IFileSystem { public: virtual ~IFileSystem() = default; // File operations virtual std::vector ReadFile(const std::string& path) = 0; virtual bool WriteFile(const std::string& path, const std::vector& data) = 0; virtual bool DeleteFile(const std::string& path) = 0; virtual bool FileExists(const std::string& path) = 0; // Directory operations virtual std::vector ListDirectory(const std::string& path) = 0; virtual bool CreateDirectory(const std::string& path) = 0; virtual bool DeleteDirectory(const std::string& path) = 0; // Sandboxed paths virtual std::string GetAppDataPath(const std::string& app_id) = 0; virtual std::string GetSharedMediaPath() = 0; }; } // namespace mosis ``` **App Storage Structure**: ``` /data/ ├── apps/ │ ├── com.example.app1/ │ │ ├── files/ │ │ ├── cache/ │ │ └── databases/ │ └── com.example.app2/ ├── shared/ │ ├── photos/ │ ├── downloads/ │ └── music/ ``` ### 3.5 Network Interface **Header**: `src/main/kernel/include/network.h` ```cpp namespace mosis { struct HttpRequest { std::string method; // GET, POST, PUT, DELETE std::string url; std::map headers; std::vector body; }; struct HttpResponse { int status_code; std::map headers; std::vector body; }; class INetwork { public: virtual ~INetwork() = default; // HTTP using HttpCallback = std::function; virtual void Fetch(const HttpRequest& request, HttpCallback callback) = 0; // Connectivity virtual bool IsOnline() const = 0; virtual std::string GetConnectionType() const = 0; // "wifi", "cellular", "none" }; } // namespace mosis ``` --- ## Platform Integration ### Platform Interface Extension **File**: `src/main/kernel/include/platform.h` ```cpp class IPlatform { public: // ... existing methods ... // Hardware providers virtual ICamera* GetCamera() = 0; virtual IMicrophone* GetMicrophone() = 0; virtual ISpeaker* GetSpeaker() = 0; virtual IFileSystem* GetFileSystem() = 0; virtual INetwork* GetNetwork() = 0; }; ``` ### Desktop Implementation **File**: `designer/src/desktop_platform.cpp` ```cpp class DesktopPlatform : public IPlatform { std::unique_ptr m_camera; std::unique_ptr m_microphone; std::unique_ptr m_speaker; std::unique_ptr m_filesystem; std::unique_ptr m_network; public: ICamera* GetCamera() override { return m_camera.get(); } // ... }; ``` ### Android Implementation **File**: `src/main/cpp/android_platform.cpp` ```cpp class AndroidPlatform : public IPlatform { // Use JNI to access Android APIs std::unique_ptr m_camera; // ... }; ``` ### Game Engine Implementation **File**: Unity plugin or Unreal plugin ```cpp class GamePlatform : public IPlatform { // Receives textures/audio from game engine std::unique_ptr m_camera; // ... }; ``` --- ## Implementation Plan ### Phase 1: Interfaces Only - [ ] Define all interface headers - [ ] Add to platform abstraction - [ ] Create mock implementations for testing ### Phase 2: Desktop Implementation - [ ] MockCamera (test patterns) - [ ] PortAudio for speaker output - [ ] Standard filesystem access - [ ] libcurl for HTTP ### Phase 3: Android Implementation - [ ] Camera2 API wrapper (JNI) - [ ] AudioRecord/AudioTrack wrappers - [ ] Android filesystem with proper sandboxing - [ ] OkHttp or native networking ### Phase 4: Game Engine Integration - [ ] Unity RenderTexture → ICamera - [ ] Unity AudioSource → IMicrophone - [ ] Unity AudioListener → ISpeaker - [ ] Unreal equivalents --- ## Dependencies | Dependency | Purpose | vcpkg Package | |------------|---------|---------------| | PortAudio | Desktop audio I/O | `portaudio` | | OpenCV | Desktop webcam (optional) | `opencv4` | | libcurl | HTTP client | `curl` | --- ## Test Cases 1. **MockCamera**: Renders test pattern, verify frame callback 2. **FileSystem**: Create, read, write, delete operations 3. **Network**: Mock HTTP responses, verify request/response 4. **Audio**: Verify sample rates, buffer formats --- ## Acceptance Criteria - [ ] All interfaces defined in kernel/include/ - [ ] Mock implementations work on desktop - [ ] Camera app can display camera frames - [ ] Browser app can make HTTP requests - [ ] Apps can persist data to filesystem