# MosisSDK for Unity

Unity package providing integration with MosisService for virtual smartphone display in VR applications.

## Overview

This package connects to the MosisService Android app via AIDL Binder IPC to:
- Receive rendered phone screen frames via AHardwareBuffer
- Forward VR controller touch input to the virtual phone
- Support both OpenGL ES and Vulkan graphics backends

## Requirements

- Unity 6000.3.2f1 or later
- Android target platform
- MosisService app installed on device

## Installation

1. Open Window > Package Manager
2. Click + > Add package from disk
3. Navigate to `Packages/com.omarator.mosissdk/package.json`

## Project Structure

```
Packages/com.omarator.mosissdk/
├── package.json
├── README.md
├── Runtime/
│   ├── KotlinBridge.cs          # C# to native bridge
│   └── com.omarator.mosissdk.asmdef
├── Plugins/
│   └── Android/
│       ├── MosisSDK.aar         # Kotlin service connection
│       ├── libs/
│       │   └── arm64-v8a/
│       │       └── libmy_native_lib.so
│       └── cpp/                 # Native source
│           ├── CMakeLists.txt
│           ├── my_native_code.cpp
│           ├── texture_backend.h
│           ├── opengl_backend.h/cpp
│           └── vulkan_backend.h/cpp
└── Assets/
    └── PhoneInteraction.cs      # VR controller input
```

## Build Commands

### Unity Build (Export to Android Studio)

Unity exports a Gradle project which is then built in Android Studio:

1. File > Build Settings
2. Switch Platform to Android
3. Player Settings > Other Settings:
   - Scripting Backend: IL2CPP
   - Target Architectures: ARM64
   - Graphics APIs: Vulkan, OpenGLES3 (in order of preference)
   - Minimum API Level: 29
4. Check "Export Project"
5. Click Export and select output folder (e.g., `D:\Dev\Mosis\Builds\Unity\Android\MosisVR`)

### Build Gradle Project

After exporting from Unity:

```batch
cd D:\Dev\Mosis\Builds\Unity\Android\MosisVR
gradle assembleRelease

:: APK will be at:
:: launcher\build\outputs\apk\release\launcher-release.apk
```

Or open in Android Studio and build from there.

### Command Line Export (Optional)

```batch
"C:\Program Files\Unity\Hub\Editor\6000.3.2f1\Editor\Unity.exe" ^
    -batchmode -quit -nographics ^
    -projectPath "D:\Dev\Mosis\MosisVR" ^
    -executeMethod BuildScript.BuildAndroidCI ^
    -outputPath "D:\Dev\Mosis\Builds\Unity\Android\MosisVR" ^
    -logFile "D:\Dev\Mosis\Builds\Unity\build.log"
```

### Native Plugin Build (Manual)

To rebuild the native library manually:

```batch
cd Packages/com.omarator.mosissdk/Plugins/Android/cpp

:: Configure with Android NDK
cmake -B build ^
    -DCMAKE_TOOLCHAIN_FILE=%ANDROID_NDK_HOME%/build/cmake/android.toolchain.cmake ^
    -DANDROID_ABI=arm64-v8a ^
    -DANDROID_PLATFORM=android-29

:: Build
cmake --build build

:: Copy output
copy build\libmy_native_lib.so ..\libs\arm64-v8a\
```

## Usage

### Basic Setup

1. Add `KotlinBridge` component to a GameObject in your scene
2. Add `PhoneInteraction` component for VR input handling
3. Assign VR controller and phone plane references

### KotlinBridge

The `KotlinBridge` component manages the connection to MosisService:

```csharp
public class KotlinBridge : MonoBehaviour
{
    public Material phoneMaterial;  // Material to apply phone texture

    // Called automatically when texture is ready
    // Texture is applied to phoneMaterial's _MainTex
}
```

### PhoneInteraction

Handles VR controller raycasting and touch input:

```csharp
public class PhoneInteraction : MonoBehaviour
{
    public GameObject Controller;       // VR controller
    public GameObject Plane;            // Phone display plane
    public XRNode inputDevice;          // Controller hand
    public float triggerThreshold;      // Trigger press threshold
}
```

### Touch API

Send touch events from custom scripts:

```csharp
// Touch down at normalized UV (0-1)
KotlinBridge.SendTouchDown(0.5f, 0.5f);

// Touch move
KotlinBridge.SendTouchMove(0.6f, 0.5f);

// Touch up
KotlinBridge.SendTouchUp(0.6f, 0.5f);
```

## Graphics Backend

The native plugin automatically selects the appropriate backend:

| Graphics API | Backend | Status |
|--------------|---------|--------|
| Vulkan | VulkanTextureBackend | Preferred |
| OpenGL ES 3.0 | OpenGLTextureBackend | Fallback |
| OpenGL ES 2.0 | OpenGLTextureBackend | Fallback |

Backend selection happens in `UnityPluginLoad()` based on the active renderer.

### Vulkan Import

When using Vulkan, the plugin imports AHardwareBuffer directly as a VkImage:

1. Query buffer properties via `vkGetAndroidHardwareBufferPropertiesANDROID`
2. Create VkImage with external memory handle type
3. Import memory with `VkImportAndroidHardwareBufferInfoANDROID`
4. Copy to local image each frame for safe rendering

### OpenGL Import

When using OpenGL ES, the plugin uses EGL image extension:

1. Create EGLImage from AHardwareBuffer via `eglCreateImageKHR`
2. Bind to OpenGL texture via `glEGLImageTargetTexture2DOES`
3. Blit to local texture each frame

## Device Testing

```bash
# Install MosisService first
adb install -r MosisService-debug.apk

# Install Unity app
adb install -r MosisVR.apk

# Launch service
adb shell am start -n com.omixlab.mosis/.MainActivity

# Launch Unity app
adb shell am start -n com.omixlab.mosisvr/com.unity3d.player.UnityPlayerActivity

# Monitor logs
adb logcat -s Unity MosisSDK Vulkan
```

## Troubleshooting

### Black Screen / No Texture

- Verify MosisService is running (`adb shell ps | grep mosis`)
- Check logcat for connection errors
- Ensure both apps have same user ID or are debuggable

### Vulkan Errors

- Device must support `VK_ANDROID_external_memory_android_hardware_buffer`
- Check `adb logcat -s Vulkan` for extension availability
- Falls back to OpenGL if Vulkan init fails

### Touch Not Working

- Verify `PhoneInteraction` references are set
- Check XR input device selection matches controller
- Test with mouse click fallback in editor

## Implementation Notes

### Files Modified in Vulkan Implementation

| File | Changes |
|------|---------|
| `my_native_code.cpp` | Backend abstraction, `UnityPluginLoad` detection |
| `CMakeLists.txt` | Added Vulkan library, new source files |
| `KotlinBridge.cs` | Added `SendTouchDown`/`SendTouchUp`, dynamic resolution |
| `PhoneInteraction.cs` | Added VR trigger-based touch handling |

### New Files Added

| File | Purpose |
|------|---------|
| `texture_backend.h` | ITextureBackend abstract interface |
| `opengl_backend.h/cpp` | OpenGL ES implementation |
| `vulkan_backend.h/cpp` | Vulkan implementation |

## Version History

- **1.0.0** - Initial release with OpenGL support
- **1.1.0** - Added Vulkan backend, touch down/up events, dynamic resolution
