omar/MosisUnreal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

implement phone actor with UE5 RHI texture import for Vulkan HardwareBuffer

Browse Source
This commit is contained in:
omigamedev
2026-01-17 14:53:19 +01:00
parent 87191abbce
commit b349b86108
10 changed files with 545 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files

63
Plugins/MosisSDK/README.md
View File

@@ -12,24 +12,33 @@ The MosisSDK plugin connects to the MosisService Android application via AIDL (A
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Unreal Engine Game │
│ ┌─────────────────────┐ ┌─────────────────────────────┐ │
│ │ UMosisPhoneComponent│◄───│ MosisVulkanTexture │ │
│ │ (Touch Input) │ │ (HardwareBuffer Import) │ │
│ └──────────┬──────────┘ └──────────────▲──────────────┘ │
│ │ │ │
│ ▼ │ │
│ ┌─────────────────────────────────────────┴───────────────┐│
│ │ MosisClient ││
│ │ (AIDL IMosisListener) ││
│ └──────────────────────────┬──────────────────────────────┘│
└─────────────────────────────┼───────────────────────────────┘
│ Binder IPC
┌─────────────────────────────▼───────────────────────────────┐
│ MosisService │
│ (Renders phone UI via RmlUi) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│ Unreal Engine Game │
│ │
│ ┌─────────────────┐ ┌──────────────────────────────────────────┐ │
│ │ AMosisPhoneActor│ │ UMosisPhoneTexture │ │
│ │ (Plane Mesh) │ │ │ │ │
│ │ (Material) │ │ ▼ │ │
│ └────────┬────────┘ │ ┌────────────────────────────────────┐ │ │
│ │ │ │ FMosisPhoneTextureResource │ │ │
│ ▼ │ │ │ │ │ │
│ ┌─────────────────────┐ │ │ ▼ ENQUEUE_RENDER_CMD │ │ │
│ │ UMosisPhoneComponent│──┼──► IVulkanDynamicRHI:: │ │ │
│ │ (Touch Input) │ │ │ RHICreateTexture2DFromAndroid │ │ │
│ │ (Callbacks) │ │ │ HardwareBuffer() │ │ │
│ └──────────┬──────────┘ │ └────────────────────────────────────┘ │ │
│ │ └──────────────────────────────────────────┘ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ MosisClient │ │
│ │ (AIDL IMosisListener) │ │
│ └──────────────────────────────┬───────────────────────────────────┘ │
└─────────────────────────────────┼───────────────────────────────────────┘
│ Binder IPC
┌─────────────────────────────────▼───────────────────────────────────────┐
│ MosisService │
│ (Renders phone UI via RmlUi) │
└─────────────────────────────────────────────────────────────────────────┘
```
## Prerequisites
@@ -73,12 +82,14 @@ Plugins/MosisSDK/
│ │ └── BpMosisService.h
│ ├── Public/
│ │ ├── MosisSDK.h # Module interface
│ │ ├── MosisPhoneComponent.h # UE5 component
│ │ └── MosisPhoneActor.h # Blueprint actor
│ │ ├── MosisPhoneComponent.h # UE5 component (touch, callbacks)
│ │ └── MosisPhoneActor.h # Blueprint actor (mesh, material)
│ └── Private/
│ ├── MosisSDK.cpp # Module + JNI callbacks
│ ├── MosisClient.h/cpp # AIDL client implementation
│ ├── MosisVulkanTexture.h/cpp # Vulkan HardwareBuffer import
│ ├── MosisPhoneTexture.h/cpp # UTexture for phone screen
│ ├── MosisPhoneTextureResource.h/cpp # Render thread resource
│ ├── MosisVulkanTexture.h/cpp # Legacy Vulkan import (unused)
│ ├── MosisPhoneComponent.cpp
│ ├── MosisPhoneActor.cpp
│ └── Android/
@@ -152,7 +163,8 @@ The separation ensures Windows builds don't try to compile Android-specific AIDL
| Slate, SlateCore | ✓ | ✓ | UI framework |
| RenderCore, RHI | ✓ | ✓ | Rendering abstraction |
| Launch, ApplicationCore | ✗ | ✓ | Android JNI access |
| Vulkan | ✗ | ✓ | Hardware buffer import |
| VulkanRHI | ✗ | ✓ | IVulkanDynamicRHI for HardwareBuffer import |
| Vulkan | ✗ | ✓ | Vulkan headers |
| binder_ndk, android, nativewindow | ✗ | ✓ | Android system libs |
### NDK Header Compatibility
@@ -353,6 +365,13 @@ HandTrackingVersion=V2
## Version History
- **v1.1** - UE5 RHI texture integration
- New `UMosisPhoneTexture` using UE5's `IVulkanDynamicRHI::RHICreateTexture2DFromAndroidHardwareBuffer()`
- `FMosisPhoneTextureResource` for render thread HardwareBuffer import
- `AMosisPhoneActor` now includes default plane mesh and material setup
- Thread-safe callbacks using `TWeakObjectPtr` and `TAtomic`
- Automatic texture-to-material binding in tick
- **v1.0** - Initial implementation
- AIDL client for MosisService connection
- Vulkan HardwareBuffer import