522 lines
13 KiB
Markdown
522 lines
13 KiB
Markdown
# Milestone 11: Virtual Hardware - Camera
|
|
|
|
**Status**: Complete
|
|
**Goal**: Camera access with mandatory recording indicators and security controls.
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
This milestone implements secure camera access for Lua apps:
|
|
- Permission required (`camera` permission)
|
|
- User gesture required to start session
|
|
- Mandatory recording indicator (system-controlled, cannot be hidden)
|
|
- Single session per app
|
|
- Frame rate limiting
|
|
- Automatic cleanup on app stop
|
|
|
|
### Key Deliverables
|
|
|
|
1. **CameraInterface class** - Session management, permission checks
|
|
2. **CameraSession class** - Active camera session wrapper
|
|
3. **Lua camera API** - `camera.start()`, session methods
|
|
4. **Recording indicator** - System-level UI notification
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
```
|
|
src/main/cpp/sandbox/
|
|
├── camera_interface.h # NEW - Camera API header
|
|
└── camera_interface.cpp # NEW - Camera implementation
|
|
```
|
|
|
|
---
|
|
|
|
## Implementation Details
|
|
|
|
### 1. CameraInterface Class
|
|
|
|
```cpp
|
|
// camera_interface.h
|
|
#pragma once
|
|
|
|
#include <string>
|
|
#include <memory>
|
|
#include <functional>
|
|
#include <mutex>
|
|
#include <atomic>
|
|
#include <vector>
|
|
|
|
struct lua_State;
|
|
|
|
namespace mosis {
|
|
|
|
class PermissionGate;
|
|
|
|
enum class CameraFacing {
|
|
Front,
|
|
Back
|
|
};
|
|
|
|
enum class CameraResolution {
|
|
VGA, // 640x480
|
|
HD, // 1280x720
|
|
FullHD // 1920x1080
|
|
};
|
|
|
|
struct CameraConfig {
|
|
CameraFacing facing = CameraFacing::Back;
|
|
CameraResolution resolution = CameraResolution::HD;
|
|
int max_fps = 30;
|
|
};
|
|
|
|
struct CameraFrame {
|
|
std::vector<uint8_t> data; // RGBA or JPEG data
|
|
int width;
|
|
int height;
|
|
uint64_t timestamp_ms;
|
|
bool is_jpeg;
|
|
};
|
|
|
|
class CameraSession {
|
|
public:
|
|
using FrameCallback = std::function<void(const CameraFrame& frame)>;
|
|
|
|
CameraSession(int id, const CameraConfig& config);
|
|
~CameraSession();
|
|
|
|
int GetId() const { return m_id; }
|
|
const CameraConfig& GetConfig() const { return m_config; }
|
|
bool IsActive() const { return m_active; }
|
|
|
|
// Capture single photo (returns copy of current frame)
|
|
CameraFrame Capture();
|
|
|
|
// Set frame callback (for preview)
|
|
void SetOnFrame(FrameCallback cb) { m_on_frame = std::move(cb); }
|
|
|
|
// Stop session
|
|
void Stop();
|
|
|
|
// For mock mode - simulate frame arrival
|
|
void SimulateFrame(const CameraFrame& frame);
|
|
|
|
private:
|
|
int m_id;
|
|
CameraConfig m_config;
|
|
std::atomic<bool> m_active{true};
|
|
FrameCallback m_on_frame;
|
|
CameraFrame m_last_frame;
|
|
std::mutex m_mutex;
|
|
};
|
|
|
|
class CameraInterface {
|
|
public:
|
|
CameraInterface(const std::string& app_id, PermissionGate* permissions);
|
|
~CameraInterface();
|
|
|
|
// Start camera session
|
|
// Returns session on success, nullptr on failure (sets error)
|
|
// Requires camera permission and user gesture
|
|
std::shared_ptr<CameraSession> StartSession(const CameraConfig& config, std::string& error);
|
|
|
|
// Stop active session
|
|
void StopSession();
|
|
|
|
// Check if session is active
|
|
bool HasActiveSession() const;
|
|
|
|
// Check if recording indicator should be shown
|
|
bool IsIndicatorVisible() const;
|
|
|
|
// Cleanup on app stop
|
|
void Shutdown();
|
|
|
|
// For testing
|
|
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
|
|
bool IsMockMode() const { return m_mock_mode; }
|
|
|
|
// Simulate user gesture for testing
|
|
void SimulateUserGesture();
|
|
|
|
private:
|
|
std::string m_app_id;
|
|
PermissionGate* m_permissions;
|
|
std::shared_ptr<CameraSession> m_active_session;
|
|
mutable std::mutex m_mutex;
|
|
bool m_mock_mode = true;
|
|
std::atomic<bool> m_indicator_visible{false};
|
|
|
|
// Track user gesture timing
|
|
uint64_t m_last_gesture_time = 0;
|
|
static constexpr uint64_t GESTURE_VALIDITY_MS = 5000; // 5 seconds
|
|
|
|
bool HasRecentUserGesture() const;
|
|
};
|
|
|
|
// Register camera.* APIs as globals
|
|
void RegisterCameraAPI(lua_State* L, CameraInterface* camera);
|
|
|
|
} // namespace mosis
|
|
```
|
|
|
|
### 2. Permission Requirements
|
|
|
|
Camera access requires:
|
|
1. `camera` permission declared in manifest
|
|
2. Permission granted by user (dangerous permission)
|
|
3. Recent user gesture (within 5 seconds)
|
|
|
|
```cpp
|
|
// Permission check flow
|
|
bool CameraInterface::StartSession(...) {
|
|
// 1. Check permission
|
|
if (!m_permissions->Check("camera")) {
|
|
error = "Camera permission not granted";
|
|
return nullptr;
|
|
}
|
|
|
|
// 2. Check user gesture
|
|
if (!HasRecentUserGesture()) {
|
|
error = "Camera requires user gesture";
|
|
return nullptr;
|
|
}
|
|
|
|
// 3. Check no existing session
|
|
if (m_active_session) {
|
|
error = "Camera session already active";
|
|
return nullptr;
|
|
}
|
|
|
|
// ... create session
|
|
}
|
|
```
|
|
|
|
### 3. Recording Indicator
|
|
|
|
The recording indicator is mandatory and system-controlled:
|
|
- Shown whenever camera session is active
|
|
- Cannot be hidden or obscured by app
|
|
- Positioned in system UI area
|
|
- Shows camera icon with "Recording" text
|
|
|
|
```cpp
|
|
// Indicator control
|
|
void CameraInterface::ShowIndicator() {
|
|
m_indicator_visible = true;
|
|
// In real implementation: notify system UI
|
|
}
|
|
|
|
void CameraInterface::HideIndicator() {
|
|
m_indicator_visible = false;
|
|
// In real implementation: notify system UI
|
|
}
|
|
```
|
|
|
|
### 4. Lua API
|
|
|
|
```lua
|
|
-- Start camera session (requires permission + user gesture)
|
|
local session, err = camera.start({
|
|
facing = "back", -- "front" or "back"
|
|
resolution = "720p", -- "480p", "720p", or "1080p"
|
|
fps = 30 -- max frames per second
|
|
})
|
|
|
|
if not session then
|
|
print("Failed to start camera:", err)
|
|
return
|
|
end
|
|
|
|
-- Capture single photo
|
|
local photo = session:capture()
|
|
if photo then
|
|
print("Captured", photo.width, "x", photo.height, "image")
|
|
-- photo.data contains JPEG bytes
|
|
end
|
|
|
|
-- Set frame callback (for preview)
|
|
session:on("frame", function(frame)
|
|
-- frame.data, frame.width, frame.height, frame.timestamp
|
|
end)
|
|
|
|
-- Stop session
|
|
session:stop()
|
|
|
|
-- Check if camera is active
|
|
if camera.isActive() then
|
|
print("Camera is running")
|
|
end
|
|
```
|
|
|
|
### 5. Frame Rate Limiting
|
|
|
|
To prevent resource abuse:
|
|
- Maximum 30 FPS
|
|
- Minimum 16ms between frames
|
|
- Frame delivery is asynchronous
|
|
|
|
```cpp
|
|
struct FrameRateLimiter {
|
|
int max_fps = 30;
|
|
uint64_t min_frame_interval_ms = 33; // ~30 FPS
|
|
uint64_t last_frame_time = 0;
|
|
|
|
bool ShouldDeliverFrame() {
|
|
uint64_t now = GetCurrentTimeMs();
|
|
if (now - last_frame_time >= min_frame_interval_ms) {
|
|
last_frame_time = now;
|
|
return true;
|
|
}
|
|
return false;
|
|
}
|
|
};
|
|
```
|
|
|
|
### 6. Session Lifecycle
|
|
|
|
```
|
|
User Gesture ──► camera.start() ──► Session Active ──► Indicator Shown
|
|
│ │
|
|
│ error │ session:stop()
|
|
▼ ▼
|
|
nil, err Session Closed ──► Indicator Hidden
|
|
|
|
App Stop ──► CameraInterface::Shutdown() ──► All Sessions Closed
|
|
```
|
|
|
|
---
|
|
|
|
## Test Cases
|
|
|
|
### Test 1: Requires Permission
|
|
|
|
```cpp
|
|
bool Test_CameraRequiresPermission(std::string& error_msg) {
|
|
// Create permission gate WITHOUT camera permission
|
|
PermissionGate permissions("test.app", {}, false);
|
|
|
|
mosis::CameraInterface camera("test.app", &permissions);
|
|
camera.SimulateUserGesture();
|
|
|
|
std::string err;
|
|
mosis::CameraConfig config;
|
|
auto session = camera.StartSession(config, err);
|
|
|
|
EXPECT_TRUE(session == nullptr);
|
|
EXPECT_TRUE(err.find("permission") != std::string::npos);
|
|
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 2: Requires User Gesture
|
|
|
|
```cpp
|
|
bool Test_CameraRequiresUserGesture(std::string& error_msg) {
|
|
// Create permission gate WITH camera permission
|
|
PermissionGate permissions("test.app", {"camera"}, false);
|
|
permissions.Grant("camera");
|
|
|
|
mosis::CameraInterface camera("test.app", &permissions);
|
|
// Note: NOT calling SimulateUserGesture()
|
|
|
|
std::string err;
|
|
mosis::CameraConfig config;
|
|
auto session = camera.StartSession(config, err);
|
|
|
|
EXPECT_TRUE(session == nullptr);
|
|
EXPECT_TRUE(err.find("gesture") != std::string::npos);
|
|
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 3: Shows Indicator
|
|
|
|
```cpp
|
|
bool Test_CameraShowsIndicator(std::string& error_msg) {
|
|
PermissionGate permissions("test.app", {"camera"}, false);
|
|
permissions.Grant("camera");
|
|
|
|
mosis::CameraInterface camera("test.app", &permissions);
|
|
camera.SimulateUserGesture();
|
|
|
|
EXPECT_FALSE(camera.IsIndicatorVisible());
|
|
|
|
std::string err;
|
|
mosis::CameraConfig config;
|
|
auto session = camera.StartSession(config, err);
|
|
|
|
// Indicator should be visible while session is active
|
|
EXPECT_TRUE(camera.IsIndicatorVisible());
|
|
|
|
session->Stop();
|
|
|
|
// Indicator should be hidden after stop
|
|
EXPECT_FALSE(camera.IsIndicatorVisible());
|
|
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 4: Single Session Only
|
|
|
|
```cpp
|
|
bool Test_CameraSingleSession(std::string& error_msg) {
|
|
PermissionGate permissions("test.app", {"camera"}, false);
|
|
permissions.Grant("camera");
|
|
|
|
mosis::CameraInterface camera("test.app", &permissions);
|
|
camera.SimulateUserGesture();
|
|
|
|
std::string err;
|
|
mosis::CameraConfig config;
|
|
|
|
// First session should succeed
|
|
auto session1 = camera.StartSession(config, err);
|
|
EXPECT_TRUE(session1 != nullptr);
|
|
|
|
// Second session should fail
|
|
camera.SimulateUserGesture();
|
|
auto session2 = camera.StartSession(config, err);
|
|
EXPECT_TRUE(session2 == nullptr);
|
|
EXPECT_TRUE(err.find("active") != std::string::npos ||
|
|
err.find("already") != std::string::npos);
|
|
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 5: Stops On App Stop
|
|
|
|
```cpp
|
|
bool Test_CameraStopsOnAppStop(std::string& error_msg) {
|
|
PermissionGate permissions("test.app", {"camera"}, false);
|
|
permissions.Grant("camera");
|
|
|
|
mosis::CameraInterface camera("test.app", &permissions);
|
|
camera.SimulateUserGesture();
|
|
|
|
std::string err;
|
|
mosis::CameraConfig config;
|
|
auto session = camera.StartSession(config, err);
|
|
|
|
EXPECT_TRUE(session != nullptr);
|
|
EXPECT_TRUE(camera.HasActiveSession());
|
|
|
|
// Simulate app stop
|
|
camera.Shutdown();
|
|
|
|
EXPECT_FALSE(camera.HasActiveSession());
|
|
EXPECT_FALSE(camera.IsIndicatorVisible());
|
|
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 6: Lua Integration
|
|
|
|
```cpp
|
|
bool Test_CameraLuaIntegration(std::string& error_msg) {
|
|
SandboxContext ctx = TestContext();
|
|
ctx.permissions = {"camera"};
|
|
LuaSandbox sandbox(ctx);
|
|
|
|
PermissionGate permissions("test.app", {"camera"}, false);
|
|
permissions.Grant("camera");
|
|
|
|
mosis::CameraInterface camera("test.app", &permissions);
|
|
camera.SimulateUserGesture();
|
|
mosis::RegisterCameraAPI(sandbox.GetState(), &camera);
|
|
|
|
std::string script = R"lua(
|
|
-- Test that camera global exists
|
|
if not camera then
|
|
error("camera global not found")
|
|
end
|
|
if not camera.start then
|
|
error("camera.start not found")
|
|
end
|
|
if not camera.isActive then
|
|
error("camera.isActive not found")
|
|
end
|
|
|
|
-- isActive should be false initially
|
|
if camera.isActive() then
|
|
error("camera should not be active initially")
|
|
end
|
|
)lua";
|
|
|
|
bool ok = sandbox.LoadString(script, "camera_test");
|
|
if (!ok) {
|
|
error_msg = "Lua test failed: " + sandbox.GetLastError();
|
|
return false;
|
|
}
|
|
return true;
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Acceptance Criteria
|
|
|
|
All tests must pass:
|
|
|
|
- [x] `Test_CameraRequiresPermission` - Permission check works
|
|
- [x] `Test_CameraRequiresUserGesture` - User gesture required
|
|
- [x] `Test_CameraShowsIndicator` - Recording indicator shown/hidden
|
|
- [x] `Test_CameraSingleSession` - Only one session allowed
|
|
- [x] `Test_CameraStopsOnShutdown` - Cleanup on shutdown
|
|
- [x] `Test_CameraLuaIntegration` - Lua API works
|
|
|
|
---
|
|
|
|
## Dependencies
|
|
|
|
- Milestone 1 (LuaSandbox)
|
|
- Milestone 2 (PermissionGate + user gesture)
|
|
- Milestone 3 (AuditLog)
|
|
|
|
---
|
|
|
|
## Notes
|
|
|
|
### Desktop vs Android Implementation
|
|
|
|
For desktop testing, CameraInterface operates in mock mode:
|
|
- Permission and gesture checks run normally
|
|
- Indicator state is tracked but not displayed
|
|
- No actual camera hardware access
|
|
- Frames can be simulated for testing
|
|
|
|
On Android, the real implementation would:
|
|
1. Use Camera2 API through JNI
|
|
2. Display system-level recording indicator
|
|
3. Handle camera hardware lifecycle
|
|
4. Deliver frames through SurfaceTexture
|
|
|
|
### Security Considerations
|
|
|
|
1. **Permission**: Camera access is a dangerous permission requiring user grant
|
|
2. **User gesture**: Prevents background camera activation
|
|
3. **Indicator**: User always knows when camera is active
|
|
4. **Single session**: Prevents resource abuse
|
|
5. **Cleanup**: Sessions closed when app stops
|
|
|
|
### Future Integration
|
|
|
|
The camera will integrate with game engines to:
|
|
1. Stream frames to VR/AR display
|
|
2. Support mixed reality passthrough
|
|
3. Enable barcode/QR scanning
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
After Milestone 11 passes:
|
|
1. Milestone 12: Virtual Hardware - Microphone
|