move docs to docs/ folder, merge architecture files, update references
This commit is contained in:
521
docs/SANDBOX_MILESTONE_11.md
Normal file
521
docs/SANDBOX_MILESTONE_11.md
Normal file
@@ -0,0 +1,521 @@
|
||||
# Milestone 11: Virtual Hardware - Camera
|
||||
|
||||
**Status**: Complete
|
||||
**Goal**: Camera access with mandatory recording indicators and security controls.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This milestone implements secure camera access for Lua apps:
|
||||
- Permission required (`camera` permission)
|
||||
- User gesture required to start session
|
||||
- Mandatory recording indicator (system-controlled, cannot be hidden)
|
||||
- Single session per app
|
||||
- Frame rate limiting
|
||||
- Automatic cleanup on app stop
|
||||
|
||||
### Key Deliverables
|
||||
|
||||
1. **CameraInterface class** - Session management, permission checks
|
||||
2. **CameraSession class** - Active camera session wrapper
|
||||
3. **Lua camera API** - `camera.start()`, session methods
|
||||
4. **Recording indicator** - System-level UI notification
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
src/main/cpp/sandbox/
|
||||
├── camera_interface.h # NEW - Camera API header
|
||||
└── camera_interface.cpp # NEW - Camera implementation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### 1. CameraInterface Class
|
||||
|
||||
```cpp
|
||||
// camera_interface.h
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
#include <memory>
|
||||
#include <functional>
|
||||
#include <mutex>
|
||||
#include <atomic>
|
||||
#include <vector>
|
||||
|
||||
struct lua_State;
|
||||
|
||||
namespace mosis {
|
||||
|
||||
class PermissionGate;
|
||||
|
||||
enum class CameraFacing {
|
||||
Front,
|
||||
Back
|
||||
};
|
||||
|
||||
enum class CameraResolution {
|
||||
VGA, // 640x480
|
||||
HD, // 1280x720
|
||||
FullHD // 1920x1080
|
||||
};
|
||||
|
||||
struct CameraConfig {
|
||||
CameraFacing facing = CameraFacing::Back;
|
||||
CameraResolution resolution = CameraResolution::HD;
|
||||
int max_fps = 30;
|
||||
};
|
||||
|
||||
struct CameraFrame {
|
||||
std::vector<uint8_t> data; // RGBA or JPEG data
|
||||
int width;
|
||||
int height;
|
||||
uint64_t timestamp_ms;
|
||||
bool is_jpeg;
|
||||
};
|
||||
|
||||
class CameraSession {
|
||||
public:
|
||||
using FrameCallback = std::function<void(const CameraFrame& frame)>;
|
||||
|
||||
CameraSession(int id, const CameraConfig& config);
|
||||
~CameraSession();
|
||||
|
||||
int GetId() const { return m_id; }
|
||||
const CameraConfig& GetConfig() const { return m_config; }
|
||||
bool IsActive() const { return m_active; }
|
||||
|
||||
// Capture single photo (returns copy of current frame)
|
||||
CameraFrame Capture();
|
||||
|
||||
// Set frame callback (for preview)
|
||||
void SetOnFrame(FrameCallback cb) { m_on_frame = std::move(cb); }
|
||||
|
||||
// Stop session
|
||||
void Stop();
|
||||
|
||||
// For mock mode - simulate frame arrival
|
||||
void SimulateFrame(const CameraFrame& frame);
|
||||
|
||||
private:
|
||||
int m_id;
|
||||
CameraConfig m_config;
|
||||
std::atomic<bool> m_active{true};
|
||||
FrameCallback m_on_frame;
|
||||
CameraFrame m_last_frame;
|
||||
std::mutex m_mutex;
|
||||
};
|
||||
|
||||
class CameraInterface {
|
||||
public:
|
||||
CameraInterface(const std::string& app_id, PermissionGate* permissions);
|
||||
~CameraInterface();
|
||||
|
||||
// Start camera session
|
||||
// Returns session on success, nullptr on failure (sets error)
|
||||
// Requires camera permission and user gesture
|
||||
std::shared_ptr<CameraSession> StartSession(const CameraConfig& config, std::string& error);
|
||||
|
||||
// Stop active session
|
||||
void StopSession();
|
||||
|
||||
// Check if session is active
|
||||
bool HasActiveSession() const;
|
||||
|
||||
// Check if recording indicator should be shown
|
||||
bool IsIndicatorVisible() const;
|
||||
|
||||
// Cleanup on app stop
|
||||
void Shutdown();
|
||||
|
||||
// For testing
|
||||
void SetMockMode(bool enabled) { m_mock_mode = enabled; }
|
||||
bool IsMockMode() const { return m_mock_mode; }
|
||||
|
||||
// Simulate user gesture for testing
|
||||
void SimulateUserGesture();
|
||||
|
||||
private:
|
||||
std::string m_app_id;
|
||||
PermissionGate* m_permissions;
|
||||
std::shared_ptr<CameraSession> m_active_session;
|
||||
mutable std::mutex m_mutex;
|
||||
bool m_mock_mode = true;
|
||||
std::atomic<bool> m_indicator_visible{false};
|
||||
|
||||
// Track user gesture timing
|
||||
uint64_t m_last_gesture_time = 0;
|
||||
static constexpr uint64_t GESTURE_VALIDITY_MS = 5000; // 5 seconds
|
||||
|
||||
bool HasRecentUserGesture() const;
|
||||
};
|
||||
|
||||
// Register camera.* APIs as globals
|
||||
void RegisterCameraAPI(lua_State* L, CameraInterface* camera);
|
||||
|
||||
} // namespace mosis
|
||||
```
|
||||
|
||||
### 2. Permission Requirements
|
||||
|
||||
Camera access requires:
|
||||
1. `camera` permission declared in manifest
|
||||
2. Permission granted by user (dangerous permission)
|
||||
3. Recent user gesture (within 5 seconds)
|
||||
|
||||
```cpp
|
||||
// Permission check flow
|
||||
bool CameraInterface::StartSession(...) {
|
||||
// 1. Check permission
|
||||
if (!m_permissions->Check("camera")) {
|
||||
error = "Camera permission not granted";
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
// 2. Check user gesture
|
||||
if (!HasRecentUserGesture()) {
|
||||
error = "Camera requires user gesture";
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
// 3. Check no existing session
|
||||
if (m_active_session) {
|
||||
error = "Camera session already active";
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
// ... create session
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Recording Indicator
|
||||
|
||||
The recording indicator is mandatory and system-controlled:
|
||||
- Shown whenever camera session is active
|
||||
- Cannot be hidden or obscured by app
|
||||
- Positioned in system UI area
|
||||
- Shows camera icon with "Recording" text
|
||||
|
||||
```cpp
|
||||
// Indicator control
|
||||
void CameraInterface::ShowIndicator() {
|
||||
m_indicator_visible = true;
|
||||
// In real implementation: notify system UI
|
||||
}
|
||||
|
||||
void CameraInterface::HideIndicator() {
|
||||
m_indicator_visible = false;
|
||||
// In real implementation: notify system UI
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Lua API
|
||||
|
||||
```lua
|
||||
-- Start camera session (requires permission + user gesture)
|
||||
local session, err = camera.start({
|
||||
facing = "back", -- "front" or "back"
|
||||
resolution = "720p", -- "480p", "720p", or "1080p"
|
||||
fps = 30 -- max frames per second
|
||||
})
|
||||
|
||||
if not session then
|
||||
print("Failed to start camera:", err)
|
||||
return
|
||||
end
|
||||
|
||||
-- Capture single photo
|
||||
local photo = session:capture()
|
||||
if photo then
|
||||
print("Captured", photo.width, "x", photo.height, "image")
|
||||
-- photo.data contains JPEG bytes
|
||||
end
|
||||
|
||||
-- Set frame callback (for preview)
|
||||
session:on("frame", function(frame)
|
||||
-- frame.data, frame.width, frame.height, frame.timestamp
|
||||
end)
|
||||
|
||||
-- Stop session
|
||||
session:stop()
|
||||
|
||||
-- Check if camera is active
|
||||
if camera.isActive() then
|
||||
print("Camera is running")
|
||||
end
|
||||
```
|
||||
|
||||
### 5. Frame Rate Limiting
|
||||
|
||||
To prevent resource abuse:
|
||||
- Maximum 30 FPS
|
||||
- Minimum 16ms between frames
|
||||
- Frame delivery is asynchronous
|
||||
|
||||
```cpp
|
||||
struct FrameRateLimiter {
|
||||
int max_fps = 30;
|
||||
uint64_t min_frame_interval_ms = 33; // ~30 FPS
|
||||
uint64_t last_frame_time = 0;
|
||||
|
||||
bool ShouldDeliverFrame() {
|
||||
uint64_t now = GetCurrentTimeMs();
|
||||
if (now - last_frame_time >= min_frame_interval_ms) {
|
||||
last_frame_time = now;
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### 6. Session Lifecycle
|
||||
|
||||
```
|
||||
User Gesture ──► camera.start() ──► Session Active ──► Indicator Shown
|
||||
│ │
|
||||
│ error │ session:stop()
|
||||
▼ ▼
|
||||
nil, err Session Closed ──► Indicator Hidden
|
||||
|
||||
App Stop ──► CameraInterface::Shutdown() ──► All Sessions Closed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Test 1: Requires Permission
|
||||
|
||||
```cpp
|
||||
bool Test_CameraRequiresPermission(std::string& error_msg) {
|
||||
// Create permission gate WITHOUT camera permission
|
||||
PermissionGate permissions("test.app", {}, false);
|
||||
|
||||
mosis::CameraInterface camera("test.app", &permissions);
|
||||
camera.SimulateUserGesture();
|
||||
|
||||
std::string err;
|
||||
mosis::CameraConfig config;
|
||||
auto session = camera.StartSession(config, err);
|
||||
|
||||
EXPECT_TRUE(session == nullptr);
|
||||
EXPECT_TRUE(err.find("permission") != std::string::npos);
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Test 2: Requires User Gesture
|
||||
|
||||
```cpp
|
||||
bool Test_CameraRequiresUserGesture(std::string& error_msg) {
|
||||
// Create permission gate WITH camera permission
|
||||
PermissionGate permissions("test.app", {"camera"}, false);
|
||||
permissions.Grant("camera");
|
||||
|
||||
mosis::CameraInterface camera("test.app", &permissions);
|
||||
// Note: NOT calling SimulateUserGesture()
|
||||
|
||||
std::string err;
|
||||
mosis::CameraConfig config;
|
||||
auto session = camera.StartSession(config, err);
|
||||
|
||||
EXPECT_TRUE(session == nullptr);
|
||||
EXPECT_TRUE(err.find("gesture") != std::string::npos);
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Test 3: Shows Indicator
|
||||
|
||||
```cpp
|
||||
bool Test_CameraShowsIndicator(std::string& error_msg) {
|
||||
PermissionGate permissions("test.app", {"camera"}, false);
|
||||
permissions.Grant("camera");
|
||||
|
||||
mosis::CameraInterface camera("test.app", &permissions);
|
||||
camera.SimulateUserGesture();
|
||||
|
||||
EXPECT_FALSE(camera.IsIndicatorVisible());
|
||||
|
||||
std::string err;
|
||||
mosis::CameraConfig config;
|
||||
auto session = camera.StartSession(config, err);
|
||||
|
||||
// Indicator should be visible while session is active
|
||||
EXPECT_TRUE(camera.IsIndicatorVisible());
|
||||
|
||||
session->Stop();
|
||||
|
||||
// Indicator should be hidden after stop
|
||||
EXPECT_FALSE(camera.IsIndicatorVisible());
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Test 4: Single Session Only
|
||||
|
||||
```cpp
|
||||
bool Test_CameraSingleSession(std::string& error_msg) {
|
||||
PermissionGate permissions("test.app", {"camera"}, false);
|
||||
permissions.Grant("camera");
|
||||
|
||||
mosis::CameraInterface camera("test.app", &permissions);
|
||||
camera.SimulateUserGesture();
|
||||
|
||||
std::string err;
|
||||
mosis::CameraConfig config;
|
||||
|
||||
// First session should succeed
|
||||
auto session1 = camera.StartSession(config, err);
|
||||
EXPECT_TRUE(session1 != nullptr);
|
||||
|
||||
// Second session should fail
|
||||
camera.SimulateUserGesture();
|
||||
auto session2 = camera.StartSession(config, err);
|
||||
EXPECT_TRUE(session2 == nullptr);
|
||||
EXPECT_TRUE(err.find("active") != std::string::npos ||
|
||||
err.find("already") != std::string::npos);
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Test 5: Stops On App Stop
|
||||
|
||||
```cpp
|
||||
bool Test_CameraStopsOnAppStop(std::string& error_msg) {
|
||||
PermissionGate permissions("test.app", {"camera"}, false);
|
||||
permissions.Grant("camera");
|
||||
|
||||
mosis::CameraInterface camera("test.app", &permissions);
|
||||
camera.SimulateUserGesture();
|
||||
|
||||
std::string err;
|
||||
mosis::CameraConfig config;
|
||||
auto session = camera.StartSession(config, err);
|
||||
|
||||
EXPECT_TRUE(session != nullptr);
|
||||
EXPECT_TRUE(camera.HasActiveSession());
|
||||
|
||||
// Simulate app stop
|
||||
camera.Shutdown();
|
||||
|
||||
EXPECT_FALSE(camera.HasActiveSession());
|
||||
EXPECT_FALSE(camera.IsIndicatorVisible());
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### Test 6: Lua Integration
|
||||
|
||||
```cpp
|
||||
bool Test_CameraLuaIntegration(std::string& error_msg) {
|
||||
SandboxContext ctx = TestContext();
|
||||
ctx.permissions = {"camera"};
|
||||
LuaSandbox sandbox(ctx);
|
||||
|
||||
PermissionGate permissions("test.app", {"camera"}, false);
|
||||
permissions.Grant("camera");
|
||||
|
||||
mosis::CameraInterface camera("test.app", &permissions);
|
||||
camera.SimulateUserGesture();
|
||||
mosis::RegisterCameraAPI(sandbox.GetState(), &camera);
|
||||
|
||||
std::string script = R"lua(
|
||||
-- Test that camera global exists
|
||||
if not camera then
|
||||
error("camera global not found")
|
||||
end
|
||||
if not camera.start then
|
||||
error("camera.start not found")
|
||||
end
|
||||
if not camera.isActive then
|
||||
error("camera.isActive not found")
|
||||
end
|
||||
|
||||
-- isActive should be false initially
|
||||
if camera.isActive() then
|
||||
error("camera should not be active initially")
|
||||
end
|
||||
)lua";
|
||||
|
||||
bool ok = sandbox.LoadString(script, "camera_test");
|
||||
if (!ok) {
|
||||
error_msg = "Lua test failed: " + sandbox.GetLastError();
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
All tests must pass:
|
||||
|
||||
- [x] `Test_CameraRequiresPermission` - Permission check works
|
||||
- [x] `Test_CameraRequiresUserGesture` - User gesture required
|
||||
- [x] `Test_CameraShowsIndicator` - Recording indicator shown/hidden
|
||||
- [x] `Test_CameraSingleSession` - Only one session allowed
|
||||
- [x] `Test_CameraStopsOnShutdown` - Cleanup on shutdown
|
||||
- [x] `Test_CameraLuaIntegration` - Lua API works
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Milestone 1 (LuaSandbox)
|
||||
- Milestone 2 (PermissionGate + user gesture)
|
||||
- Milestone 3 (AuditLog)
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
### Desktop vs Android Implementation
|
||||
|
||||
For desktop testing, CameraInterface operates in mock mode:
|
||||
- Permission and gesture checks run normally
|
||||
- Indicator state is tracked but not displayed
|
||||
- No actual camera hardware access
|
||||
- Frames can be simulated for testing
|
||||
|
||||
On Android, the real implementation would:
|
||||
1. Use Camera2 API through JNI
|
||||
2. Display system-level recording indicator
|
||||
3. Handle camera hardware lifecycle
|
||||
4. Deliver frames through SurfaceTexture
|
||||
|
||||
### Security Considerations
|
||||
|
||||
1. **Permission**: Camera access is a dangerous permission requiring user grant
|
||||
2. **User gesture**: Prevents background camera activation
|
||||
3. **Indicator**: User always knows when camera is active
|
||||
4. **Single session**: Prevents resource abuse
|
||||
5. **Cleanup**: Sessions closed when app stops
|
||||
|
||||
### Future Integration
|
||||
|
||||
The camera will integrate with game engines to:
|
||||
1. Stream frames to VR/AR display
|
||||
2. Support mixed reality passthrough
|
||||
3. Enable barcode/QR scanning
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
After Milestone 11 passes:
|
||||
1. Milestone 12: Virtual Hardware - Microphone
|
||||
Reference in New Issue
Block a user