MosisService/docs/SANDBOX_MILESTONE_13.md

Milestone 13: Virtual Hardware - Audio Output

Status: Complete Goal: Safe audio playback with volume limits and concurrent sound management.

---

Overview

This milestone implements secure audio output for Lua apps:

• No special permission required (normal capability)
• System volume limit (app cannot exceed system volume)
• Concurrent sound limit (10 per app)
• Automatic cleanup on app stop
• Support for PCM and basic audio formats

Key Deliverables

1. AudioOutputInterface class - Sound playback management
2. SoundPlayer class - Individual sound instance
3. Lua audio API - audio.play(), player methods
4. Volume enforcement - System volume cap

---

File Structure

src/main/cpp/sandbox/
├── audio_output.h      # NEW - Audio output API header
└── audio_output.cpp    # NEW - Audio output implementation

---

Implementation Details

1. AudioOutputInterface Class

// audio_output.h
#pragma once

#include <string>
#include <memory>
#include <functional>
#include <mutex>
#include <atomic>
#include <vector>
#include <unordered_map>

struct lua_State;

namespace mosis {

enum class AudioState {
    Playing,
    Paused,
    Stopped
};

struct PlaybackConfig {
    float volume = 1.0f;      // 0.0 to 1.0
    bool loop = false;
    float pitch = 1.0f;       // 0.5 to 2.0
    float pan = 0.0f;         // -1.0 (left) to 1.0 (right)
};

struct AudioData {
    std::vector<uint8_t> data;
    int sample_rate = 44100;
    int channels = 1;
    int bits_per_sample = 16;
};

class SoundPlayer {
public:
    using EndCallback = std::function<void()>;

    SoundPlayer(int id, const AudioData& audio, const PlaybackConfig& config);
    ~SoundPlayer();

    int GetId() const { return m_id; }
    AudioState GetState() const { return m_state; }
    float GetVolume() const { return m_volume; }
    bool IsLooping() const { return m_loop; }

    // Control
    void Play();
    void Pause();
    void Stop();
    void SetVolume(float volume);
    void SetPitch(float pitch);
    void SetPan(float pan);

    // Callback when playback ends
    void SetOnEnd(EndCallback cb) { m_on_end = std::move(cb); }

    // For mock mode - simulate playback completion
    void SimulateEnd();

    // Get playback position (0.0 to 1.0)
    float GetPosition() const { return m_position; }

private:
    int m_id;
    AudioData m_audio;
    std::atomic<AudioState> m_state{AudioState::Stopped};
    float m_volume = 1.0f;
    float m_pitch = 1.0f;
    float m_pan = 0.0f;
    bool m_loop = false;
    float m_position = 0.0f;
    EndCallback m_on_end;
    mutable std::mutex m_mutex;
};

class AudioOutputInterface {
public:
    AudioOutputInterface(const std::string& app_id);
    ~AudioOutputInterface();

    // Play a sound
    // Returns player on success, nullptr on failure (sets error)
    std::shared_ptr<SoundPlayer> Play(const AudioData& audio, const PlaybackConfig& config, std::string& error);

    // Stop a specific player
    void StopPlayer(int player_id);

    // Stop all sounds for this app
    void StopAll();

    // Get active player count
    size_t GetActivePlayerCount() const;

    // Get system volume (0.0 to 1.0)
    float GetSystemVolume() const { return m_system_volume; }

    // Set system volume (for testing)
    void SetSystemVolume(float volume);

    // Cleanup on app stop
    void Shutdown();

    // For testing
    void SetMockMode(bool enabled) { m_mock_mode = enabled; }
    bool IsMockMode() const { return m_mock_mode; }

    // Get player by ID
    std::shared_ptr<SoundPlayer> GetPlayer(int id);

private:
    std::string m_app_id;
    std::unordered_map<int, std::shared_ptr<SoundPlayer>> m_players;
    mutable std::mutex m_mutex;
    bool m_mock_mode = true;
    float m_system_volume = 1.0f;
    int m_next_player_id = 1;

    static constexpr int MAX_CONCURRENT_SOUNDS = 10;

    void CleanupStoppedPlayers();
};

// Register audio.* APIs as globals
void RegisterAudioAPI(lua_State* L, AudioOutputInterface* audio);

} // namespace mosis

2. Volume Enforcement

App volume is capped at system volume:

• Effective volume = app_volume × system_volume
• If system volume is 0.5 and app sets 1.0, actual volume is 0.5
• Apps cannot play louder than system setting

3. Concurrent Sound Limit

Maximum 10 simultaneous sounds per app:

• Prevents resource exhaustion
• New sounds fail when limit reached
• Stopped sounds don't count toward limit

4. Lua API

-- Play a sound
local player = audio.play(soundData, {
    volume = 0.8,       -- 0.0 to 1.0 (default 1.0)
    loop = false,       -- Loop playback (default false)
    pitch = 1.0,        -- Playback speed (default 1.0)
    pan = 0.0           -- -1.0 left, 0.0 center, 1.0 right
})

if not player then
    print("Failed to play sound")
    return
end

-- Player methods
player:pause()
player:resume()
player:stop()
player:setVolume(0.5)
player:setPitch(1.2)
player:setPan(-0.5)

-- Check state
local state = player:getState()  -- "playing", "paused", "stopped"
local pos = player:getPosition() -- 0.0 to 1.0

-- Event callback
player:on("ended", function()
    print("Sound finished playing")
end)

-- Stop all sounds
audio.stopAll()

-- Get active sound count
local count = audio.getActiveCount()

5. Audio Data Format

For mock mode testing, audio data is a table:

local soundData = {
    data = binaryString,     -- Raw PCM data
    sampleRate = 44100,      -- Hz
    channels = 1,            -- 1 (mono) or 2 (stereo)
    bitsPerSample = 16       -- 8 or 16
}

In real implementation, this would interface with Android AudioTrack or game engine audio systems.

---

Test Cases

Test 1: Basic Playback

bool Test_AudioPlaysSound(std::string& error_msg) {
    mosis::AudioOutputInterface audio("test.app");

    mosis::AudioData data;
    data.sample_rate = 44100;
    data.channels = 1;
    data.bits_per_sample = 16;
    data.data = std::vector<uint8_t>(1000);  // Dummy data

    mosis::PlaybackConfig config;
    std::string err;
    auto player = audio.Play(data, config, err);

    EXPECT_TRUE(player != nullptr);
    EXPECT_TRUE(player->GetState() == mosis::AudioState::Playing);

    return true;
}

Test 2: Volume Limit

bool Test_AudioRespectsSystemVolume(std::string& error_msg) {
    mosis::AudioOutputInterface audio("test.app");
    audio.SetSystemVolume(0.5f);

    mosis::AudioData data;
    data.data = std::vector<uint8_t>(1000);

    mosis::PlaybackConfig config;
    config.volume = 1.0f;  // App requests full volume

    std::string err;
    auto player = audio.Play(data, config, err);

    // Player is created with requested volume
    EXPECT_TRUE(player != nullptr);
    EXPECT_TRUE(player->GetVolume() == 1.0f);

    // But effective volume is capped by system
    // (In real implementation, audio subsystem applies this)

    return true;
}

Test 3: Concurrent Limit

bool Test_AudioLimitsConcurrent(std::string& error_msg) {
    mosis::AudioOutputInterface audio("test.app");

    mosis::AudioData data;
    data.data = std::vector<uint8_t>(1000);
    mosis::PlaybackConfig config;

    // Create MAX_CONCURRENT_SOUNDS players
    std::vector<std::shared_ptr<mosis::SoundPlayer>> players;
    for (int i = 0; i < 10; i++) {
        std::string err;
        auto player = audio.Play(data, config, err);
        EXPECT_TRUE(player != nullptr);
        players.push_back(player);
    }

    // 11th should fail
    std::string err;
    auto extra = audio.Play(data, config, err);
    EXPECT_TRUE(extra == nullptr);
    EXPECT_TRUE(err.find("limit") != std::string::npos ||
                err.find("concurrent") != std::string::npos);

    return true;
}

Test 4: Stop All

bool Test_AudioStopAll(std::string& error_msg) {
    mosis::AudioOutputInterface audio("test.app");

    mosis::AudioData data;
    data.data = std::vector<uint8_t>(1000);
    mosis::PlaybackConfig config;

    std::string err;
    auto player1 = audio.Play(data, config, err);
    auto player2 = audio.Play(data, config, err);

    EXPECT_TRUE(audio.GetActivePlayerCount() == 2);

    audio.StopAll();

    EXPECT_TRUE(audio.GetActivePlayerCount() == 0);

    return true;
}

Test 5: Cleanup on Shutdown

bool Test_AudioStopsOnShutdown(std::string& error_msg) {
    mosis::AudioOutputInterface audio("test.app");

    mosis::AudioData data;
    data.data = std::vector<uint8_t>(1000);
    mosis::PlaybackConfig config;

    std::string err;
    auto player = audio.Play(data, config, err);
    EXPECT_TRUE(player != nullptr);

    // Simulate app stop
    audio.Shutdown();

    EXPECT_TRUE(audio.GetActivePlayerCount() == 0);

    return true;
}

Test 6: Lua Integration

bool Test_AudioLuaIntegration(std::string& error_msg) {
    SandboxContext ctx = TestContext();
    LuaSandbox sandbox(ctx);

    mosis::AudioOutputInterface audio("test.app");
    mosis::RegisterAudioAPI(sandbox.GetState(), &audio);

    std::string script = R"lua(
        -- Test that audio global exists
        if not audio then
            error("audio global not found")
        end
        if not audio.play then
            error("audio.play not found")
        end
        if not audio.stopAll then
            error("audio.stopAll not found")
        end
        if not audio.getActiveCount then
            error("audio.getActiveCount not found")
        end

        -- Active count should be 0 initially
        if audio.getActiveCount() ~= 0 then
            error("should have no active sounds initially")
        end
    )lua";

    bool ok = sandbox.LoadString(script, "audio_test");
    if (!ok) {
        error_msg = "Lua test failed: " + sandbox.GetLastError();
        return false;
    }
    return true;
}

---

Acceptance Criteria

All tests must pass:

• Test_AudioPlaysSound - Basic playback works
• Test_AudioRespectsSystemVolume - Volume capped by system
• Test_AudioLimitsConcurrent - 10 sound limit enforced
• Test_AudioStopAll - Stop all sounds works
• Test_AudioStopsOnShutdown - Cleanup on shutdown
• Test_AudioLuaIntegration - Lua API works

---

Dependencies

• Milestone 1 (LuaSandbox)

---

Notes

Desktop vs Android Implementation

For desktop testing, AudioOutputInterface operates in mock mode:

• Players are created and tracked
• State transitions work (play/pause/stop)
• No actual audio hardware access
• Playback completion can be simulated for testing

On Android, the real implementation would:

1. Use AudioTrack API through JNI
2. Handle audio focus properly
3. Respect system volume settings
4. Mix multiple sounds appropriately

Security Considerations

1. No permission required: Audio playback is a normal capability
2. Volume limit: Cannot exceed system volume setting
3. Concurrent limit: Prevents resource exhaustion
4. Cleanup: All sounds stopped when app stops

Future Integration

Audio output will integrate with:

1. Game sound effects
2. UI feedback sounds
3. Music playback
4. Voice chat output (Milestone 12 companion)

---

Next Steps

After Milestone 13 passes:

1. Milestone 14: Virtual Hardware - Location