477 lines
13 KiB
Markdown
477 lines
13 KiB
Markdown
# Milestone 8: SQLite Database
|
|
|
|
**Status**: Complete ✅
|
|
**Goal**: Per-app SQLite with injection prevention.
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
This milestone implements a sandboxed SQLite database for Lua apps:
|
|
- Per-app isolated database files
|
|
- SQL injection prevention via prepared statements
|
|
- SQLite authorizer to block dangerous operations
|
|
- Database size limits
|
|
|
|
### Key Deliverables
|
|
|
|
1. **DatabaseManager class** - SQLite wrapper with security
|
|
2. **Lua db API** - open, execute, query, close
|
|
3. **SQL authorizer** - Block ATTACH, DETACH, dangerous PRAGMAs
|
|
4. **Prepared statements** - Parameterized queries only
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
```
|
|
src/main/cpp/sandbox/
|
|
├── database_manager.h # NEW - DatabaseManager header
|
|
└── database_manager.cpp # NEW - DatabaseManager implementation
|
|
```
|
|
|
|
---
|
|
|
|
## Implementation Details
|
|
|
|
### 1. Database Path Mapping
|
|
|
|
```
|
|
Database Name → Physical Path
|
|
─────────────────────────────────────────────────────────────
|
|
"mydata" → <app_root>/db/mydata.db
|
|
"cache" → <app_root>/db/cache.db
|
|
```
|
|
|
|
For testing, `app_root` is configurable (e.g., `./test_apps/<app_id>/`).
|
|
|
|
### 2. DatabaseManager Class
|
|
|
|
```cpp
|
|
// database_manager.h
|
|
#pragma once
|
|
|
|
#include <string>
|
|
#include <vector>
|
|
#include <variant>
|
|
#include <optional>
|
|
#include <memory>
|
|
|
|
struct sqlite3;
|
|
struct lua_State;
|
|
|
|
namespace mosis {
|
|
|
|
// SQL value types
|
|
using SqlValue = std::variant<std::nullptr_t, int64_t, double, std::string, std::vector<uint8_t>>;
|
|
using SqlRow = std::vector<SqlValue>;
|
|
using SqlResult = std::vector<SqlRow>;
|
|
|
|
struct DatabaseLimits {
|
|
size_t max_database_size = 50 * 1024 * 1024; // 50 MB per database
|
|
int max_databases_per_app = 5; // Max open databases
|
|
int max_query_time_ms = 5000; // 5 second query timeout
|
|
int max_result_rows = 10000; // Max rows returned
|
|
};
|
|
|
|
class DatabaseHandle;
|
|
|
|
class DatabaseManager {
|
|
public:
|
|
DatabaseManager(const std::string& app_id,
|
|
const std::string& app_root,
|
|
const DatabaseLimits& limits = DatabaseLimits{});
|
|
~DatabaseManager();
|
|
|
|
// Database operations
|
|
std::shared_ptr<DatabaseHandle> Open(const std::string& name, std::string& error);
|
|
void CloseAll();
|
|
|
|
// Stats
|
|
size_t GetOpenDatabaseCount() const;
|
|
|
|
private:
|
|
std::string m_app_id;
|
|
std::string m_app_root;
|
|
DatabaseLimits m_limits;
|
|
std::vector<std::shared_ptr<DatabaseHandle>> m_databases;
|
|
|
|
std::string ResolvePath(const std::string& name);
|
|
bool ValidateName(const std::string& name, std::string& error);
|
|
};
|
|
|
|
class DatabaseHandle {
|
|
public:
|
|
DatabaseHandle(sqlite3* db, const std::string& path, const DatabaseLimits& limits);
|
|
~DatabaseHandle();
|
|
|
|
// Execute (INSERT, UPDATE, DELETE, CREATE, etc.)
|
|
bool Execute(const std::string& sql, const std::vector<SqlValue>& params, std::string& error);
|
|
|
|
// Query (SELECT)
|
|
std::optional<SqlResult> Query(const std::string& sql, const std::vector<SqlValue>& params,
|
|
std::string& error);
|
|
|
|
// Get last insert rowid
|
|
int64_t GetLastInsertRowId() const;
|
|
|
|
// Get affected rows
|
|
int GetChanges() const;
|
|
|
|
bool IsOpen() const { return m_db != nullptr; }
|
|
void Close();
|
|
|
|
private:
|
|
sqlite3* m_db;
|
|
std::string m_path;
|
|
DatabaseLimits m_limits;
|
|
|
|
static int Authorizer(void* user_data, int action, const char* arg1,
|
|
const char* arg2, const char* arg3, const char* arg4);
|
|
};
|
|
|
|
// Register db.* APIs as globals
|
|
void RegisterDatabaseAPI(lua_State* L, DatabaseManager* manager);
|
|
|
|
} // namespace mosis
|
|
```
|
|
|
|
### 3. SQLite Authorizer Rules
|
|
|
|
Block dangerous operations:
|
|
|
|
| Action | Decision | Reason |
|
|
|--------|----------|--------|
|
|
| ATTACH | DENY | Prevents accessing other databases |
|
|
| DETACH | DENY | Prevents detaching |
|
|
| PRAGMA (most) | DENY | Many PRAGMAs are dangerous |
|
|
| PRAGMA table_info | ALLOW | Safe introspection |
|
|
| PRAGMA index_list | ALLOW | Safe introspection |
|
|
| PRAGMA foreign_keys | ALLOW | Safe setting |
|
|
| Function (load_extension) | DENY | Prevents loading native code |
|
|
|
|
### 4. Lua API
|
|
|
|
```lua
|
|
-- Open database (creates if not exists)
|
|
local db, err = database.open("mydata")
|
|
if not db then
|
|
print("Error: " .. err)
|
|
return
|
|
end
|
|
|
|
-- Create table
|
|
local ok, err = db:execute([[
|
|
CREATE TABLE IF NOT EXISTS items (
|
|
id INTEGER PRIMARY KEY,
|
|
name TEXT NOT NULL,
|
|
price REAL
|
|
)
|
|
]])
|
|
|
|
-- Insert with parameters (prevents SQL injection)
|
|
db:execute("INSERT INTO items (name, price) VALUES (?, ?)", {"Widget", 19.99})
|
|
|
|
-- Query with parameters
|
|
local rows, err = db:query("SELECT * FROM items WHERE price > ?", {10.0})
|
|
if rows then
|
|
for _, row in ipairs(rows) do
|
|
print(row[1], row[2], row[3]) -- id, name, price
|
|
end
|
|
end
|
|
|
|
-- Get last insert ID
|
|
local id = db:lastInsertId()
|
|
|
|
-- Get affected rows
|
|
local changes = db:changes()
|
|
|
|
-- Close database
|
|
db:close()
|
|
```
|
|
|
|
### 5. Security Features
|
|
|
|
1. **Prepared Statements**: All queries use parameterized binding
|
|
2. **Authorizer**: Block dangerous SQL operations
|
|
3. **Path Sandboxing**: Database files only in app's db/ directory
|
|
4. **Size Limits**: Enforce max database size
|
|
5. **Query Timeout**: Prevent long-running queries
|
|
6. **Result Limits**: Cap number of returned rows
|
|
|
|
---
|
|
|
|
## Test Cases
|
|
|
|
### Test 1: Create and Query Tables
|
|
|
|
```cpp
|
|
bool Test_DatabaseCreatesTables(std::string& error_msg) {
|
|
std::string test_root = "test_db_app";
|
|
mosis::DatabaseManager manager("test.app", test_root);
|
|
|
|
std::string err;
|
|
auto db = manager.Open("test", err);
|
|
EXPECT_TRUE(db != nullptr);
|
|
|
|
// Create table
|
|
EXPECT_TRUE(db->Execute(
|
|
"CREATE TABLE IF NOT EXISTS items (id INTEGER PRIMARY KEY, name TEXT)",
|
|
{}, err));
|
|
|
|
// Insert
|
|
EXPECT_TRUE(db->Execute(
|
|
"INSERT INTO items (name) VALUES (?)",
|
|
{std::string("Test Item")}, err));
|
|
|
|
// Query
|
|
auto rows = db->Query("SELECT * FROM items", {}, err);
|
|
EXPECT_TRUE(rows.has_value());
|
|
EXPECT_TRUE(rows->size() == 1);
|
|
|
|
db->Close();
|
|
manager.CloseAll();
|
|
|
|
// Cleanup
|
|
std::filesystem::remove_all(test_root);
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 2: Prepared Statements Prevent Injection
|
|
|
|
```cpp
|
|
bool Test_DatabasePreparedStatements(std::string& error_msg) {
|
|
std::string test_root = "test_db_app";
|
|
mosis::DatabaseManager manager("test.app", test_root);
|
|
|
|
std::string err;
|
|
auto db = manager.Open("test", err);
|
|
EXPECT_TRUE(db != nullptr);
|
|
|
|
db->Execute("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)", {}, err);
|
|
db->Execute("INSERT INTO users (name) VALUES (?)", {std::string("Alice")}, err);
|
|
|
|
// Attempt SQL injection via parameter - should be safely escaped
|
|
std::string malicious = "'; DROP TABLE users; --";
|
|
auto rows = db->Query("SELECT * FROM users WHERE name = ?", {malicious}, err);
|
|
|
|
// Query should succeed (finding nothing) and table should still exist
|
|
EXPECT_TRUE(rows.has_value());
|
|
EXPECT_TRUE(rows->size() == 0);
|
|
|
|
// Table should still exist
|
|
auto check = db->Query("SELECT * FROM users", {}, err);
|
|
EXPECT_TRUE(check.has_value());
|
|
EXPECT_TRUE(check->size() == 1);
|
|
|
|
db->Close();
|
|
std::filesystem::remove_all(test_root);
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 3: Blocks ATTACH
|
|
|
|
```cpp
|
|
bool Test_DatabaseBlocksAttach(std::string& error_msg) {
|
|
std::string test_root = "test_db_app";
|
|
mosis::DatabaseManager manager("test.app", test_root);
|
|
|
|
std::string err;
|
|
auto db = manager.Open("test", err);
|
|
EXPECT_TRUE(db != nullptr);
|
|
|
|
// Try to attach another database - should fail
|
|
EXPECT_FALSE(db->Execute("ATTACH DATABASE '/etc/passwd' AS evil", {}, err));
|
|
EXPECT_TRUE(err.find("authorized") != std::string::npos ||
|
|
err.find("denied") != std::string::npos);
|
|
|
|
db->Close();
|
|
std::filesystem::remove_all(test_root);
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 4: Blocks Dangerous PRAGMA
|
|
|
|
```cpp
|
|
bool Test_DatabaseBlocksDangerousPragma(std::string& error_msg) {
|
|
std::string test_root = "test_db_app";
|
|
mosis::DatabaseManager manager("test.app", test_root);
|
|
|
|
std::string err;
|
|
auto db = manager.Open("test", err);
|
|
EXPECT_TRUE(db != nullptr);
|
|
|
|
// Try dangerous PRAGMAs - should fail
|
|
EXPECT_FALSE(db->Execute("PRAGMA journal_mode = OFF", {}, err));
|
|
EXPECT_FALSE(db->Execute("PRAGMA synchronous = OFF", {}, err));
|
|
|
|
// Safe PRAGMAs should work
|
|
auto rows = db->Query("PRAGMA table_info(sqlite_master)", {}, err);
|
|
EXPECT_TRUE(rows.has_value());
|
|
|
|
db->Close();
|
|
std::filesystem::remove_all(test_root);
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 5: Multiple Databases
|
|
|
|
```cpp
|
|
bool Test_DatabaseMultiple(std::string& error_msg) {
|
|
std::string test_root = "test_db_app";
|
|
mosis::DatabaseManager manager("test.app", test_root);
|
|
|
|
std::string err;
|
|
auto db1 = manager.Open("db1", err);
|
|
auto db2 = manager.Open("db2", err);
|
|
|
|
EXPECT_TRUE(db1 != nullptr);
|
|
EXPECT_TRUE(db2 != nullptr);
|
|
EXPECT_TRUE(manager.GetOpenDatabaseCount() == 2);
|
|
|
|
// They should be independent
|
|
db1->Execute("CREATE TABLE t1 (x INTEGER)", {}, err);
|
|
db2->Execute("CREATE TABLE t2 (y INTEGER)", {}, err);
|
|
|
|
// t1 shouldn't exist in db2
|
|
auto rows = db2->Query("SELECT * FROM t1", {}, err);
|
|
EXPECT_FALSE(rows.has_value()); // Should fail - table doesn't exist
|
|
|
|
manager.CloseAll();
|
|
std::filesystem::remove_all(test_root);
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 6: Lua Integration
|
|
|
|
```cpp
|
|
bool Test_DatabaseLuaIntegration(std::string& error_msg) {
|
|
SandboxContext ctx = TestContext();
|
|
LuaSandbox sandbox(ctx);
|
|
|
|
std::string test_root = "test_db_lua";
|
|
mosis::DatabaseManager manager("test.app", test_root);
|
|
mosis::RegisterDatabaseAPI(sandbox.GetState(), &manager);
|
|
|
|
std::string script = R"(
|
|
local db, err = database.open("test")
|
|
assert(db, "failed to open: " .. (err or ""))
|
|
|
|
-- Create table
|
|
local ok, err = db:execute("CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)")
|
|
assert(ok, "create failed: " .. (err or ""))
|
|
|
|
-- Insert
|
|
ok, err = db:execute("INSERT INTO items (name) VALUES (?)", {"Test"})
|
|
assert(ok, "insert failed: " .. (err or ""))
|
|
|
|
-- Query
|
|
local rows, err = db:query("SELECT * FROM items")
|
|
assert(rows, "query failed: " .. (err or ""))
|
|
assert(#rows == 1, "expected 1 row")
|
|
|
|
db:close()
|
|
)";
|
|
|
|
bool ok = sandbox.LoadString(script, "db_test");
|
|
manager.CloseAll();
|
|
std::filesystem::remove_all(test_root);
|
|
|
|
if (!ok) {
|
|
error_msg = "Lua test failed: " + sandbox.GetLastError();
|
|
return false;
|
|
}
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 7: Invalid Database Names
|
|
|
|
```cpp
|
|
bool Test_DatabaseInvalidNames(std::string& error_msg) {
|
|
std::string test_root = "test_db_app";
|
|
mosis::DatabaseManager manager("test.app", test_root);
|
|
|
|
std::string err;
|
|
|
|
// Path traversal
|
|
auto db1 = manager.Open("../evil", err);
|
|
EXPECT_TRUE(db1 == nullptr);
|
|
|
|
// Absolute path
|
|
auto db2 = manager.Open("/etc/passwd", err);
|
|
EXPECT_TRUE(db2 == nullptr);
|
|
|
|
// Special characters
|
|
auto db3 = manager.Open("test;drop", err);
|
|
EXPECT_TRUE(db3 == nullptr);
|
|
|
|
std::filesystem::remove_all(test_root);
|
|
return true;
|
|
}
|
|
```
|
|
|
|
### Test 8: Last Insert ID and Changes
|
|
|
|
```cpp
|
|
bool Test_DatabaseLastInsertAndChanges(std::string& error_msg) {
|
|
std::string test_root = "test_db_app";
|
|
mosis::DatabaseManager manager("test.app", test_root);
|
|
|
|
std::string err;
|
|
auto db = manager.Open("test", err);
|
|
EXPECT_TRUE(db != nullptr);
|
|
|
|
db->Execute("CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)", {}, err);
|
|
|
|
db->Execute("INSERT INTO items (name) VALUES (?)", {std::string("Item 1")}, err);
|
|
EXPECT_TRUE(db->GetLastInsertRowId() == 1);
|
|
EXPECT_TRUE(db->GetChanges() == 1);
|
|
|
|
db->Execute("INSERT INTO items (name) VALUES (?)", {std::string("Item 2")}, err);
|
|
EXPECT_TRUE(db->GetLastInsertRowId() == 2);
|
|
|
|
db->Execute("UPDATE items SET name = ?", {std::string("Updated")}, err);
|
|
EXPECT_TRUE(db->GetChanges() == 2); // Updated 2 rows
|
|
|
|
db->Close();
|
|
std::filesystem::remove_all(test_root);
|
|
return true;
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Acceptance Criteria
|
|
|
|
All tests must pass:
|
|
|
|
- [x] `Test_DatabaseCreatesTables` - Basic create/insert/query
|
|
- [x] `Test_DatabasePreparedStatements` - SQL injection prevention
|
|
- [x] `Test_DatabaseBlocksAttach` - ATTACH blocked
|
|
- [x] `Test_DatabaseBlocksDangerousPragma` - Dangerous PRAGMAs blocked
|
|
- [x] `Test_DatabaseMultiple` - Multiple isolated databases
|
|
- [x] `Test_DatabaseLuaIntegration` - Lua API works
|
|
- [x] `Test_DatabaseInvalidNames` - Path validation
|
|
- [x] `Test_DatabaseLastInsertAndChanges` - Metadata functions
|
|
|
|
---
|
|
|
|
## Dependencies
|
|
|
|
- Milestone 1 (LuaSandbox)
|
|
- SQLite3 library (add to vcpkg.json)
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
After Milestone 8 passes:
|
|
1. Milestone 9: Network - HTTP
|
|
2. Milestone 10: Network - WebSocket
|