21 KiB
21 KiB
Milestone 9: App Review System
Status: Decided Goal: Automated and manual review process for app submissions.
Decision
Go validation workers + SQLite for self-hosted review pipeline:
Validation: Go workers with concurrent file processing
Storage: SQLite (review state in portal.db)
Queue: In-memory channel + SQLite persistence
UI: htmx server-rendered pages (admin section)
Rationale
- Go concurrency - Process multiple files in parallel with goroutines
- Single binary - No separate queue service needed
- Simple state - Review state in SQLite alongside app data
- htmx admin UI - Server-rendered review queue, no SPA needed
Architecture
┌─────────────────────────────────────────────────────────────────┐
│ mosis-portal container │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Go Binary │ │
│ │ ┌─────────────┐ ┌────────────────┐ │ │
│ │ │ Upload API │───►│ Review Service │ │ │
│ │ │ POST /v1/ │ │ - Queue submit │ │ │
│ │ │ versions │ │ - Track state │ │ │
│ │ └─────────────┘ └───────┬────────┘ │ │
│ │ │ │ │
│ │ ┌─────────────────────────▼────────────────────────────┐ │ │
│ │ │ Validation Worker Pool │ │ │
│ │ │ • Tier 1: Package validation (ZIP, manifest, sig) │ │ │
│ │ │ • Tier 2: Content validation (RML, RCSS, Lua) │ │ │
│ │ │ • Tier 3: Security analysis (patterns, perms) │ │ │
│ │ │ • Tier 4: Quality checks (description, icons) │ │ │
│ │ └───────────────────────────────────────────────────────┘ │ │
│ │ │ │ │
│ │ ┌─────────────────────────▼────────────────────────────┐ │ │
│ │ │ Admin Review UI (htmx) │ │ │
│ │ │ • /admin/review-queue │ │ │
│ │ │ • /admin/review/:id │ │ │
│ │ └───────────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────┬─────────────────────────────┘ │
│ │ │
│ /volume1/mosis/ │ │
│ ├── data/portal.db ◄───────────┘ │
│ └── packages/{dev}/{app}/{ver}/ (validation target) │
└─────────────────────────────────────────────────────────────────┘
Overview
The review system ensures apps meet quality and security standards before publication. Balances automation with manual review for edge cases.
Review Pipeline
┌─────────┐ ┌───────────┐ ┌─────────┐ ┌──────────┐ ┌───────────┐
│ Submit │──►│ Automated │──►│ Manual │──►│ Approved │──►│ Published │
│ │ │ Checks │ │ Review │ │ │ │ │
└─────────┘ └───────────┘ └─────────┘ └──────────┘ └───────────┘
│ │
▼ ▼
┌─────────┐ ┌─────────┐
│ Failed │ │Rejected │
│(auto-fix)│ │(feedback)│
└─────────┘ └─────────┘
Automated Checks
Tier 1: Package Validation (Blocking)
| Check | Description | Action on Fail |
|---|---|---|
| Valid ZIP | Package is valid ZIP format | Reject |
| Manifest exists | manifest.json at root | Reject |
| Manifest valid | JSON parses, required fields | Reject |
| Signature valid | Package signed with registered key | Reject |
| Entry exists | Entry point file exists | Reject |
| Size limits | Under max package size | Reject |
| No forbidden files | No .exe, .dll, etc. | Reject |
Tier 2: Content Validation (Blocking)
| Check | Description | Action on Fail |
|---|---|---|
| RML valid | All RML files parse | Reject |
| RCSS valid | All RCSS files parse | Reject |
| Lua syntax | All Lua files parse | Reject |
| Icons valid | Icons are valid images | Reject |
| Icon sizes | Required icon sizes present | Reject |
| Path safety | No path traversal attempts | Reject |
Tier 3: Security Analysis (Warning/Flag)
| Check | Description | Action on Fail |
|---|---|---|
| Dangerous patterns | Known malicious Lua patterns | Flag for review |
| Excessive permissions | Unusual permission combos | Flag for review |
| Obfuscated code | Heavily obfuscated Lua | Flag for review |
| External URLs | Hardcoded external URLs | Flag for review |
| Large assets | Unusually large files | Warning |
Tier 4: Quality Checks (Warning)
| Check | Description | Action on Fail |
|---|---|---|
| Description length | Meaningful description | Warning |
| Release notes | Non-empty release notes | Warning |
| Icon quality | Not placeholder/blank | Warning |
| Localization | Locale files complete | Warning |
Implementation
Validation Worker
type ValidationResult struct {
Passed bool
Errors []ValidationError
Warnings []ValidationWarning
Flags []ReviewFlag
}
type ValidationError struct {
Code string
Message string
File string
Line int
}
func ValidatePackage(packagePath string) ValidationResult {
result := ValidationResult{Passed: true}
// Tier 1: Package validation
if err := validateZip(packagePath); err != nil {
result.AddError("INVALID_ZIP", err.Error())
return result
}
manifest, err := extractManifest(packagePath)
if err != nil {
result.AddError("INVALID_MANIFEST", err.Error())
return result
}
if err := validateSignature(packagePath, manifest); err != nil {
result.AddError("INVALID_SIGNATURE", err.Error())
return result
}
// Tier 2: Content validation
files, _ := listFiles(packagePath)
for _, file := range files {
switch filepath.Ext(file) {
case ".rml":
if err := validateRML(file); err != nil {
result.AddError("INVALID_RML", err.Error(), file)
}
case ".rcss":
if err := validateRCSS(file); err != nil {
result.AddError("INVALID_RCSS", err.Error(), file)
}
case ".lua":
if err := validateLua(file); err != nil {
result.AddError("INVALID_LUA", err.Error(), file)
}
if flags := analyzeLuaSecurity(file); len(flags) > 0 {
result.Flags = append(result.Flags, flags...)
}
}
}
// Tier 3: Security analysis
if hasDangerousPatterns(files) {
result.AddFlag("DANGEROUS_PATTERNS", "Code contains suspicious patterns")
}
// Tier 4: Quality checks
if len(manifest.Description) < 10 {
result.AddWarning("SHORT_DESCRIPTION", "Description is very short")
}
result.Passed = len(result.Errors) == 0
return result
}
Dangerous Pattern Detection
var dangerousPatterns = []struct {
Pattern *regexp.Regexp
Reason string
}{
{
regexp.MustCompile(`loadstring\s*\(`),
"Dynamic code execution",
},
{
regexp.MustCompile(`debug\s*\.\s*\w+`),
"Debug library usage",
},
{
regexp.MustCompile(`os\s*\.\s*execute`),
"OS command execution",
},
{
regexp.MustCompile(`io\s*\.\s*\w+`),
"Direct I/O operations",
},
{
regexp.MustCompile(`ffi\s*\.\s*\w+`),
"FFI usage",
},
{
regexp.MustCompile(`package\s*\.\s*loadlib`),
"Native library loading",
},
}
func analyzeLuaSecurity(content string) []ReviewFlag {
var flags []ReviewFlag
for _, dp := range dangerousPatterns {
if dp.Pattern.MatchString(content) {
flags = append(flags, ReviewFlag{
Type: "SECURITY",
Reason: dp.Reason,
})
}
}
return flags
}
Manual Review
When Required
| Trigger | Reason |
|---|---|
| New developer | First app submission |
| Dangerous permissions | camera, microphone, contacts, location |
| Security flags | Automated checks flagged concerns |
| User reports | Existing app reported |
| Appeal | Developer contests rejection |
Review Queue UI
┌─────────────────────────────────────────────────────────────┐
│ Review Queue [14 pending] │
├─────────────────────────────────────────────────────────────┤
│ │
│ Filter: [All ▼] [Flagged first ▼] │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ 🚩 Weather Pro v2.0.0 │ │
│ │ com.newdev.weather • New developer │ │
│ │ Permissions: location, network │ │
│ │ Flags: First submission │ │
│ │ Submitted: 2 hours ago │ │
│ │ [Review] [Auto-approve]│ │
│ ├──────────────────────────────────────────────────────┤ │
│ │ 🚩 Photo Editor v1.5.0 │ │
│ │ com.trusted.photos • Verified developer │ │
│ │ Permissions: camera, storage │ │
│ │ Flags: DANGEROUS_PATTERNS (1) │ │
│ │ Submitted: 5 hours ago │ │
│ │ [Review] [Auto-approve]│ │
│ └──────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
Review Detail View
┌─────────────────────────────────────────────────────────────┐
│ Review: Weather Pro v2.0.0 │
├─────────────────────────────────────────────────────────────┤
│ │
│ Package: com.newdev.weather │
│ Developer: newdev@example.com (New - first app) │
│ Submitted: Jan 15, 2024 10:30 AM │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Automated Checks │ │
│ │ ✓ Package validation passed │ │
│ │ ✓ Content validation passed │ │
│ │ ⚠ Warning: SHORT_DESCRIPTION │ │
│ │ 🚩 Flag: First submission from new developer │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ Permissions Requested: │
│ • location (fine) - Used for weather forecasts │
│ • network - Required for API calls │
│ │
│ Files: [Expand to browse] │
│ ├── manifest.json │
│ ├── assets/ │
│ │ ├── main.rml │
│ │ └── scripts/ │
│ │ └── weather.lua [View source] │
│ └── icons/ │
│ │
│ Reviewer Notes: │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ [Approve] [Reject with feedback] [Request changes] │
│ │
└─────────────────────────────────────────────────────────────┘
Review States
┌─────────┐
│ Draft │
└────┬────┘
│ submit
▼
┌─────────┐
┌───────│Uploaded │
│ └────┬────┘
│ │ validation
│ ▼
│ ┌──────────┐
failed │ │Validating│
│ └────┬─────┘
│ │
│ ┌───────┴───────┐
│ │ │
▼ ▼ ▼
┌────────┐ ┌─────────┐
│ Failed │ │In Review│
└────────┘ └────┬────┘
│
┌───────────┼───────────┐
│ │ │
▼ ▼ ▼
┌──────────┐ ┌────────┐ ┌──────────┐
│ Approved │ │Rejected│ │ Changes │
└────┬─────┘ └────────┘ │ Requested│
│ └──────────┘
│ publish
▼
┌──────────┐
│Published │
└──────────┘
Rejection Feedback
Feedback Template
{
"reason": "SECURITY_CONCERN",
"message": "Your app contains patterns that raise security concerns.",
"details": [
{
"file": "scripts/main.lua",
"line": 42,
"issue": "Usage of loadstring() is not allowed",
"suggestion": "Use static Lua code instead of dynamic evaluation"
}
],
"can_resubmit": true,
"appeal_available": true
}
Common Rejection Reasons
| Code | Description |
|---|---|
SECURITY_CONCERN |
Security issues found |
QUALITY_ISSUE |
Doesn't meet quality standards |
POLICY_VIOLATION |
Violates content policy |
METADATA_ISSUE |
Incorrect/misleading metadata |
PERMISSION_ABUSE |
Unnecessary permissions |
COPYRIGHT |
Copyright/trademark issues |
Appeal Process
1. Developer receives rejection
2. Developer clicks "Appeal" (within 14 days)
3. Provides justification
4. Different reviewer examines
5. Final decision (approve/uphold rejection)
Review SLA
| Submission Type | Target Time |
|---|---|
| Auto-approved | Instant |
| New developer | 24 hours |
| Flagged | 48 hours |
| Appeal | 72 hours |
API Endpoints
# Developer endpoints
POST /v1/apps/:id/versions/:vid/submit:
summary: Submit for review
response: { version, estimated_review_time }
GET /v1/apps/:id/versions/:vid/review-status:
summary: Get review status
response: { status, feedback?, estimated_completion }
POST /v1/apps/:id/versions/:vid/appeal:
summary: Appeal rejection
body: { justification }
response: { appeal_id, status }
# Internal review endpoints (admin only)
GET /v1/admin/review-queue:
summary: Get pending reviews
response: { items[], total }
GET /v1/admin/review/:version_id:
summary: Get review details
response: { version, validation_result, flags }
POST /v1/admin/review/:version_id/approve:
summary: Approve version
body: { notes? }
response: { version }
POST /v1/admin/review/:version_id/reject:
summary: Reject version
body: { reason, message, details[] }
response: { version }
Metrics
Review Performance
| Metric | Target |
|---|---|
| Auto-approval rate | > 70% |
| Average review time | < 24 hours |
| Rejection rate | < 20% |
| Appeal overturn rate | < 10% |
Dashboard
-- Review stats
SELECT
DATE_TRUNC('week', submitted_at) as week,
COUNT(*) as submissions,
COUNT(*) FILTER (WHERE status = 'published') as approved,
COUNT(*) FILTER (WHERE status = 'rejected') as rejected,
AVG(EXTRACT(EPOCH FROM (reviewed_at - submitted_at))/3600) as avg_hours
FROM app_versions
WHERE submitted_at > NOW() - INTERVAL '30 days'
GROUP BY week;
Deliverables
- Review approach decided (Go workers + SQLite + htmx admin)
- Validation worker implementation (Go concurrent file processing)
- Dangerous pattern database (regex patterns in code)
- Review queue UI (htmx server-rendered)
- Reviewer tools (file browser, source viewer)
- Rejection feedback system
- Appeal workflow
- Review metrics queries
Open Questions
Automated approval for trusted developers?→ Yes, after 3+ approved appsCommunity moderators?→ Defer to post-MVP (single admin for now)- Content policy document? → Create during M12 Docs
Rate limiting resubmissions?→ Max 3 resubmits per day per app