17 KiB
17 KiB
Milestone 9: App Review System
Status: Planning Goal: Automated and manual review process for app submissions.
Overview
The review system ensures apps meet quality and security standards before publication. Balances automation with manual review for edge cases.
Review Pipeline
┌─────────┐ ┌───────────┐ ┌─────────┐ ┌──────────┐ ┌───────────┐
│ Submit │──►│ Automated │──►│ Manual │──►│ Approved │──►│ Published │
│ │ │ Checks │ │ Review │ │ │ │ │
└─────────┘ └───────────┘ └─────────┘ └──────────┘ └───────────┘
│ │
▼ ▼
┌─────────┐ ┌─────────┐
│ Failed │ │Rejected │
│(auto-fix)│ │(feedback)│
└─────────┘ └─────────┘
Automated Checks
Tier 1: Package Validation (Blocking)
| Check | Description | Action on Fail |
|---|---|---|
| Valid ZIP | Package is valid ZIP format | Reject |
| Manifest exists | manifest.json at root | Reject |
| Manifest valid | JSON parses, required fields | Reject |
| Signature valid | Package signed with registered key | Reject |
| Entry exists | Entry point file exists | Reject |
| Size limits | Under max package size | Reject |
| No forbidden files | No .exe, .dll, etc. | Reject |
Tier 2: Content Validation (Blocking)
| Check | Description | Action on Fail |
|---|---|---|
| RML valid | All RML files parse | Reject |
| RCSS valid | All RCSS files parse | Reject |
| Lua syntax | All Lua files parse | Reject |
| Icons valid | Icons are valid images | Reject |
| Icon sizes | Required icon sizes present | Reject |
| Path safety | No path traversal attempts | Reject |
Tier 3: Security Analysis (Warning/Flag)
| Check | Description | Action on Fail |
|---|---|---|
| Dangerous patterns | Known malicious Lua patterns | Flag for review |
| Excessive permissions | Unusual permission combos | Flag for review |
| Obfuscated code | Heavily obfuscated Lua | Flag for review |
| External URLs | Hardcoded external URLs | Flag for review |
| Large assets | Unusually large files | Warning |
Tier 4: Quality Checks (Warning)
| Check | Description | Action on Fail |
|---|---|---|
| Description length | Meaningful description | Warning |
| Release notes | Non-empty release notes | Warning |
| Icon quality | Not placeholder/blank | Warning |
| Localization | Locale files complete | Warning |
Implementation
Validation Worker
type ValidationResult struct {
Passed bool
Errors []ValidationError
Warnings []ValidationWarning
Flags []ReviewFlag
}
type ValidationError struct {
Code string
Message string
File string
Line int
}
func ValidatePackage(packagePath string) ValidationResult {
result := ValidationResult{Passed: true}
// Tier 1: Package validation
if err := validateZip(packagePath); err != nil {
result.AddError("INVALID_ZIP", err.Error())
return result
}
manifest, err := extractManifest(packagePath)
if err != nil {
result.AddError("INVALID_MANIFEST", err.Error())
return result
}
if err := validateSignature(packagePath, manifest); err != nil {
result.AddError("INVALID_SIGNATURE", err.Error())
return result
}
// Tier 2: Content validation
files, _ := listFiles(packagePath)
for _, file := range files {
switch filepath.Ext(file) {
case ".rml":
if err := validateRML(file); err != nil {
result.AddError("INVALID_RML", err.Error(), file)
}
case ".rcss":
if err := validateRCSS(file); err != nil {
result.AddError("INVALID_RCSS", err.Error(), file)
}
case ".lua":
if err := validateLua(file); err != nil {
result.AddError("INVALID_LUA", err.Error(), file)
}
if flags := analyzeLuaSecurity(file); len(flags) > 0 {
result.Flags = append(result.Flags, flags...)
}
}
}
// Tier 3: Security analysis
if hasDangerousPatterns(files) {
result.AddFlag("DANGEROUS_PATTERNS", "Code contains suspicious patterns")
}
// Tier 4: Quality checks
if len(manifest.Description) < 10 {
result.AddWarning("SHORT_DESCRIPTION", "Description is very short")
}
result.Passed = len(result.Errors) == 0
return result
}
Dangerous Pattern Detection
var dangerousPatterns = []struct {
Pattern *regexp.Regexp
Reason string
}{
{
regexp.MustCompile(`loadstring\s*\(`),
"Dynamic code execution",
},
{
regexp.MustCompile(`debug\s*\.\s*\w+`),
"Debug library usage",
},
{
regexp.MustCompile(`os\s*\.\s*execute`),
"OS command execution",
},
{
regexp.MustCompile(`io\s*\.\s*\w+`),
"Direct I/O operations",
},
{
regexp.MustCompile(`ffi\s*\.\s*\w+`),
"FFI usage",
},
{
regexp.MustCompile(`package\s*\.\s*loadlib`),
"Native library loading",
},
}
func analyzeLuaSecurity(content string) []ReviewFlag {
var flags []ReviewFlag
for _, dp := range dangerousPatterns {
if dp.Pattern.MatchString(content) {
flags = append(flags, ReviewFlag{
Type: "SECURITY",
Reason: dp.Reason,
})
}
}
return flags
}
Manual Review
When Required
| Trigger | Reason |
|---|---|
| New developer | First app submission |
| Dangerous permissions | camera, microphone, contacts, location |
| Security flags | Automated checks flagged concerns |
| User reports | Existing app reported |
| Appeal | Developer contests rejection |
Review Queue UI
┌─────────────────────────────────────────────────────────────┐
│ Review Queue [14 pending] │
├─────────────────────────────────────────────────────────────┤
│ │
│ Filter: [All ▼] [Flagged first ▼] │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ 🚩 Weather Pro v2.0.0 │ │
│ │ com.newdev.weather • New developer │ │
│ │ Permissions: location, network │ │
│ │ Flags: First submission │ │
│ │ Submitted: 2 hours ago │ │
│ │ [Review] [Auto-approve]│ │
│ ├──────────────────────────────────────────────────────┤ │
│ │ 🚩 Photo Editor v1.5.0 │ │
│ │ com.trusted.photos • Verified developer │ │
│ │ Permissions: camera, storage │ │
│ │ Flags: DANGEROUS_PATTERNS (1) │ │
│ │ Submitted: 5 hours ago │ │
│ │ [Review] [Auto-approve]│ │
│ └──────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
Review Detail View
┌─────────────────────────────────────────────────────────────┐
│ Review: Weather Pro v2.0.0 │
├─────────────────────────────────────────────────────────────┤
│ │
│ Package: com.newdev.weather │
│ Developer: newdev@example.com (New - first app) │
│ Submitted: Jan 15, 2024 10:30 AM │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Automated Checks │ │
│ │ ✓ Package validation passed │ │
│ │ ✓ Content validation passed │ │
│ │ ⚠ Warning: SHORT_DESCRIPTION │ │
│ │ 🚩 Flag: First submission from new developer │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ Permissions Requested: │
│ • location (fine) - Used for weather forecasts │
│ • network - Required for API calls │
│ │
│ Files: [Expand to browse] │
│ ├── manifest.json │
│ ├── assets/ │
│ │ ├── main.rml │
│ │ └── scripts/ │
│ │ └── weather.lua [View source] │
│ └── icons/ │
│ │
│ Reviewer Notes: │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ [Approve] [Reject with feedback] [Request changes] │
│ │
└─────────────────────────────────────────────────────────────┘
Review States
┌─────────┐
│ Draft │
└────┬────┘
│ submit
▼
┌─────────┐
┌───────│Uploaded │
│ └────┬────┘
│ │ validation
│ ▼
│ ┌──────────┐
failed │ │Validating│
│ └────┬─────┘
│ │
│ ┌───────┴───────┐
│ │ │
▼ ▼ ▼
┌────────┐ ┌─────────┐
│ Failed │ │In Review│
└────────┘ └────┬────┘
│
┌───────────┼───────────┐
│ │ │
▼ ▼ ▼
┌──────────┐ ┌────────┐ ┌──────────┐
│ Approved │ │Rejected│ │ Changes │
└────┬─────┘ └────────┘ │ Requested│
│ └──────────┘
│ publish
▼
┌──────────┐
│Published │
└──────────┘
Rejection Feedback
Feedback Template
{
"reason": "SECURITY_CONCERN",
"message": "Your app contains patterns that raise security concerns.",
"details": [
{
"file": "scripts/main.lua",
"line": 42,
"issue": "Usage of loadstring() is not allowed",
"suggestion": "Use static Lua code instead of dynamic evaluation"
}
],
"can_resubmit": true,
"appeal_available": true
}
Common Rejection Reasons
| Code | Description |
|---|---|
SECURITY_CONCERN |
Security issues found |
QUALITY_ISSUE |
Doesn't meet quality standards |
POLICY_VIOLATION |
Violates content policy |
METADATA_ISSUE |
Incorrect/misleading metadata |
PERMISSION_ABUSE |
Unnecessary permissions |
COPYRIGHT |
Copyright/trademark issues |
Appeal Process
1. Developer receives rejection
2. Developer clicks "Appeal" (within 14 days)
3. Provides justification
4. Different reviewer examines
5. Final decision (approve/uphold rejection)
Review SLA
| Submission Type | Target Time |
|---|---|
| Auto-approved | Instant |
| New developer | 24 hours |
| Flagged | 48 hours |
| Appeal | 72 hours |
API Endpoints
# Developer endpoints
POST /v1/apps/:id/versions/:vid/submit:
summary: Submit for review
response: { version, estimated_review_time }
GET /v1/apps/:id/versions/:vid/review-status:
summary: Get review status
response: { status, feedback?, estimated_completion }
POST /v1/apps/:id/versions/:vid/appeal:
summary: Appeal rejection
body: { justification }
response: { appeal_id, status }
# Internal review endpoints (admin only)
GET /v1/admin/review-queue:
summary: Get pending reviews
response: { items[], total }
GET /v1/admin/review/:version_id:
summary: Get review details
response: { version, validation_result, flags }
POST /v1/admin/review/:version_id/approve:
summary: Approve version
body: { notes? }
response: { version }
POST /v1/admin/review/:version_id/reject:
summary: Reject version
body: { reason, message, details[] }
response: { version }
Metrics
Review Performance
| Metric | Target |
|---|---|
| Auto-approval rate | > 70% |
| Average review time | < 24 hours |
| Rejection rate | < 20% |
| Appeal overturn rate | < 10% |
Dashboard
-- Review stats
SELECT
DATE_TRUNC('week', submitted_at) as week,
COUNT(*) as submissions,
COUNT(*) FILTER (WHERE status = 'published') as approved,
COUNT(*) FILTER (WHERE status = 'rejected') as rejected,
AVG(EXTRACT(EPOCH FROM (reviewed_at - submitted_at))/3600) as avg_hours
FROM app_versions
WHERE submitted_at > NOW() - INTERVAL '30 days'
GROUP BY week;
Deliverables
- Validation worker implementation
- Dangerous pattern database
- Review queue UI
- Reviewer tools
- Rejection feedback system
- Appeal workflow
- Review metrics dashboard
- SLA monitoring
Open Questions
- Automated approval for trusted developers?
- Community moderators?
- Content policy document?
- Rate limiting resubmissions?