add developer portal planning documentation (M01-M12)
This commit is contained in:
372
DEV_PORTAL_M01_APP_PACKAGE.md
Normal file
372
DEV_PORTAL_M01_APP_PACKAGE.md
Normal file
@@ -0,0 +1,372 @@
|
||||
# Milestone 1: App Package Format
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Define how apps are bundled, signed, and validated.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The app package format is the foundation of the entire ecosystem. Every tool (CLI, portal, device) needs to understand this format.
|
||||
|
||||
---
|
||||
|
||||
## Questions to Answer
|
||||
|
||||
1. What files comprise an app package?
|
||||
2. How is the manifest structured?
|
||||
3. How are apps signed for integrity?
|
||||
4. How are updates handled (full vs delta)?
|
||||
5. What metadata is required (name, version, permissions, icons)?
|
||||
|
||||
---
|
||||
|
||||
## Package Format Options
|
||||
|
||||
### Option A: ZIP Archive
|
||||
|
||||
```
|
||||
myapp.zip
|
||||
├── manifest.json
|
||||
├── icon.png
|
||||
└── assets/
|
||||
├── main.rml
|
||||
├── styles.rcss
|
||||
└── app.lua
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Simple, standard tooling | No built-in signing |
|
||||
| Easy to inspect | No metadata in filename |
|
||||
| Wide compatibility | Need separate signature file |
|
||||
|
||||
### Option B: Custom Format (.mosis)
|
||||
|
||||
```
|
||||
myapp.mosis (binary format)
|
||||
┌─────────────────────────┐
|
||||
│ Magic bytes (4) │
|
||||
│ Version (2) │
|
||||
│ Manifest length (4) │
|
||||
│ Manifest JSON │
|
||||
│ Signature (256) │
|
||||
│ Compressed payload │
|
||||
└─────────────────────────┘
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Signature built-in | Custom tooling needed |
|
||||
| Efficient metadata access | Harder to inspect |
|
||||
| Single file | More complex implementation |
|
||||
|
||||
### Option C: Signed ZIP (Recommended)
|
||||
|
||||
```
|
||||
myapp.mosis/ (actually a ZIP)
|
||||
├── manifest.json
|
||||
├── META-INF/
|
||||
│ ├── MANIFEST.MF # File hashes
|
||||
│ └── SIGNATURE.SF # Signed manifest
|
||||
├── icon.png
|
||||
└── assets/
|
||||
└── ...
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Standard ZIP tooling | Slightly more complex |
|
||||
| JAR/APK-style signing | |
|
||||
| Easy inspection | |
|
||||
| Proven approach | |
|
||||
|
||||
---
|
||||
|
||||
## Proposed Structure
|
||||
|
||||
```
|
||||
com.developer.appname-1.0.0.mosis
|
||||
├── manifest.json # App metadata
|
||||
├── META-INF/
|
||||
│ ├── CERT.PEM # Developer certificate
|
||||
│ ├── CERT.SIG # Signature of MANIFEST.MF
|
||||
│ └── MANIFEST.MF # SHA-256 hashes of all files
|
||||
├── icons/
|
||||
│ ├── icon-32.png
|
||||
│ ├── icon-64.png
|
||||
│ └── icon-128.png
|
||||
├── assets/
|
||||
│ ├── main.rml # Entry point
|
||||
│ ├── screens/
|
||||
│ │ ├── home.rml
|
||||
│ │ └── settings.rml
|
||||
│ ├── styles/
|
||||
│ │ └── theme.rcss
|
||||
│ └── scripts/
|
||||
│ ├── main.lua
|
||||
│ └── utils.lua
|
||||
└── locales/ # Optional i18n
|
||||
├── en.json
|
||||
└── es.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Manifest Schema
|
||||
|
||||
### Required Fields
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://mosis.dev/schemas/manifest-v1.json",
|
||||
"id": "com.developer.appname",
|
||||
"name": "My App",
|
||||
"version": "1.0.0",
|
||||
"version_code": 1,
|
||||
"entry": "assets/main.rml",
|
||||
"min_mosis_version": "1.0.0"
|
||||
}
|
||||
```
|
||||
|
||||
### Full Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://mosis.dev/schemas/manifest-v1.json",
|
||||
|
||||
"id": "com.developer.appname",
|
||||
"name": "My App",
|
||||
"version": "1.0.0",
|
||||
"version_code": 1,
|
||||
|
||||
"description": "A short description of the app",
|
||||
"entry": "assets/main.rml",
|
||||
|
||||
"author": {
|
||||
"name": "Developer Name",
|
||||
"email": "dev@example.com",
|
||||
"url": "https://developer.com"
|
||||
},
|
||||
|
||||
"permissions": [
|
||||
"storage",
|
||||
"network",
|
||||
"camera"
|
||||
],
|
||||
|
||||
"icons": {
|
||||
"32": "icons/icon-32.png",
|
||||
"64": "icons/icon-64.png",
|
||||
"128": "icons/icon-128.png"
|
||||
},
|
||||
|
||||
"min_mosis_version": "1.0.0",
|
||||
"target_mosis_version": "1.2.0",
|
||||
|
||||
"category": "utilities",
|
||||
"tags": ["productivity", "tools"],
|
||||
|
||||
"orientation": "portrait",
|
||||
"background_color": "#FFFFFF",
|
||||
|
||||
"locales": ["en", "es", "fr"],
|
||||
"default_locale": "en"
|
||||
}
|
||||
```
|
||||
|
||||
### Field Definitions
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `id` | string | Yes | Unique package identifier (reverse domain) |
|
||||
| `name` | string | Yes | Display name (max 30 chars) |
|
||||
| `version` | string | Yes | Semantic version (X.Y.Z) |
|
||||
| `version_code` | integer | Yes | Incremental build number |
|
||||
| `entry` | string | Yes | Path to entry RML file |
|
||||
| `min_mosis_version` | string | Yes | Minimum Mosis version required |
|
||||
| `description` | string | No | Short description (max 80 chars) |
|
||||
| `author` | object | No | Author information |
|
||||
| `permissions` | array | No | Required permissions |
|
||||
| `icons` | object | No | Icon paths by size |
|
||||
| `category` | string | No | App store category |
|
||||
| `tags` | array | No | Searchable tags |
|
||||
| `orientation` | string | No | portrait, landscape, any |
|
||||
| `background_color` | string | No | Hex color for loading |
|
||||
| `locales` | array | No | Supported locales |
|
||||
|
||||
---
|
||||
|
||||
## Signing Mechanism
|
||||
|
||||
### Algorithm
|
||||
|
||||
- **Key type**: Ed25519 (fast, secure, small signatures)
|
||||
- **Hash**: SHA-256 for file manifests
|
||||
- **Format**: PEM for keys, base64 for signatures
|
||||
|
||||
### MANIFEST.MF Format
|
||||
|
||||
```
|
||||
Manifest-Version: 1.0
|
||||
Created-By: mosis-cli 1.0.0
|
||||
|
||||
Name: manifest.json
|
||||
SHA-256-Digest: base64encodedHash==
|
||||
|
||||
Name: assets/main.rml
|
||||
SHA-256-Digest: base64encodedHash==
|
||||
|
||||
Name: assets/scripts/main.lua
|
||||
SHA-256-Digest: base64encodedHash==
|
||||
```
|
||||
|
||||
### Signing Flow
|
||||
|
||||
```
|
||||
1. Generate MANIFEST.MF with SHA-256 of each file
|
||||
2. Sign MANIFEST.MF with developer's Ed25519 private key
|
||||
3. Store signature in META-INF/CERT.SIG
|
||||
4. Include developer's public key in META-INF/CERT.PEM
|
||||
```
|
||||
|
||||
### Verification Flow
|
||||
|
||||
```
|
||||
1. Extract META-INF/MANIFEST.MF
|
||||
2. Verify signature using CERT.PEM
|
||||
3. Verify CERT.PEM is registered with developer account
|
||||
4. Verify each file hash matches MANIFEST.MF
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Size Limits
|
||||
|
||||
| Limit | Value | Rationale |
|
||||
|-------|-------|-----------|
|
||||
| Max package size | 50 MB | Reasonable for mobile |
|
||||
| Max individual file | 10 MB | Prevent abuse |
|
||||
| Max files count | 1000 | Prevent zip bombs |
|
||||
| Max path length | 256 chars | Filesystem compat |
|
||||
| Max manifest size | 64 KB | Prevent abuse |
|
||||
|
||||
---
|
||||
|
||||
## Validation Rules
|
||||
|
||||
### Manifest Validation
|
||||
|
||||
- [ ] Valid JSON
|
||||
- [ ] All required fields present
|
||||
- [ ] `id` matches reverse domain pattern: `^[a-z][a-z0-9]*(\.[a-z][a-z0-9]*)+$`
|
||||
- [ ] `version` matches semver pattern
|
||||
- [ ] `version_code` is positive integer
|
||||
- [ ] `entry` file exists in package
|
||||
- [ ] All `permissions` are valid permission names
|
||||
- [ ] All `icons` paths exist and are valid images
|
||||
|
||||
### Package Validation
|
||||
|
||||
- [ ] Valid ZIP format
|
||||
- [ ] manifest.json at root
|
||||
- [ ] No path traversal (../)
|
||||
- [ ] No absolute paths
|
||||
- [ ] No symlinks
|
||||
- [ ] Under size limits
|
||||
- [ ] No duplicate files
|
||||
- [ ] Valid file extensions only
|
||||
|
||||
### Signature Validation
|
||||
|
||||
- [ ] MANIFEST.MF present
|
||||
- [ ] CERT.SIG present
|
||||
- [ ] CERT.PEM present
|
||||
- [ ] Signature valid for MANIFEST.MF
|
||||
- [ ] All file hashes match
|
||||
- [ ] Certificate registered with developer account (for store)
|
||||
|
||||
---
|
||||
|
||||
## File Extension Rules
|
||||
|
||||
### Allowed Extensions
|
||||
|
||||
| Category | Extensions |
|
||||
|----------|------------|
|
||||
| UI | .rml |
|
||||
| Styles | .rcss |
|
||||
| Scripts | .lua |
|
||||
| Images | .png, .jpg, .jpeg, .tga, .webp |
|
||||
| Fonts | .ttf, .otf |
|
||||
| Data | .json |
|
||||
| Localization | .json (in locales/) |
|
||||
| Audio | .ogg, .wav, .mp3 |
|
||||
|
||||
### Forbidden
|
||||
|
||||
- Executables: .exe, .dll, .so, .dylib
|
||||
- Scripts: .sh, .bat, .ps1, .py, .js
|
||||
- Archives: .zip, .tar, .gz (nested)
|
||||
|
||||
---
|
||||
|
||||
## Update Handling
|
||||
|
||||
### Full Update
|
||||
|
||||
- Download complete new package
|
||||
- Verify signature
|
||||
- Atomic replacement of app directory
|
||||
- Preserve user data in separate location
|
||||
|
||||
### Delta Updates (Future)
|
||||
|
||||
- bsdiff-style patches
|
||||
- Reduces bandwidth for minor updates
|
||||
- More complex implementation
|
||||
- Consider for v2
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] JSON Schema for manifest validation
|
||||
- [ ] Package format specification document
|
||||
- [ ] Reference implementation: package creator (Go/Rust)
|
||||
- [ ] Reference implementation: package validator
|
||||
- [ ] Reference implementation: signature tools
|
||||
- [ ] Integration with mosis-cli
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
| Test | Description |
|
||||
|------|-------------|
|
||||
| ValidPackage | Accept well-formed package |
|
||||
| InvalidManifest | Reject malformed JSON |
|
||||
| MissingRequired | Reject missing required fields |
|
||||
| BadSignature | Reject invalid signature |
|
||||
| TamperedFile | Reject if file hash mismatch |
|
||||
| PathTraversal | Reject ../ in paths |
|
||||
| OversizePackage | Reject over size limit |
|
||||
| BadExtension | Reject forbidden file types |
|
||||
| DuplicateFiles | Reject duplicate entries |
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Should we support multiple entry points (e.g., widget vs full app)?
|
||||
2. Should icons be required or have defaults?
|
||||
3. Delta updates in v1 or defer to v2?
|
||||
4. Support for app bundles (multiple apps in one package)?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Android APK format](https://developer.android.com/guide/components/fundamentals)
|
||||
- [JAR signing](https://docs.oracle.com/javase/tutorial/deployment/jar/signing.html)
|
||||
- [Ed25519 specification](https://ed25519.cr.yp.to/)
|
||||
333
DEV_PORTAL_M02_WEB_STACK.md
Normal file
333
DEV_PORTAL_M02_WEB_STACK.md
Normal file
@@ -0,0 +1,333 @@
|
||||
# Milestone 2: Web Stack Selection
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Choose backend technologies for the developer portal and app store API.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The web stack powers the developer portal, app store API, and telemetry ingestion. This decision affects development speed, hosting costs, and long-term maintenance.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional
|
||||
|
||||
- REST API for developer portal
|
||||
- File upload handling (app packages)
|
||||
- Authentication (OAuth2, API keys)
|
||||
- Database integration
|
||||
- Background jobs (review queue, notifications)
|
||||
- WebSocket support (optional, for real-time updates)
|
||||
|
||||
### Non-Functional
|
||||
|
||||
- Handle 1000+ developers initially
|
||||
- Scale to 10,000+ apps
|
||||
- 99.9% uptime target
|
||||
- < 200ms API response time (p95)
|
||||
- Secure by default
|
||||
|
||||
---
|
||||
|
||||
## Options Analysis
|
||||
|
||||
### Option A: Node.js + TypeScript
|
||||
|
||||
#### Stack
|
||||
```
|
||||
Runtime: Node.js 20 LTS
|
||||
Language: TypeScript
|
||||
Framework: Fastify or Hono
|
||||
ORM: Prisma or Drizzle
|
||||
Validation: Zod
|
||||
Auth: Lucia or custom JWT
|
||||
```
|
||||
|
||||
#### Pros
|
||||
| Advantage | Details |
|
||||
|-----------|---------|
|
||||
| Fast development | Large ecosystem, familiar syntax |
|
||||
| Type safety | TypeScript catches errors early |
|
||||
| Easy hiring | Many JS/TS developers |
|
||||
| Ecosystem | npm has libraries for everything |
|
||||
| Hosting | Vercel, Railway, Render, any VPS |
|
||||
|
||||
#### Cons
|
||||
| Disadvantage | Details |
|
||||
|--------------|---------|
|
||||
| Single-threaded | Need clustering for CPU tasks |
|
||||
| Memory usage | Higher than compiled languages |
|
||||
| Callback complexity | Async can get messy |
|
||||
|
||||
#### Example Structure
|
||||
```
|
||||
src/
|
||||
├── index.ts
|
||||
├── routes/
|
||||
│ ├── auth.ts
|
||||
│ ├── apps.ts
|
||||
│ └── telemetry.ts
|
||||
├── services/
|
||||
│ ├── auth.service.ts
|
||||
│ └── storage.service.ts
|
||||
├── db/
|
||||
│ ├── schema.ts
|
||||
│ └── client.ts
|
||||
└── utils/
|
||||
```
|
||||
|
||||
#### Hosting Cost Estimate
|
||||
| Service | Cost/month |
|
||||
|---------|------------|
|
||||
| Railway (starter) | $5-20 |
|
||||
| Vercel Pro | $20 |
|
||||
| VPS (4GB) | $20-40 |
|
||||
|
||||
---
|
||||
|
||||
### Option B: Go
|
||||
|
||||
#### Stack
|
||||
```
|
||||
Language: Go 1.22+
|
||||
Framework: Chi, Echo, or Gin
|
||||
ORM: sqlc or GORM
|
||||
Validation: go-playground/validator
|
||||
Auth: Custom JWT
|
||||
```
|
||||
|
||||
#### Pros
|
||||
| Advantage | Details |
|
||||
|-----------|---------|
|
||||
| Performance | Fast, low memory |
|
||||
| Single binary | Easy deployment |
|
||||
| Concurrency | Goroutines handle load well |
|
||||
| Standard library | HTTP, JSON, crypto built-in |
|
||||
|
||||
#### Cons
|
||||
| Disadvantage | Details |
|
||||
|--------------|---------|
|
||||
| Verbose | Error handling boilerplate |
|
||||
| Smaller ecosystem | Fewer ready-made solutions |
|
||||
| Learning curve | Different paradigm |
|
||||
|
||||
#### Example Structure
|
||||
```
|
||||
cmd/
|
||||
├── server/
|
||||
│ └── main.go
|
||||
internal/
|
||||
├── api/
|
||||
│ ├── handlers/
|
||||
│ ├── middleware/
|
||||
│ └── routes.go
|
||||
├── domain/
|
||||
│ ├── app.go
|
||||
│ └── developer.go
|
||||
├── repository/
|
||||
│ └── postgres/
|
||||
└── service/
|
||||
```
|
||||
|
||||
#### Hosting Cost Estimate
|
||||
| Service | Cost/month |
|
||||
|---------|------------|
|
||||
| Fly.io (shared) | $5-15 |
|
||||
| Cloud Run | Pay per use |
|
||||
| VPS (2GB) | $10-20 |
|
||||
|
||||
---
|
||||
|
||||
### Option C: Rust + Axum
|
||||
|
||||
#### Stack
|
||||
```
|
||||
Language: Rust
|
||||
Framework: Axum
|
||||
ORM: sqlx or SeaORM
|
||||
Validation: validator crate
|
||||
Auth: Custom JWT
|
||||
```
|
||||
|
||||
#### Pros
|
||||
| Advantage | Details |
|
||||
|-----------|---------|
|
||||
| Maximum performance | Fastest option |
|
||||
| Memory safety | No runtime errors |
|
||||
| Single binary | Easy deployment |
|
||||
| Long-term reliability | Compiler catches bugs |
|
||||
|
||||
#### Cons
|
||||
| Disadvantage | Details |
|
||||
|--------------|---------|
|
||||
| Steep learning curve | Borrow checker takes time |
|
||||
| Slower development | More code to write |
|
||||
| Smaller ecosystem | Fewer web libraries |
|
||||
| Compile times | Can be slow |
|
||||
|
||||
#### Example Structure
|
||||
```
|
||||
src/
|
||||
├── main.rs
|
||||
├── api/
|
||||
│ ├── mod.rs
|
||||
│ ├── handlers/
|
||||
│ └── middleware/
|
||||
├── domain/
|
||||
├── repository/
|
||||
└── service/
|
||||
```
|
||||
|
||||
#### Hosting Cost Estimate
|
||||
| Service | Cost/month |
|
||||
|---------|------------|
|
||||
| Fly.io | $5-10 |
|
||||
| Shuttle | Free tier available |
|
||||
| VPS (1GB) | $5-10 |
|
||||
|
||||
---
|
||||
|
||||
### Option D: .NET (ASP.NET Core)
|
||||
|
||||
#### Stack
|
||||
```
|
||||
Language: C#
|
||||
Framework: ASP.NET Core 8
|
||||
ORM: Entity Framework Core
|
||||
Validation: FluentValidation
|
||||
Auth: ASP.NET Identity
|
||||
```
|
||||
|
||||
#### Pros
|
||||
| Advantage | Details |
|
||||
|-----------|---------|
|
||||
| Enterprise-ready | Battle-tested at scale |
|
||||
| Great tooling | Visual Studio, debugging |
|
||||
| Performance | Very fast (Kestrel) |
|
||||
| Full-featured | Auth, validation, DI built-in |
|
||||
|
||||
#### Cons
|
||||
| Disadvantage | Details |
|
||||
|--------------|---------|
|
||||
| Heavier runtime | Larger container images |
|
||||
| Microsoft ecosystem | May feel locked in |
|
||||
| Verbose | More boilerplate |
|
||||
|
||||
#### Hosting Cost Estimate
|
||||
| Service | Cost/month |
|
||||
|---------|------------|
|
||||
| Azure App Service | $15-50 |
|
||||
| VPS (4GB) | $20-40 |
|
||||
|
||||
---
|
||||
|
||||
## Evaluation Matrix
|
||||
|
||||
| Criteria | Weight | Node.js | Go | Rust | .NET |
|
||||
|----------|--------|---------|-----|------|------|
|
||||
| Dev speed | 25% | 9 | 7 | 5 | 7 |
|
||||
| Performance | 20% | 6 | 9 | 10 | 8 |
|
||||
| Hosting cost | 15% | 7 | 9 | 9 | 6 |
|
||||
| Ecosystem | 15% | 9 | 7 | 6 | 8 |
|
||||
| Team familiarity | 15% | ? | ? | ? | ? |
|
||||
| Long-term maintenance | 10% | 7 | 8 | 9 | 8 |
|
||||
| **Weighted Score** | | **7.4** | **7.8** | **7.2** | **7.3** |
|
||||
|
||||
*Team familiarity needs to be filled in based on actual experience*
|
||||
|
||||
---
|
||||
|
||||
## Framework Comparison
|
||||
|
||||
### Node.js Frameworks
|
||||
|
||||
| Framework | Req/sec | Features | Learning Curve |
|
||||
|-----------|---------|----------|----------------|
|
||||
| Express | 15k | Minimal, flexible | Easy |
|
||||
| Fastify | 45k | Fast, schema validation | Medium |
|
||||
| Hono | 50k | Ultra-light, edge-ready | Easy |
|
||||
| NestJS | 20k | Full-featured, Angular-like | Steep |
|
||||
|
||||
### Go Frameworks
|
||||
|
||||
| Framework | Req/sec | Features | Learning Curve |
|
||||
|-----------|---------|----------|----------------|
|
||||
| net/http | 60k | Standard library | Medium |
|
||||
| Chi | 55k | Lightweight router | Easy |
|
||||
| Gin | 50k | Popular, middleware | Easy |
|
||||
| Echo | 55k | Similar to Gin | Easy |
|
||||
|
||||
---
|
||||
|
||||
## Prototype Plan
|
||||
|
||||
### Phase 1: Build minimal API in top 2 choices
|
||||
|
||||
Endpoints to implement:
|
||||
```
|
||||
POST /auth/register
|
||||
POST /auth/login
|
||||
GET /apps
|
||||
POST /apps
|
||||
POST /apps/:id/versions
|
||||
```
|
||||
|
||||
### Phase 2: Benchmark
|
||||
|
||||
- Requests per second
|
||||
- Memory usage under load
|
||||
- Development time tracking
|
||||
- Code complexity comparison
|
||||
|
||||
### Phase 3: Decision
|
||||
|
||||
Based on:
|
||||
- Benchmark results
|
||||
- Developer experience during prototype
|
||||
- Hosting cost analysis
|
||||
|
||||
---
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Primary: Go with Chi/Echo**
|
||||
- Best balance of performance and simplicity
|
||||
- Low hosting costs
|
||||
- Single binary deployment
|
||||
- Good enough ecosystem for our needs
|
||||
|
||||
**Fallback: Node.js with Hono/Fastify**
|
||||
- If Go feels too slow to develop in
|
||||
- Larger ecosystem for edge cases
|
||||
- More developers familiar with it
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] Prototype API in Go
|
||||
- [ ] Prototype API in Node.js (if needed)
|
||||
- [ ] Benchmark comparison document
|
||||
- [ ] Final selection with rationale
|
||||
- [ ] Project structure template
|
||||
- [ ] Development environment setup guide
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Do we need GraphQL or is REST sufficient?
|
||||
2. Do we need real-time features (WebSocket)?
|
||||
3. What's the team's current language experience?
|
||||
4. Any preference for specific cloud providers?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [TechEmpower Benchmarks](https://www.techempower.com/benchmarks/)
|
||||
- [Go vs Node.js comparison](https://blog.logrocket.com/node-js-vs-golang/)
|
||||
- [Fastify benchmarks](https://fastify.dev/benchmarks/)
|
||||
418
DEV_PORTAL_M03_DATABASE.md
Normal file
418
DEV_PORTAL_M03_DATABASE.md
Normal file
@@ -0,0 +1,418 @@
|
||||
# Milestone 3: Database Selection
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Choose database for developer accounts, app metadata, and analytics.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The database stores all persistent data: developer accounts, app metadata, versions, telemetry events, and audit logs.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
### Data Characteristics
|
||||
|
||||
| Data Type | Volume | Access Pattern | Consistency |
|
||||
|-----------|--------|----------------|-------------|
|
||||
| Developers | 10K rows | Read-heavy, low write | Strong |
|
||||
| Apps | 100K rows | Read-heavy | Strong |
|
||||
| Versions | 500K rows | Read-heavy | Strong |
|
||||
| API Keys | 50K rows | Read-heavy | Strong |
|
||||
| Telemetry | 100M+ rows | Write-heavy, append | Eventual OK |
|
||||
| Audit Logs | 10M+ rows | Write-heavy, append | Eventual OK |
|
||||
|
||||
### Query Patterns
|
||||
|
||||
- Get developer by email
|
||||
- List apps by developer
|
||||
- Get app with latest version
|
||||
- Search apps by name/tags
|
||||
- Aggregate telemetry by app/day
|
||||
- Time-range queries on events
|
||||
|
||||
---
|
||||
|
||||
## Options Analysis
|
||||
|
||||
### Option A: PostgreSQL
|
||||
|
||||
#### Characteristics
|
||||
```
|
||||
Type: Relational (SQL)
|
||||
ACID: Full
|
||||
JSON: Native JSONB support
|
||||
Full-text: Built-in tsvector
|
||||
Scaling: Vertical + read replicas
|
||||
```
|
||||
|
||||
#### Pros
|
||||
| Advantage | Details |
|
||||
|-----------|---------|
|
||||
| Battle-tested | Decades of reliability |
|
||||
| ACID compliance | Strong consistency |
|
||||
| JSON support | JSONB for flexible data |
|
||||
| Full-text search | No separate search engine needed |
|
||||
| Extensions | PostGIS, pg_trgm, etc. |
|
||||
| Tooling | pgAdmin, great ORMs |
|
||||
|
||||
#### Cons
|
||||
| Disadvantage | Details |
|
||||
|--------------|---------|
|
||||
| Ops overhead | Need connection pooling |
|
||||
| Scaling writes | Vertical scaling limits |
|
||||
| Time-series | Not optimized for telemetry |
|
||||
|
||||
#### Hosting Options
|
||||
| Provider | Free Tier | Paid |
|
||||
|----------|-----------|------|
|
||||
| Supabase | 500MB | $25/mo |
|
||||
| Neon | 512MB | $19/mo |
|
||||
| Railway | 1GB | $5/mo |
|
||||
| AWS RDS | - | $15/mo+ |
|
||||
| Self-hosted | - | VPS cost |
|
||||
|
||||
---
|
||||
|
||||
### Option B: SQLite + Litestream
|
||||
|
||||
#### Characteristics
|
||||
```
|
||||
Type: Embedded relational
|
||||
ACID: Full
|
||||
Scaling: Single writer
|
||||
Backup: Litestream to S3
|
||||
```
|
||||
|
||||
#### Pros
|
||||
| Advantage | Details |
|
||||
|-----------|---------|
|
||||
| Zero ops | No separate DB server |
|
||||
| Fast reads | In-process, no network |
|
||||
| Simple backup | Litestream handles replication |
|
||||
| Low cost | Just storage costs |
|
||||
| Portable | Easy local development |
|
||||
|
||||
#### Cons
|
||||
| Disadvantage | Details |
|
||||
|--------------|---------|
|
||||
| Single writer | Limits write concurrency |
|
||||
| No horizontal scale | One server only |
|
||||
| Limited features | No full-text (without FTS5) |
|
||||
|
||||
#### Cost Estimate
|
||||
| Component | Cost/month |
|
||||
|-----------|------------|
|
||||
| S3 storage (10GB) | $0.25 |
|
||||
| Compute | Included in app server |
|
||||
|
||||
---
|
||||
|
||||
### Option C: PostgreSQL + TimescaleDB
|
||||
|
||||
#### Characteristics
|
||||
```
|
||||
Type: Time-series extension
|
||||
Base: PostgreSQL
|
||||
Scaling: Automatic partitioning
|
||||
Compression: Native
|
||||
```
|
||||
|
||||
#### Pros
|
||||
| Advantage | Details |
|
||||
|-----------|---------|
|
||||
| Best of both | Relational + time-series |
|
||||
| Auto-partition | Handles telemetry scale |
|
||||
| Compression | 90%+ compression ratio |
|
||||
| Continuous aggregates | Pre-computed rollups |
|
||||
|
||||
#### Cons
|
||||
| Disadvantage | Details |
|
||||
|--------------|---------|
|
||||
| Complexity | More to manage |
|
||||
| Cost | Higher than plain Postgres |
|
||||
| Learning curve | New concepts |
|
||||
|
||||
---
|
||||
|
||||
### Option D: Hybrid Approach
|
||||
|
||||
```
|
||||
PostgreSQL → Developers, Apps, Versions, API Keys
|
||||
ClickHouse/QuestDB → Telemetry, Analytics
|
||||
Redis → Caching, Sessions
|
||||
```
|
||||
|
||||
#### Pros
|
||||
| Advantage | Details |
|
||||
|-----------|---------|
|
||||
| Right tool for job | Optimized for each use case |
|
||||
| Scale independently | Telemetry won't affect main DB |
|
||||
| Performance | Best possible for each workload |
|
||||
|
||||
#### Cons
|
||||
| Disadvantage | Details |
|
||||
|--------------|---------|
|
||||
| Complexity | Multiple systems to manage |
|
||||
| Cost | More infrastructure |
|
||||
| Consistency | Cross-DB transactions hard |
|
||||
|
||||
---
|
||||
|
||||
## Schema Design
|
||||
|
||||
### Core Tables
|
||||
|
||||
```sql
|
||||
-- Developers
|
||||
CREATE TABLE developers (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
email VARCHAR(255) UNIQUE NOT NULL,
|
||||
name VARCHAR(100) NOT NULL,
|
||||
password_hash VARCHAR(255),
|
||||
oauth_provider VARCHAR(50),
|
||||
oauth_id VARCHAR(255),
|
||||
verified BOOLEAN DEFAULT FALSE,
|
||||
created_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- API Keys
|
||||
CREATE TABLE api_keys (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
developer_id UUID REFERENCES developers(id) ON DELETE CASCADE,
|
||||
name VARCHAR(100) NOT NULL,
|
||||
key_hash VARCHAR(255) NOT NULL,
|
||||
key_prefix VARCHAR(10) NOT NULL, -- For display: "mk_abc..."
|
||||
permissions JSONB DEFAULT '[]',
|
||||
last_used_at TIMESTAMPTZ,
|
||||
expires_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Apps
|
||||
CREATE TABLE apps (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
developer_id UUID REFERENCES developers(id) ON DELETE CASCADE,
|
||||
package_id VARCHAR(255) UNIQUE NOT NULL, -- com.dev.app
|
||||
name VARCHAR(100) NOT NULL,
|
||||
description TEXT,
|
||||
category VARCHAR(50),
|
||||
tags VARCHAR(50)[] DEFAULT '{}',
|
||||
status VARCHAR(20) DEFAULT 'draft', -- draft, published, suspended
|
||||
created_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- App Versions
|
||||
CREATE TABLE app_versions (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
app_id UUID REFERENCES apps(id) ON DELETE CASCADE,
|
||||
version_code INTEGER NOT NULL,
|
||||
version_name VARCHAR(20) NOT NULL,
|
||||
package_url TEXT NOT NULL,
|
||||
package_size BIGINT NOT NULL,
|
||||
signature VARCHAR(512) NOT NULL,
|
||||
permissions JSONB DEFAULT '[]',
|
||||
min_mosis_version VARCHAR(20),
|
||||
release_notes TEXT,
|
||||
status VARCHAR(20) DEFAULT 'draft', -- draft, review, approved, published, rejected
|
||||
review_notes TEXT,
|
||||
published_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
UNIQUE(app_id, version_code)
|
||||
);
|
||||
|
||||
-- Developer Signing Keys
|
||||
CREATE TABLE signing_keys (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
developer_id UUID REFERENCES developers(id) ON DELETE CASCADE,
|
||||
name VARCHAR(100) NOT NULL,
|
||||
public_key TEXT NOT NULL,
|
||||
fingerprint VARCHAR(64) NOT NULL,
|
||||
is_active BOOLEAN DEFAULT TRUE,
|
||||
created_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
### Telemetry Tables (if using PostgreSQL)
|
||||
|
||||
```sql
|
||||
-- Telemetry Events (consider partitioning by time)
|
||||
CREATE TABLE telemetry_events (
|
||||
id BIGSERIAL,
|
||||
app_id UUID NOT NULL,
|
||||
device_id VARCHAR(64) NOT NULL, -- Hashed
|
||||
event_type VARCHAR(50) NOT NULL,
|
||||
event_data JSONB,
|
||||
mosis_version VARCHAR(20),
|
||||
timestamp TIMESTAMPTZ NOT NULL,
|
||||
PRIMARY KEY (timestamp, id)
|
||||
) PARTITION BY RANGE (timestamp);
|
||||
|
||||
-- Create monthly partitions
|
||||
CREATE TABLE telemetry_events_2024_01 PARTITION OF telemetry_events
|
||||
FOR VALUES FROM ('2024-01-01') TO ('2024-02-01');
|
||||
|
||||
-- Crash Reports
|
||||
CREATE TABLE crash_reports (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
app_id UUID NOT NULL,
|
||||
app_version VARCHAR(20) NOT NULL,
|
||||
device_id VARCHAR(64) NOT NULL,
|
||||
crash_type VARCHAR(50) NOT NULL,
|
||||
message TEXT,
|
||||
stack_trace TEXT,
|
||||
context JSONB,
|
||||
mosis_version VARCHAR(20),
|
||||
timestamp TIMESTAMPTZ NOT NULL,
|
||||
created_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Daily aggregates (materialized or computed)
|
||||
CREATE TABLE telemetry_daily (
|
||||
app_id UUID NOT NULL,
|
||||
date DATE NOT NULL,
|
||||
event_type VARCHAR(50) NOT NULL,
|
||||
count BIGINT NOT NULL,
|
||||
unique_devices BIGINT NOT NULL,
|
||||
PRIMARY KEY (app_id, date, event_type)
|
||||
);
|
||||
```
|
||||
|
||||
### Indexes
|
||||
|
||||
```sql
|
||||
-- Developers
|
||||
CREATE INDEX idx_developers_email ON developers(email);
|
||||
CREATE INDEX idx_developers_oauth ON developers(oauth_provider, oauth_id);
|
||||
|
||||
-- Apps
|
||||
CREATE INDEX idx_apps_developer ON apps(developer_id);
|
||||
CREATE INDEX idx_apps_package ON apps(package_id);
|
||||
CREATE INDEX idx_apps_status ON apps(status);
|
||||
CREATE INDEX idx_apps_search ON apps USING gin(to_tsvector('english', name || ' ' || COALESCE(description, '')));
|
||||
|
||||
-- Versions
|
||||
CREATE INDEX idx_versions_app ON app_versions(app_id);
|
||||
CREATE INDEX idx_versions_status ON app_versions(status);
|
||||
|
||||
-- Telemetry
|
||||
CREATE INDEX idx_telemetry_app ON telemetry_events(app_id, timestamp);
|
||||
CREATE INDEX idx_telemetry_type ON telemetry_events(event_type, timestamp);
|
||||
|
||||
-- Crashes
|
||||
CREATE INDEX idx_crashes_app ON crash_reports(app_id, timestamp);
|
||||
CREATE INDEX idx_crashes_type ON crash_reports(crash_type);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Approach: Incremental Migrations
|
||||
|
||||
```
|
||||
migrations/
|
||||
├── 001_create_developers.sql
|
||||
├── 002_create_apps.sql
|
||||
├── 003_create_versions.sql
|
||||
├── 004_create_telemetry.sql
|
||||
└── ...
|
||||
```
|
||||
|
||||
### Tools
|
||||
- **Go**: golang-migrate, goose
|
||||
- **Node.js**: Prisma Migrate, Drizzle Kit
|
||||
- **Rust**: sqlx migrate, refinery
|
||||
|
||||
### Rollback Strategy
|
||||
- Every migration has up/down
|
||||
- Test rollbacks in staging
|
||||
- Keep migrations small and focused
|
||||
|
||||
---
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### PostgreSQL
|
||||
```bash
|
||||
# Daily full backup
|
||||
pg_dump -Fc $DATABASE_URL > backup_$(date +%Y%m%d).dump
|
||||
|
||||
# Continuous WAL archiving to S3
|
||||
archive_command = 'aws s3 cp %p s3://backups/wal/%f'
|
||||
```
|
||||
|
||||
### SQLite + Litestream
|
||||
```yaml
|
||||
# litestream.yml
|
||||
dbs:
|
||||
- path: /data/mosis.db
|
||||
replicas:
|
||||
- url: s3://backups/mosis
|
||||
retention: 720h # 30 days
|
||||
```
|
||||
|
||||
### Recovery Time Objectives
|
||||
| Scenario | RTO | RPO |
|
||||
|----------|-----|-----|
|
||||
| Hardware failure | 1 hour | 5 minutes |
|
||||
| Data corruption | 4 hours | 1 hour |
|
||||
| Disaster recovery | 24 hours | 24 hours |
|
||||
|
||||
---
|
||||
|
||||
## Recommendation
|
||||
|
||||
### For MVP/Early Stage
|
||||
**SQLite + Litestream**
|
||||
- Simplest to operate
|
||||
- Lowest cost
|
||||
- Good enough for initial scale
|
||||
- Easy migration to PostgreSQL later
|
||||
|
||||
### For Production Scale
|
||||
**PostgreSQL + TimescaleDB**
|
||||
- Handles all data types well
|
||||
- Time-series for telemetry
|
||||
- Proven at scale
|
||||
- Good tooling ecosystem
|
||||
|
||||
### Hybrid (If needed later)
|
||||
```
|
||||
PostgreSQL → Core data (developers, apps)
|
||||
TimescaleDB → Telemetry (same cluster, extension)
|
||||
Redis → Caching, rate limiting
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] Final database selection
|
||||
- [ ] Complete schema design
|
||||
- [ ] Migration scripts
|
||||
- [ ] Backup/restore procedures
|
||||
- [ ] Connection pooling setup (if PostgreSQL)
|
||||
- [ ] Monitoring queries
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Expected telemetry volume per day?
|
||||
2. How long to retain raw telemetry?
|
||||
3. Need for real-time analytics vs batch?
|
||||
4. Multi-region requirements?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [PostgreSQL JSONB performance](https://www.postgresql.org/docs/current/datatype-json.html)
|
||||
- [TimescaleDB vs InfluxDB](https://www.timescale.com/blog/timescaledb-vs-influxdb/)
|
||||
- [Litestream documentation](https://litestream.io/)
|
||||
- [SQLite at scale](https://www.sqlite.org/whentouse.html)
|
||||
429
DEV_PORTAL_M04_AUTH.md
Normal file
429
DEV_PORTAL_M04_AUTH.md
Normal file
@@ -0,0 +1,429 @@
|
||||
# Milestone 4: Authentication System
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Secure developer authentication and app signing infrastructure.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Authentication covers two areas:
|
||||
1. **Developer authentication** - Login to portal, API access
|
||||
2. **App signing** - Package integrity and developer verification
|
||||
|
||||
---
|
||||
|
||||
## Developer Authentication
|
||||
|
||||
### Methods Required
|
||||
|
||||
| Method | Use Case | Priority |
|
||||
|--------|----------|----------|
|
||||
| OAuth2 (GitHub) | Primary login | P0 |
|
||||
| OAuth2 (Google) | Alternative login | P1 |
|
||||
| Email + Password | Fallback | P2 |
|
||||
| API Keys | CLI tools, CI/CD | P0 |
|
||||
|
||||
### OAuth2 Flow
|
||||
|
||||
```
|
||||
┌─────────┐ ┌─────────┐ ┌──────────┐ ┌─────────┐
|
||||
│ Browser │────►│ Portal │────►│ Provider │────►│ Callback│
|
||||
└─────────┘ └─────────┘ │(GitHub) │ └────┬────┘
|
||||
└──────────┘ │
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ Create/Link │
|
||||
│ Account │
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
### OAuth2 Implementation
|
||||
|
||||
#### GitHub OAuth
|
||||
|
||||
```
|
||||
Authorization URL: https://github.com/login/oauth/authorize
|
||||
Token URL: https://github.com/login/oauth/access_token
|
||||
User Info: https://api.github.com/user
|
||||
Scopes: read:user, user:email
|
||||
```
|
||||
|
||||
#### Google OAuth
|
||||
|
||||
```
|
||||
Authorization URL: https://accounts.google.com/o/oauth2/v2/auth
|
||||
Token URL: https://oauth2.googleapis.com/token
|
||||
User Info: https://www.googleapis.com/oauth2/v2/userinfo
|
||||
Scopes: openid, email, profile
|
||||
```
|
||||
|
||||
### Session Management
|
||||
|
||||
#### JWT Tokens
|
||||
|
||||
```json
|
||||
{
|
||||
"sub": "dev_uuid",
|
||||
"email": "dev@example.com",
|
||||
"iat": 1704067200,
|
||||
"exp": 1704153600,
|
||||
"type": "access"
|
||||
}
|
||||
```
|
||||
|
||||
| Token Type | Lifetime | Storage |
|
||||
|------------|----------|---------|
|
||||
| Access Token | 1 hour | Memory/Cookie |
|
||||
| Refresh Token | 30 days | HttpOnly Cookie |
|
||||
|
||||
#### Token Refresh Flow
|
||||
|
||||
```
|
||||
1. Access token expires
|
||||
2. Client sends refresh token
|
||||
3. Server validates refresh token
|
||||
4. Issue new access + refresh tokens
|
||||
5. Invalidate old refresh token
|
||||
```
|
||||
|
||||
### API Key Authentication
|
||||
|
||||
#### Key Format
|
||||
|
||||
```
|
||||
mk_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||
│ │ └── 32 random bytes (base62)
|
||||
│ └── Environment (live/test)
|
||||
└── Prefix (mosis key)
|
||||
```
|
||||
|
||||
#### Key Storage
|
||||
|
||||
```sql
|
||||
-- Only store hash, never the key itself
|
||||
INSERT INTO api_keys (
|
||||
developer_id,
|
||||
name,
|
||||
key_hash, -- bcrypt or argon2
|
||||
key_prefix, -- "mk_live_abc" for display
|
||||
permissions
|
||||
) VALUES (...);
|
||||
```
|
||||
|
||||
#### Key Permissions
|
||||
|
||||
```json
|
||||
{
|
||||
"permissions": [
|
||||
"apps:read",
|
||||
"apps:write",
|
||||
"versions:upload",
|
||||
"telemetry:read"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
| Endpoint Category | Limit | Window |
|
||||
|-------------------|-------|--------|
|
||||
| Auth endpoints | 10 | 1 minute |
|
||||
| API (authenticated) | 1000 | 1 hour |
|
||||
| API (per key) | 100 | 1 minute |
|
||||
| Upload | 10 | 1 hour |
|
||||
|
||||
---
|
||||
|
||||
## App Signing
|
||||
|
||||
### Key Generation
|
||||
|
||||
#### Algorithm: Ed25519
|
||||
|
||||
```
|
||||
Private key: 32 bytes (256 bits)
|
||||
Public key: 32 bytes (256 bits)
|
||||
Signature: 64 bytes (512 bits)
|
||||
```
|
||||
|
||||
#### Why Ed25519?
|
||||
- Fast signing and verification
|
||||
- Small key and signature sizes
|
||||
- No configuration choices (secure by default)
|
||||
- Widely supported
|
||||
|
||||
### Key Generation Flow
|
||||
|
||||
```bash
|
||||
# Developer generates keypair locally
|
||||
mosis keys generate
|
||||
|
||||
# Output:
|
||||
# Private key saved to: ~/.mosis/signing_key.pem
|
||||
# Public key saved to: ~/.mosis/signing_key.pub
|
||||
#
|
||||
# Fingerprint: SHA256:xxxxx
|
||||
#
|
||||
# IMPORTANT: Keep your private key secure!
|
||||
# Upload your public key to the developer portal.
|
||||
```
|
||||
|
||||
### Key Format
|
||||
|
||||
#### Private Key (PEM)
|
||||
|
||||
```
|
||||
-----BEGIN PRIVATE KEY-----
|
||||
MC4CAQAwBQYDK2VwBCIEIGxxxxx...
|
||||
-----END PRIVATE KEY-----
|
||||
```
|
||||
|
||||
#### Public Key (PEM)
|
||||
|
||||
```
|
||||
-----BEGIN PUBLIC KEY-----
|
||||
MCowBQYDK2VwAyEAxxxxx...
|
||||
-----END PUBLIC KEY-----
|
||||
```
|
||||
|
||||
### Signing Flow
|
||||
|
||||
```
|
||||
1. Build package (ZIP all files)
|
||||
2. Generate MANIFEST.MF with SHA-256 hashes
|
||||
3. Sign MANIFEST.MF with private key
|
||||
4. Store signature in META-INF/CERT.SIG
|
||||
5. Include public key in META-INF/CERT.PEM
|
||||
```
|
||||
|
||||
#### MANIFEST.MF Example
|
||||
|
||||
```
|
||||
Manifest-Version: 1.0
|
||||
Created-By: mosis-cli 1.0.0
|
||||
Package-Id: com.developer.myapp
|
||||
Version-Code: 1
|
||||
|
||||
Name: manifest.json
|
||||
SHA-256-Digest: K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols=
|
||||
|
||||
Name: assets/main.rml
|
||||
SHA-256-Digest: uU0nuZNNPgilLlLX2n2r+sSE7+N6U4DukIj3rOLvzek=
|
||||
```
|
||||
|
||||
### Verification Flow
|
||||
|
||||
```
|
||||
1. Extract MANIFEST.MF from package
|
||||
2. Extract CERT.SIG (signature)
|
||||
3. Extract CERT.PEM (public key)
|
||||
4. Verify signature of MANIFEST.MF using public key
|
||||
5. Verify CERT.PEM matches registered developer key
|
||||
6. Verify each file hash matches MANIFEST.MF entry
|
||||
```
|
||||
|
||||
### Key Registration
|
||||
|
||||
```
|
||||
Developer Portal:
|
||||
├── Go to Settings > Signing Keys
|
||||
├── Click "Add Key"
|
||||
├── Paste public key (PEM format)
|
||||
├── Verify fingerprint matches local
|
||||
└── Key is now registered
|
||||
|
||||
Server stores:
|
||||
├── public_key (PEM text)
|
||||
├── fingerprint (SHA256 of public key)
|
||||
├── created_at
|
||||
└── is_active
|
||||
```
|
||||
|
||||
### Key Rotation
|
||||
|
||||
```
|
||||
1. Generate new keypair
|
||||
2. Register new public key in portal
|
||||
3. Sign new versions with new key
|
||||
4. (Optional) Revoke old key after transition period
|
||||
```
|
||||
|
||||
### Trust Model
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ Trust Chain │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Developer │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ Private Key ──signs──► Package │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ Public Key ──registered──► Portal │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ Portal ──verifies──► Signature │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ Device ──trusts──► Portal-verified packages │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Password Storage (if used)
|
||||
|
||||
```
|
||||
Algorithm: Argon2id
|
||||
Memory: 64 MB
|
||||
Iterations: 3
|
||||
Parallelism: 4
|
||||
Salt: 16 bytes random
|
||||
```
|
||||
|
||||
### Token Security
|
||||
|
||||
- Access tokens: Short-lived, in-memory only
|
||||
- Refresh tokens: HttpOnly, Secure, SameSite=Strict
|
||||
- API keys: Hashed with bcrypt, shown once on creation
|
||||
|
||||
### Key Security
|
||||
|
||||
- Private keys never leave developer's machine
|
||||
- Public keys verified via fingerprint
|
||||
- Key compromise: Revoke immediately, re-sign apps
|
||||
|
||||
### Audit Logging
|
||||
|
||||
```sql
|
||||
INSERT INTO auth_audit_log (
|
||||
developer_id,
|
||||
action, -- login, logout, key_create, key_revoke
|
||||
ip_address,
|
||||
user_agent,
|
||||
success,
|
||||
failure_reason,
|
||||
timestamp
|
||||
) VALUES (...);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Authentication
|
||||
|
||||
```
|
||||
POST /auth/oauth/github # Start GitHub OAuth
|
||||
GET /auth/oauth/github/callback # GitHub callback
|
||||
POST /auth/oauth/google # Start Google OAuth
|
||||
GET /auth/oauth/google/callback # Google callback
|
||||
POST /auth/refresh # Refresh tokens
|
||||
POST /auth/logout # Invalidate tokens
|
||||
GET /auth/me # Get current user
|
||||
```
|
||||
|
||||
### API Keys
|
||||
|
||||
```
|
||||
GET /api-keys # List keys
|
||||
POST /api-keys # Create key
|
||||
DELETE /api-keys/:id # Revoke key
|
||||
```
|
||||
|
||||
### Signing Keys
|
||||
|
||||
```
|
||||
GET /signing-keys # List keys
|
||||
POST /signing-keys # Register key
|
||||
DELETE /signing-keys/:id # Revoke key
|
||||
GET /signing-keys/:id/verify # Verify a signature
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Libraries
|
||||
|
||||
### Node.js
|
||||
|
||||
```json
|
||||
{
|
||||
"passport": "OAuth strategies",
|
||||
"jose": "JWT handling",
|
||||
"@noble/ed25519": "Ed25519 signing",
|
||||
"argon2": "Password hashing"
|
||||
}
|
||||
```
|
||||
|
||||
### Go
|
||||
|
||||
```go
|
||||
import (
|
||||
"golang.org/x/oauth2"
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
"crypto/ed25519"
|
||||
"golang.org/x/crypto/argon2"
|
||||
)
|
||||
```
|
||||
|
||||
### Rust
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
oauth2 = "4.4"
|
||||
jsonwebtoken = "9"
|
||||
ed25519-dalek = "2"
|
||||
argon2 = "0.5"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] OAuth2 integration (GitHub)
|
||||
- [ ] OAuth2 integration (Google)
|
||||
- [ ] JWT token management
|
||||
- [ ] API key generation and validation
|
||||
- [ ] Ed25519 key generation tool
|
||||
- [ ] Signature creation and verification
|
||||
- [ ] Key registration API
|
||||
- [ ] Audit logging
|
||||
|
||||
---
|
||||
|
||||
## Test Cases
|
||||
|
||||
| Test | Description |
|
||||
|------|-------------|
|
||||
| OAuthLogin | Complete OAuth flow |
|
||||
| TokenRefresh | Refresh expired access token |
|
||||
| InvalidToken | Reject tampered JWT |
|
||||
| APIKeyAuth | Authenticate with API key |
|
||||
| KeyGeneration | Generate valid Ed25519 keypair |
|
||||
| SignPackage | Sign and verify package |
|
||||
| InvalidSignature | Reject tampered package |
|
||||
| KeyRevocation | Revoked key fails verification |
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Support for hardware security keys (YubiKey)?
|
||||
2. Multi-factor authentication for portal?
|
||||
3. Team accounts with role-based access?
|
||||
4. Key escrow for enterprise customers?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [OAuth 2.0 RFC 6749](https://tools.ietf.org/html/rfc6749)
|
||||
- [JWT RFC 7519](https://tools.ietf.org/html/rfc7519)
|
||||
- [Ed25519 paper](https://ed25519.cr.yp.to/ed25519-20110926.pdf)
|
||||
- [OWASP Authentication Cheatsheet](https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html)
|
||||
464
DEV_PORTAL_M05_FRONTEND.md
Normal file
464
DEV_PORTAL_M05_FRONTEND.md
Normal file
@@ -0,0 +1,464 @@
|
||||
# Milestone 5: Developer Portal Frontend
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Web interface for developer account and app management.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The developer portal is the primary interface for developers to manage their accounts, create apps, submit versions, and view analytics.
|
||||
|
||||
---
|
||||
|
||||
## Pages Required
|
||||
|
||||
### Public Pages
|
||||
|
||||
| Page | URL | Purpose |
|
||||
|------|-----|---------|
|
||||
| Landing | `/` | Marketing, sign up CTA |
|
||||
| Sign In | `/login` | OAuth + password login |
|
||||
| Sign Up | `/register` | Create account |
|
||||
| Docs | `/docs/*` | Documentation (separate site?) |
|
||||
|
||||
### Authenticated Pages
|
||||
|
||||
| Page | URL | Purpose |
|
||||
|------|-----|---------|
|
||||
| Dashboard | `/dashboard` | App list, quick stats |
|
||||
| App Details | `/apps/:id` | Single app overview |
|
||||
| App Settings | `/apps/:id/settings` | Edit app metadata |
|
||||
| App Versions | `/apps/:id/versions` | Version history |
|
||||
| Submit Version | `/apps/:id/versions/new` | Upload new version |
|
||||
| App Analytics | `/apps/:id/analytics` | Telemetry dashboard |
|
||||
| Create App | `/apps/new` | New app wizard |
|
||||
| API Keys | `/settings/keys` | Manage API keys |
|
||||
| Signing Keys | `/settings/signing` | Manage signing keys |
|
||||
| Profile | `/settings/profile` | Account settings |
|
||||
|
||||
---
|
||||
|
||||
## Wireframes
|
||||
|
||||
### Dashboard
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ [Logo] Dashboard Apps Docs Settings [Avatar ▼] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Welcome back, Developer! │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Total Apps │ │ Downloads │ │ Active Users│ │
|
||||
│ │ 12 │ │ 45,230 │ │ 1,234 │ │
|
||||
│ └─────────────┘ └─────────────┘ └─────────────┘ │
|
||||
│ │
|
||||
│ Your Apps [+ New App] │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ [Icon] My Calculator v1.2.0 Published 1.2K ↓ │ │
|
||||
│ │ [Icon] Notes App v2.0.1 Published 5.4K ↓ │ │
|
||||
│ │ [Icon] Weather Widget v0.9.0 In Review --- │ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### App Details
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ ← Back to Apps │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ [Icon] My Calculator │
|
||||
│ com.developer.calculator │
|
||||
│ ● Published │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Overview │ │ Versions │ │Analytics │ │ Settings │ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
|
||||
│ │
|
||||
│ Latest Version: 1.2.0 [Submit New Version] │
|
||||
│ Published: Jan 15, 2024 │
|
||||
│ Downloads: 1,234 │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────┐ │
|
||||
│ │ Downloads over time │ │
|
||||
│ │ [Chart: line graph] │ │
|
||||
│ └────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Recent Crashes [View All →] │
|
||||
│ • attempt to index nil (v1.2.0) - 23 reports │
|
||||
│ • memory limit exceeded (v1.1.0) - 5 reports │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Submit Version
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Submit New Version - My Calculator │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Step 2 of 3: Upload Package │
|
||||
│ ○ Details ● Upload ○ Review │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ │ │
|
||||
│ │ ┌─────────┐ │ │
|
||||
│ │ │ .mosis │ Drop your package here │ │
|
||||
│ │ │ 📦 │ or click to browse │ │
|
||||
│ │ └─────────┘ │ │
|
||||
│ │ │ │
|
||||
│ │ Max size: 50 MB │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ✓ Package validated │
|
||||
│ ✓ Signature verified │
|
||||
│ ✓ Permissions: storage, network │
|
||||
│ │
|
||||
│ [Back] [Continue to Review] │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack Options
|
||||
|
||||
### Option A: Next.js 14
|
||||
|
||||
```
|
||||
Framework: Next.js 14 (App Router)
|
||||
UI: shadcn/ui + Tailwind CSS
|
||||
State: React Query + Zustand
|
||||
Forms: React Hook Form + Zod
|
||||
Charts: Recharts or Tremor
|
||||
Auth: NextAuth.js
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| SSR for SEO landing page | Complexity |
|
||||
| Great developer experience | React knowledge required |
|
||||
| Full-stack capability | Heavier bundle |
|
||||
| Large ecosystem | |
|
||||
|
||||
### Option B: SvelteKit
|
||||
|
||||
```
|
||||
Framework: SvelteKit
|
||||
UI: Skeleton UI or custom
|
||||
State: Svelte stores
|
||||
Forms: Superforms
|
||||
Charts: Chart.js or LayerCake
|
||||
Auth: Lucia
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Fast, small bundles | Smaller ecosystem |
|
||||
| Simple state management | Fewer UI libraries |
|
||||
| Great DX | Less hiring pool |
|
||||
|
||||
### Option C: Astro + React Islands
|
||||
|
||||
```
|
||||
Framework: Astro
|
||||
UI: React components (islands)
|
||||
State: Nanostores
|
||||
Forms: React Hook Form
|
||||
Charts: Recharts
|
||||
Auth: Custom
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Ultra-fast static pages | More setup |
|
||||
| Partial hydration | Newer approach |
|
||||
| Use React where needed | Less documented patterns |
|
||||
|
||||
### Option D: htmx + Go Templates
|
||||
|
||||
```
|
||||
Framework: Go templates + htmx
|
||||
UI: Tailwind CSS
|
||||
State: Server-side
|
||||
Forms: Native HTML
|
||||
Charts: Chart.js
|
||||
Auth: Server sessions
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Simple, fast | Limited interactivity |
|
||||
| No JS framework | Less polished UX |
|
||||
| Server-rendered | Complex UI harder |
|
||||
|
||||
---
|
||||
|
||||
## UI Component Library
|
||||
|
||||
### Option A: shadcn/ui
|
||||
|
||||
```
|
||||
Base: Radix UI primitives
|
||||
Styling: Tailwind CSS
|
||||
Approach: Copy-paste components
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| High quality | React only |
|
||||
| Full control | Manual updates |
|
||||
| Accessible | |
|
||||
|
||||
### Option B: Tailwind UI
|
||||
|
||||
```
|
||||
Base: Headless UI
|
||||
Styling: Tailwind CSS
|
||||
Approach: Copy-paste templates
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Beautiful designs | Paid ($299) |
|
||||
| Production-ready | Templates, not components |
|
||||
|
||||
### Option C: Mantine
|
||||
|
||||
```
|
||||
Base: Custom components
|
||||
Styling: CSS-in-JS or CSS
|
||||
Approach: npm package
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Complete solution | Opinionated |
|
||||
| Many components | Larger bundle |
|
||||
| Good docs | |
|
||||
|
||||
---
|
||||
|
||||
## Key Features
|
||||
|
||||
### File Upload
|
||||
|
||||
```typescript
|
||||
// Drag and drop with progress
|
||||
<DropZone
|
||||
accept=".mosis"
|
||||
maxSize={50 * 1024 * 1024}
|
||||
onDrop={handleUpload}
|
||||
onProgress={setProgress}
|
||||
>
|
||||
<UploadIcon />
|
||||
<p>Drop your package here</p>
|
||||
</DropZone>
|
||||
```
|
||||
|
||||
### Real-time Validation
|
||||
|
||||
```typescript
|
||||
// Validate package client-side before upload
|
||||
async function validatePackage(file: File) {
|
||||
const zip = await JSZip.loadAsync(file);
|
||||
|
||||
// Check manifest
|
||||
const manifest = await zip.file('manifest.json')?.async('text');
|
||||
if (!manifest) throw new Error('Missing manifest.json');
|
||||
|
||||
const parsed = JSON.parse(manifest);
|
||||
ManifestSchema.parse(parsed);
|
||||
|
||||
// Check required files
|
||||
const entry = parsed.entry;
|
||||
if (!zip.file(entry)) {
|
||||
throw new Error(`Entry file not found: ${entry}`);
|
||||
}
|
||||
|
||||
return parsed;
|
||||
}
|
||||
```
|
||||
|
||||
### Analytics Dashboard
|
||||
|
||||
```typescript
|
||||
// Recharts example
|
||||
<LineChart data={downloads}>
|
||||
<XAxis dataKey="date" />
|
||||
<YAxis />
|
||||
<Tooltip />
|
||||
<Line type="monotone" dataKey="count" stroke="#8884d8" />
|
||||
</LineChart>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## State Management
|
||||
|
||||
### Server State (React Query)
|
||||
|
||||
```typescript
|
||||
// Fetch apps
|
||||
const { data: apps, isLoading } = useQuery({
|
||||
queryKey: ['apps'],
|
||||
queryFn: () => api.get('/apps'),
|
||||
});
|
||||
|
||||
// Create app mutation
|
||||
const createApp = useMutation({
|
||||
mutationFn: (data) => api.post('/apps', data),
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries(['apps']);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Client State (Zustand)
|
||||
|
||||
```typescript
|
||||
// UI state
|
||||
const useStore = create((set) => ({
|
||||
sidebarOpen: true,
|
||||
toggleSidebar: () => set((s) => ({ sidebarOpen: !s.sidebarOpen })),
|
||||
}));
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Authentication Flow
|
||||
|
||||
### Login Page
|
||||
|
||||
```tsx
|
||||
export default function LoginPage() {
|
||||
return (
|
||||
<div className="min-h-screen flex items-center justify-center">
|
||||
<Card className="w-96">
|
||||
<CardHeader>
|
||||
<h1>Sign in to Mosis</h1>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<Button onClick={() => signIn('github')} className="w-full">
|
||||
<GitHubIcon /> Continue with GitHub
|
||||
</Button>
|
||||
<Button onClick={() => signIn('google')} className="w-full">
|
||||
<GoogleIcon /> Continue with Google
|
||||
</Button>
|
||||
<Separator />
|
||||
<form onSubmit={handleEmailLogin}>
|
||||
<Input type="email" placeholder="Email" />
|
||||
<Input type="password" placeholder="Password" />
|
||||
<Button type="submit">Sign in</Button>
|
||||
</form>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Protected Routes
|
||||
|
||||
```tsx
|
||||
// Middleware (Next.js)
|
||||
export function middleware(request: NextRequest) {
|
||||
const token = request.cookies.get('token');
|
||||
|
||||
if (!token && request.nextUrl.pathname.startsWith('/dashboard')) {
|
||||
return NextResponse.redirect(new URL('/login', request.url));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Responsive Design
|
||||
|
||||
### Breakpoints
|
||||
|
||||
| Size | Width | Target |
|
||||
|------|-------|--------|
|
||||
| sm | 640px | Mobile landscape |
|
||||
| md | 768px | Tablet |
|
||||
| lg | 1024px | Desktop |
|
||||
| xl | 1280px | Large desktop |
|
||||
|
||||
### Mobile Navigation
|
||||
|
||||
```tsx
|
||||
// Hamburger menu for mobile
|
||||
<Sheet>
|
||||
<SheetTrigger className="md:hidden">
|
||||
<MenuIcon />
|
||||
</SheetTrigger>
|
||||
<SheetContent side="left">
|
||||
<Navigation />
|
||||
</SheetContent>
|
||||
</Sheet>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Accessibility
|
||||
|
||||
### Requirements
|
||||
|
||||
- [ ] Keyboard navigation
|
||||
- [ ] Screen reader support
|
||||
- [ ] Color contrast (WCAG AA)
|
||||
- [ ] Focus indicators
|
||||
- [ ] Alt text for images
|
||||
- [ ] Form labels
|
||||
- [ ] Error announcements
|
||||
|
||||
### Testing
|
||||
|
||||
```bash
|
||||
# Lighthouse audit
|
||||
npx lighthouse http://localhost:3000 --view
|
||||
|
||||
# axe-core
|
||||
npm install @axe-core/react
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] Framework selection
|
||||
- [ ] UI component library selection
|
||||
- [ ] Design system (colors, typography)
|
||||
- [ ] Page wireframes
|
||||
- [ ] Authentication flow
|
||||
- [ ] Dashboard implementation
|
||||
- [ ] App management pages
|
||||
- [ ] Version submission flow
|
||||
- [ ] Settings pages
|
||||
- [ ] Responsive design
|
||||
- [ ] Accessibility audit
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Dark mode support?
|
||||
2. Internationalization (i18n)?
|
||||
3. Custom domain for docs vs integrated?
|
||||
4. Email notifications UI?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [shadcn/ui](https://ui.shadcn.com/)
|
||||
- [Next.js App Router](https://nextjs.org/docs/app)
|
||||
- [Tailwind CSS](https://tailwindcss.com/)
|
||||
- [React Query](https://tanstack.com/query/latest)
|
||||
604
DEV_PORTAL_M06_API.md
Normal file
604
DEV_PORTAL_M06_API.md
Normal file
@@ -0,0 +1,604 @@
|
||||
# Milestone 6: App Store Backend API
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: REST API for app submission, review, and distribution.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The backend API serves the developer portal, CLI tools, and device-side app management. It handles authentication, app lifecycle, file storage, and telemetry ingestion.
|
||||
|
||||
---
|
||||
|
||||
## API Design Principles
|
||||
|
||||
1. **RESTful** - Standard HTTP methods and status codes
|
||||
2. **JSON** - Request and response bodies in JSON
|
||||
3. **Versioned** - `/v1/` prefix for breaking changes
|
||||
4. **Consistent** - Same patterns across all endpoints
|
||||
5. **Documented** - OpenAPI specification
|
||||
|
||||
---
|
||||
|
||||
## Base URL
|
||||
|
||||
```
|
||||
Production: https://api.mosis.dev/v1
|
||||
Staging: https://api.staging.mosis.dev/v1
|
||||
Local: http://localhost:8080/v1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
### Headers
|
||||
|
||||
```
|
||||
Authorization: Bearer <jwt_token>
|
||||
# or
|
||||
X-API-Key: mk_live_xxxxxxxx
|
||||
```
|
||||
|
||||
### Scopes
|
||||
|
||||
| Scope | Description |
|
||||
|-------|-------------|
|
||||
| `apps:read` | Read app metadata |
|
||||
| `apps:write` | Create/update apps |
|
||||
| `versions:upload` | Upload new versions |
|
||||
| `versions:publish` | Publish versions |
|
||||
| `telemetry:read` | Read analytics |
|
||||
| `keys:manage` | Manage API keys |
|
||||
|
||||
---
|
||||
|
||||
## Endpoints
|
||||
|
||||
### Authentication
|
||||
|
||||
```yaml
|
||||
POST /v1/auth/oauth/github:
|
||||
summary: Start GitHub OAuth flow
|
||||
response: { redirect_url: string }
|
||||
|
||||
GET /v1/auth/oauth/github/callback:
|
||||
summary: GitHub OAuth callback
|
||||
query:
|
||||
code: string
|
||||
state: string
|
||||
response: { access_token, refresh_token, user }
|
||||
|
||||
POST /v1/auth/oauth/google:
|
||||
summary: Start Google OAuth flow
|
||||
response: { redirect_url: string }
|
||||
|
||||
GET /v1/auth/oauth/google/callback:
|
||||
summary: Google OAuth callback
|
||||
|
||||
POST /v1/auth/refresh:
|
||||
summary: Refresh access token
|
||||
body: { refresh_token: string }
|
||||
response: { access_token, refresh_token }
|
||||
|
||||
POST /v1/auth/logout:
|
||||
summary: Invalidate tokens
|
||||
auth: required
|
||||
|
||||
GET /v1/auth/me:
|
||||
summary: Get current user
|
||||
auth: required
|
||||
response: Developer
|
||||
```
|
||||
|
||||
### Apps
|
||||
|
||||
```yaml
|
||||
GET /v1/apps:
|
||||
summary: List developer's apps
|
||||
auth: required
|
||||
query:
|
||||
status: draft | published | suspended
|
||||
page: number
|
||||
limit: number
|
||||
response: { apps: App[], total: number }
|
||||
|
||||
POST /v1/apps:
|
||||
summary: Create new app
|
||||
auth: required
|
||||
body:
|
||||
package_id: string # com.developer.appname
|
||||
name: string
|
||||
description?: string
|
||||
category?: string
|
||||
response: App
|
||||
|
||||
GET /v1/apps/:id:
|
||||
summary: Get app details
|
||||
auth: required
|
||||
response: App
|
||||
|
||||
PATCH /v1/apps/:id:
|
||||
summary: Update app metadata
|
||||
auth: required
|
||||
body:
|
||||
name?: string
|
||||
description?: string
|
||||
category?: string
|
||||
tags?: string[]
|
||||
response: App
|
||||
|
||||
DELETE /v1/apps/:id:
|
||||
summary: Delete app (if no published versions)
|
||||
auth: required
|
||||
response: { success: true }
|
||||
```
|
||||
|
||||
### App Versions
|
||||
|
||||
```yaml
|
||||
GET /v1/apps/:id/versions:
|
||||
summary: List app versions
|
||||
auth: required
|
||||
query:
|
||||
status: draft | review | approved | published | rejected
|
||||
page: number
|
||||
limit: number
|
||||
response: { versions: AppVersion[], total: number }
|
||||
|
||||
POST /v1/apps/:id/versions:
|
||||
summary: Create new version (get upload URL)
|
||||
auth: required
|
||||
body:
|
||||
version_name: string # 1.0.0
|
||||
version_code: number # 1
|
||||
release_notes?: string
|
||||
response:
|
||||
version: AppVersion
|
||||
upload_url: string # Presigned S3 URL
|
||||
upload_expires: string # ISO timestamp
|
||||
|
||||
PUT /v1/apps/:id/versions/:vid/upload-complete:
|
||||
summary: Mark upload as complete, trigger validation
|
||||
auth: required
|
||||
response: AppVersion
|
||||
|
||||
GET /v1/apps/:id/versions/:vid:
|
||||
summary: Get version details
|
||||
auth: required
|
||||
response: AppVersion
|
||||
|
||||
POST /v1/apps/:id/versions/:vid/submit:
|
||||
summary: Submit version for review
|
||||
auth: required
|
||||
response: AppVersion # status: review
|
||||
|
||||
POST /v1/apps/:id/versions/:vid/publish:
|
||||
summary: Publish approved version
|
||||
auth: required
|
||||
response: AppVersion # status: published
|
||||
|
||||
DELETE /v1/apps/:id/versions/:vid:
|
||||
summary: Delete draft version
|
||||
auth: required
|
||||
response: { success: true }
|
||||
```
|
||||
|
||||
### Public App Store
|
||||
|
||||
```yaml
|
||||
GET /v1/store/apps:
|
||||
summary: Browse/search published apps
|
||||
auth: none
|
||||
query:
|
||||
q: string # Search query
|
||||
category: string
|
||||
sort: popular | recent | name
|
||||
page: number
|
||||
limit: number
|
||||
response: { apps: PublicApp[], total: number }
|
||||
|
||||
GET /v1/store/apps/:package_id:
|
||||
summary: Get app store listing
|
||||
auth: none
|
||||
response: PublicApp
|
||||
|
||||
GET /v1/store/apps/:package_id/download:
|
||||
summary: Get download URL for latest version
|
||||
auth: none (or device token)
|
||||
response:
|
||||
download_url: string
|
||||
version: string
|
||||
size: number
|
||||
signature: string
|
||||
|
||||
GET /v1/store/apps/:package_id/versions/:version_code/download:
|
||||
summary: Get download URL for specific version
|
||||
auth: none
|
||||
response: { download_url, version, size, signature }
|
||||
```
|
||||
|
||||
### API Keys
|
||||
|
||||
```yaml
|
||||
GET /v1/keys:
|
||||
summary: List API keys
|
||||
auth: required
|
||||
response: { keys: APIKey[] }
|
||||
|
||||
POST /v1/keys:
|
||||
summary: Create API key
|
||||
auth: required
|
||||
body:
|
||||
name: string
|
||||
permissions: string[]
|
||||
expires_at?: string
|
||||
response:
|
||||
key: APIKey
|
||||
secret: string # Only shown once!
|
||||
|
||||
DELETE /v1/keys/:id:
|
||||
summary: Revoke API key
|
||||
auth: required
|
||||
response: { success: true }
|
||||
```
|
||||
|
||||
### Signing Keys
|
||||
|
||||
```yaml
|
||||
GET /v1/signing-keys:
|
||||
summary: List signing keys
|
||||
auth: required
|
||||
response: { keys: SigningKey[] }
|
||||
|
||||
POST /v1/signing-keys:
|
||||
summary: Register signing key
|
||||
auth: required
|
||||
body:
|
||||
name: string
|
||||
public_key: string # PEM format
|
||||
response: SigningKey
|
||||
|
||||
DELETE /v1/signing-keys/:id:
|
||||
summary: Revoke signing key
|
||||
auth: required
|
||||
response: { success: true }
|
||||
```
|
||||
|
||||
### Telemetry
|
||||
|
||||
```yaml
|
||||
POST /v1/telemetry/events:
|
||||
summary: Submit telemetry events (batch)
|
||||
auth: device token or API key
|
||||
body:
|
||||
events:
|
||||
- app_id: string
|
||||
event_type: string
|
||||
event_data: object
|
||||
timestamp: string
|
||||
response: { received: number }
|
||||
|
||||
POST /v1/telemetry/crash:
|
||||
summary: Submit crash report
|
||||
auth: device token or API key
|
||||
body:
|
||||
app_id: string
|
||||
app_version: string
|
||||
crash_type: string
|
||||
message: string
|
||||
stack_trace: string
|
||||
context: object
|
||||
timestamp: string
|
||||
response: { id: string }
|
||||
|
||||
GET /v1/apps/:id/analytics:
|
||||
summary: Get app analytics
|
||||
auth: required
|
||||
query:
|
||||
start_date: string
|
||||
end_date: string
|
||||
metrics: downloads | active_users | crashes
|
||||
response:
|
||||
data:
|
||||
- date: string
|
||||
downloads: number
|
||||
active_users: number
|
||||
crashes: number
|
||||
|
||||
GET /v1/apps/:id/crashes:
|
||||
summary: Get crash reports
|
||||
auth: required
|
||||
query:
|
||||
version?: string
|
||||
page: number
|
||||
limit: number
|
||||
response: { crashes: CrashReport[], total: number }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Models
|
||||
|
||||
### Developer
|
||||
|
||||
```typescript
|
||||
interface Developer {
|
||||
id: string;
|
||||
email: string;
|
||||
name: string;
|
||||
avatar_url?: string;
|
||||
verified: boolean;
|
||||
created_at: string;
|
||||
updated_at: string;
|
||||
}
|
||||
```
|
||||
|
||||
### App
|
||||
|
||||
```typescript
|
||||
interface App {
|
||||
id: string;
|
||||
package_id: string;
|
||||
name: string;
|
||||
description?: string;
|
||||
category?: string;
|
||||
tags: string[];
|
||||
status: 'draft' | 'published' | 'suspended';
|
||||
icon_url?: string;
|
||||
latest_version?: AppVersion;
|
||||
created_at: string;
|
||||
updated_at: string;
|
||||
}
|
||||
```
|
||||
|
||||
### AppVersion
|
||||
|
||||
```typescript
|
||||
interface AppVersion {
|
||||
id: string;
|
||||
app_id: string;
|
||||
version_name: string;
|
||||
version_code: number;
|
||||
package_url?: string;
|
||||
package_size?: number;
|
||||
signature?: string;
|
||||
permissions: string[];
|
||||
min_mosis_version?: string;
|
||||
release_notes?: string;
|
||||
status: 'draft' | 'uploading' | 'validating' | 'review' | 'approved' | 'published' | 'rejected';
|
||||
review_notes?: string;
|
||||
published_at?: string;
|
||||
created_at: string;
|
||||
}
|
||||
```
|
||||
|
||||
### PublicApp
|
||||
|
||||
```typescript
|
||||
interface PublicApp {
|
||||
package_id: string;
|
||||
name: string;
|
||||
description?: string;
|
||||
category?: string;
|
||||
tags: string[];
|
||||
icon_url?: string;
|
||||
author_name: string;
|
||||
latest_version: string;
|
||||
download_count: number;
|
||||
rating?: number;
|
||||
created_at: string;
|
||||
updated_at: string;
|
||||
}
|
||||
```
|
||||
|
||||
### APIKey
|
||||
|
||||
```typescript
|
||||
interface APIKey {
|
||||
id: string;
|
||||
name: string;
|
||||
key_prefix: string; // "mk_live_abc..."
|
||||
permissions: string[];
|
||||
last_used_at?: string;
|
||||
expires_at?: string;
|
||||
created_at: string;
|
||||
}
|
||||
```
|
||||
|
||||
### CrashReport
|
||||
|
||||
```typescript
|
||||
interface CrashReport {
|
||||
id: string;
|
||||
app_id: string;
|
||||
app_version: string;
|
||||
crash_type: string;
|
||||
message: string;
|
||||
stack_trace: string;
|
||||
context: object;
|
||||
occurrences: number;
|
||||
first_seen: string;
|
||||
last_seen: string;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Error Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"code": "VALIDATION_ERROR",
|
||||
"message": "Invalid package_id format",
|
||||
"details": {
|
||||
"field": "package_id",
|
||||
"constraint": "Must match pattern: ^[a-z][a-z0-9]*(\\.[a-z][a-z0-9]*)+$"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Error Codes
|
||||
|
||||
| Code | HTTP Status | Description |
|
||||
|------|-------------|-------------|
|
||||
| `UNAUTHORIZED` | 401 | Missing or invalid auth |
|
||||
| `FORBIDDEN` | 403 | Insufficient permissions |
|
||||
| `NOT_FOUND` | 404 | Resource not found |
|
||||
| `VALIDATION_ERROR` | 400 | Invalid request body |
|
||||
| `CONFLICT` | 409 | Resource already exists |
|
||||
| `RATE_LIMITED` | 429 | Too many requests |
|
||||
| `INTERNAL_ERROR` | 500 | Server error |
|
||||
|
||||
---
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
### Headers
|
||||
|
||||
```
|
||||
X-RateLimit-Limit: 1000
|
||||
X-RateLimit-Remaining: 999
|
||||
X-RateLimit-Reset: 1704067200
|
||||
```
|
||||
|
||||
### Limits
|
||||
|
||||
| Endpoint Category | Limit | Window |
|
||||
|-------------------|-------|--------|
|
||||
| Auth | 10 | 1 minute |
|
||||
| Read | 1000 | 1 hour |
|
||||
| Write | 100 | 1 hour |
|
||||
| Upload | 10 | 1 hour |
|
||||
| Telemetry | 10000 | 1 hour |
|
||||
|
||||
---
|
||||
|
||||
## Pagination
|
||||
|
||||
### Request
|
||||
|
||||
```
|
||||
GET /v1/apps?page=2&limit=20
|
||||
```
|
||||
|
||||
### Response
|
||||
|
||||
```json
|
||||
{
|
||||
"apps": [...],
|
||||
"pagination": {
|
||||
"page": 2,
|
||||
"limit": 20,
|
||||
"total": 45,
|
||||
"total_pages": 3
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Webhooks (Future)
|
||||
|
||||
```yaml
|
||||
POST /v1/webhooks:
|
||||
summary: Register webhook
|
||||
body:
|
||||
url: string
|
||||
events: string[] # version.published, crash.new
|
||||
secret: string
|
||||
|
||||
Webhook Payload:
|
||||
headers:
|
||||
X-Mosis-Signature: sha256=xxx
|
||||
body:
|
||||
event: string
|
||||
data: object
|
||||
timestamp: string
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## OpenAPI Specification
|
||||
|
||||
Full OpenAPI 3.0 spec will be generated and hosted at:
|
||||
- `https://api.mosis.dev/v1/openapi.json`
|
||||
- `https://api.mosis.dev/v1/docs` (Swagger UI)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Structure
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.go (or index.ts)
|
||||
├── api/
|
||||
│ ├── routes.go
|
||||
│ ├── middleware/
|
||||
│ │ ├── auth.go
|
||||
│ │ ├── ratelimit.go
|
||||
│ │ └── logging.go
|
||||
│ └── handlers/
|
||||
│ ├── auth.go
|
||||
│ ├── apps.go
|
||||
│ ├── versions.go
|
||||
│ ├── store.go
|
||||
│ ├── keys.go
|
||||
│ └── telemetry.go
|
||||
├── service/
|
||||
│ ├── app_service.go
|
||||
│ ├── version_service.go
|
||||
│ ├── auth_service.go
|
||||
│ └── storage_service.go
|
||||
├── repository/
|
||||
│ ├── app_repo.go
|
||||
│ ├── version_repo.go
|
||||
│ └── developer_repo.go
|
||||
├── domain/
|
||||
│ ├── app.go
|
||||
│ ├── version.go
|
||||
│ └── developer.go
|
||||
└── pkg/
|
||||
├── validator/
|
||||
└── crypto/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] OpenAPI specification
|
||||
- [ ] Authentication middleware
|
||||
- [ ] Rate limiting middleware
|
||||
- [ ] Auth endpoints
|
||||
- [ ] Apps CRUD endpoints
|
||||
- [ ] Versions endpoints with upload flow
|
||||
- [ ] Store public endpoints
|
||||
- [ ] API keys management
|
||||
- [ ] Signing keys management
|
||||
- [ ] Telemetry ingestion
|
||||
- [ ] Error handling
|
||||
- [ ] Request validation
|
||||
- [ ] Integration tests
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. GraphQL alongside REST?
|
||||
2. WebSocket for real-time review status?
|
||||
3. Batch operations for bulk updates?
|
||||
4. API versioning strategy (URL vs header)?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [REST API Design Guidelines](https://github.com/microsoft/api-guidelines)
|
||||
- [OpenAPI 3.0 Specification](https://swagger.io/specification/)
|
||||
- [HTTP Status Codes](https://httpstatuses.com/)
|
||||
451
DEV_PORTAL_M07_STORAGE.md
Normal file
451
DEV_PORTAL_M07_STORAGE.md
Normal file
@@ -0,0 +1,451 @@
|
||||
# Milestone 7: CDN & Storage
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Scalable storage for app packages and assets with global distribution.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Storage handles app packages, icons, screenshots, and serves downloads to devices worldwide. Must be cost-effective, fast, and reliable.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional
|
||||
|
||||
- Store app packages (.mosis files)
|
||||
- Store icons (multiple sizes)
|
||||
- Store screenshots (optional)
|
||||
- Serve downloads globally
|
||||
- Support presigned upload URLs
|
||||
- Version retention (keep old versions)
|
||||
|
||||
### Non-Functional
|
||||
|
||||
| Requirement | Target |
|
||||
|-------------|--------|
|
||||
| Upload speed | < 30s for 50MB |
|
||||
| Download latency | < 100ms (p50) |
|
||||
| Availability | 99.9% |
|
||||
| Durability | 99.999999999% (11 nines) |
|
||||
| Cost | Minimize egress fees |
|
||||
|
||||
---
|
||||
|
||||
## Storage Structure
|
||||
|
||||
```
|
||||
/packages/
|
||||
/{developer_id}/
|
||||
/{app_id}/
|
||||
/{version_code}/
|
||||
package.mosis
|
||||
manifest.json (extracted for quick access)
|
||||
|
||||
/assets/
|
||||
/{app_id}/
|
||||
icon-32.png
|
||||
icon-64.png
|
||||
icon-128.png
|
||||
screenshots/
|
||||
1.png
|
||||
2.png
|
||||
3.png
|
||||
|
||||
/temp/
|
||||
/{upload_id}/
|
||||
package.mosis (pending validation)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Options Analysis
|
||||
|
||||
### Option A: Cloudflare R2
|
||||
|
||||
```
|
||||
Storage: Object storage (S3-compatible)
|
||||
CDN: Cloudflare network (automatic)
|
||||
Egress: FREE (zero egress fees)
|
||||
Pricing: $0.015/GB storage, $4.50/million requests
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| No egress fees | Newer service |
|
||||
| Global CDN included | Fewer regions than S3 |
|
||||
| S3-compatible API | Less tooling |
|
||||
| Workers integration | |
|
||||
|
||||
#### Cost Estimate (10K apps, 100GB)
|
||||
|
||||
| Component | Monthly Cost |
|
||||
|-----------|--------------|
|
||||
| Storage (100GB) | $1.50 |
|
||||
| Requests (1M) | $4.50 |
|
||||
| Egress | $0 |
|
||||
| **Total** | **~$6** |
|
||||
|
||||
---
|
||||
|
||||
### Option B: AWS S3 + CloudFront
|
||||
|
||||
```
|
||||
Storage: S3 Standard
|
||||
CDN: CloudFront
|
||||
Egress: $0.085-0.12/GB (varies by region)
|
||||
Pricing: $0.023/GB storage
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Most mature | Egress costs add up |
|
||||
| Best tooling | Complex pricing |
|
||||
| All regions | Need CloudFront config |
|
||||
| IAM integration | |
|
||||
|
||||
#### Cost Estimate (10K apps, 100GB, 1TB egress)
|
||||
|
||||
| Component | Monthly Cost |
|
||||
|-----------|--------------|
|
||||
| Storage (100GB) | $2.30 |
|
||||
| CloudFront (1TB) | $85 |
|
||||
| Requests | ~$5 |
|
||||
| **Total** | **~$92** |
|
||||
|
||||
---
|
||||
|
||||
### Option C: Backblaze B2 + Cloudflare
|
||||
|
||||
```
|
||||
Storage: Backblaze B2
|
||||
CDN: Cloudflare (free egress via Bandwidth Alliance)
|
||||
Egress: FREE (through Cloudflare)
|
||||
Pricing: $0.005/GB storage
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Cheapest storage | Two services to manage |
|
||||
| Free egress via CF | B2 API slightly different |
|
||||
| Good reliability | Need CF proxy setup |
|
||||
|
||||
#### Cost Estimate (10K apps, 100GB)
|
||||
|
||||
| Component | Monthly Cost |
|
||||
|-----------|--------------|
|
||||
| Storage (100GB) | $0.50 |
|
||||
| Egress (via CF) | $0 |
|
||||
| Requests | ~$0.40 |
|
||||
| **Total** | **~$1** |
|
||||
|
||||
---
|
||||
|
||||
### Option D: Self-hosted MinIO
|
||||
|
||||
```
|
||||
Storage: MinIO on VPS
|
||||
CDN: Cloudflare proxy
|
||||
Egress: VPS bandwidth
|
||||
Pricing: VPS cost only
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Full control | Ops overhead |
|
||||
| S3-compatible | Need to manage |
|
||||
| Predictable cost | Scaling complexity |
|
||||
|
||||
#### Cost Estimate
|
||||
|
||||
| Component | Monthly Cost |
|
||||
|-----------|--------------|
|
||||
| VPS (500GB SSD) | $20-50 |
|
||||
| Cloudflare | $0 (free tier) |
|
||||
| **Total** | **~$20-50** |
|
||||
|
||||
---
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Primary: Cloudflare R2**
|
||||
- Zero egress fees (biggest cost saver)
|
||||
- S3-compatible (easy migration)
|
||||
- Built-in global CDN
|
||||
- Good enough tooling
|
||||
|
||||
**Fallback: Backblaze B2 + Cloudflare**
|
||||
- If R2 has issues
|
||||
- Even cheaper storage
|
||||
- Slightly more setup
|
||||
|
||||
---
|
||||
|
||||
## Upload Flow
|
||||
|
||||
### Presigned URL Approach
|
||||
|
||||
```
|
||||
┌────────┐ ┌─────────┐ ┌─────────┐
|
||||
│ Client │───►│ API │───►│ R2 │
|
||||
└────────┘ └─────────┘ └─────────┘
|
||||
│ │ │
|
||||
│ 1. Request upload URL │
|
||||
│◄─────────────┤ │
|
||||
│ (presigned URL) │
|
||||
│ │
|
||||
│ 2. Upload directly │
|
||||
│────────────────────────────►│
|
||||
│ │
|
||||
│ 3. Notify complete │
|
||||
│─────────────►│ │
|
||||
│ │ 4. Validate │
|
||||
│ │─────────────►│
|
||||
│ │ │
|
||||
│ 5. Confirm │ │
|
||||
│◄─────────────┤ │
|
||||
```
|
||||
|
||||
### API Implementation
|
||||
|
||||
```go
|
||||
// 1. Request upload URL
|
||||
func CreateVersion(c *gin.Context) {
|
||||
// Create version record
|
||||
version := Version{
|
||||
AppID: appID,
|
||||
VersionCode: req.VersionCode,
|
||||
Status: "uploading",
|
||||
}
|
||||
db.Create(&version)
|
||||
|
||||
// Generate presigned URL
|
||||
key := fmt.Sprintf("temp/%s/package.mosis", version.ID)
|
||||
url, err := r2.PresignPut(key, 15*time.Minute)
|
||||
|
||||
c.JSON(200, gin.H{
|
||||
"version": version,
|
||||
"upload_url": url,
|
||||
"expires": time.Now().Add(15 * time.Minute),
|
||||
})
|
||||
}
|
||||
|
||||
// 3. Upload complete notification
|
||||
func UploadComplete(c *gin.Context) {
|
||||
// Move from temp to final location
|
||||
tempKey := fmt.Sprintf("temp/%s/package.mosis", versionID)
|
||||
finalKey := fmt.Sprintf("packages/%s/%s/%d/package.mosis",
|
||||
developerID, appID, versionCode)
|
||||
|
||||
r2.Copy(tempKey, finalKey)
|
||||
r2.Delete(tempKey)
|
||||
|
||||
// Update version status
|
||||
version.Status = "validating"
|
||||
version.PackageURL = finalKey
|
||||
|
||||
// Trigger async validation
|
||||
queue.Publish("validate-package", version.ID)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Download Flow
|
||||
|
||||
### Public Downloads
|
||||
|
||||
```go
|
||||
// Get download URL (short-lived)
|
||||
func GetDownloadURL(c *gin.Context) {
|
||||
version := getLatestPublishedVersion(packageID)
|
||||
|
||||
// Generate presigned download URL (1 hour)
|
||||
url, _ := r2.PresignGet(version.PackageURL, 1*time.Hour)
|
||||
|
||||
c.JSON(200, gin.H{
|
||||
"download_url": url,
|
||||
"version": version.VersionName,
|
||||
"size": version.PackageSize,
|
||||
"signature": version.Signature,
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
### CDN Caching
|
||||
|
||||
```
|
||||
Cache-Control: public, max-age=86400
|
||||
```
|
||||
|
||||
- Packages are immutable (version code = unique)
|
||||
- Cache aggressively at edge
|
||||
- Invalidate only if package is pulled
|
||||
|
||||
---
|
||||
|
||||
## Icon/Screenshot Handling
|
||||
|
||||
### Upload
|
||||
|
||||
```go
|
||||
// Icons uploaded with app creation/update
|
||||
func UploadIcon(c *gin.Context) {
|
||||
file, _ := c.FormFile("icon")
|
||||
|
||||
// Validate dimensions
|
||||
img, _ := png.Decode(file)
|
||||
if img.Bounds().Dx() != img.Bounds().Dy() {
|
||||
return error("Icon must be square")
|
||||
}
|
||||
|
||||
// Generate multiple sizes
|
||||
sizes := []int{32, 64, 128}
|
||||
for _, size := range sizes {
|
||||
resized := resize(img, size, size)
|
||||
key := fmt.Sprintf("assets/%s/icon-%d.png", appID, size)
|
||||
r2.Put(key, resized)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Serving
|
||||
|
||||
```
|
||||
https://cdn.mosis.dev/assets/{app_id}/icon-64.png
|
||||
```
|
||||
|
||||
- Public read access
|
||||
- Long cache TTL (icons rarely change)
|
||||
- Cloudflare image optimization (optional)
|
||||
|
||||
---
|
||||
|
||||
## Retention Policy
|
||||
|
||||
### Packages
|
||||
|
||||
| Status | Retention |
|
||||
|--------|-----------|
|
||||
| Published (current) | Forever |
|
||||
| Published (old) | 1 year |
|
||||
| Draft | 30 days |
|
||||
| Rejected | 7 days |
|
||||
| Failed validation | 24 hours |
|
||||
|
||||
### Temp Uploads
|
||||
|
||||
- Delete after 1 hour if not completed
|
||||
- Lifecycle rule on `/temp/` prefix
|
||||
|
||||
### Implementation
|
||||
|
||||
```yaml
|
||||
# R2 Lifecycle Rules
|
||||
rules:
|
||||
- prefix: "temp/"
|
||||
expiration_days: 1
|
||||
|
||||
- prefix: "packages/"
|
||||
tags:
|
||||
status: draft
|
||||
expiration_days: 30
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### R2 Built-in
|
||||
|
||||
- 11 nines durability
|
||||
- Automatic replication within region
|
||||
- No additional backup needed for packages
|
||||
|
||||
### Metadata Backup
|
||||
|
||||
- Package metadata in PostgreSQL
|
||||
- PostgreSQL backups cover this
|
||||
- Can regenerate URLs from DB
|
||||
|
||||
---
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Metrics to Track
|
||||
|
||||
| Metric | Source |
|
||||
|--------|--------|
|
||||
| Upload success rate | API logs |
|
||||
| Download latency | Cloudflare analytics |
|
||||
| Storage usage | R2 dashboard |
|
||||
| Bandwidth | R2 dashboard |
|
||||
| Error rate | API logs |
|
||||
|
||||
### Alerts
|
||||
|
||||
- Upload failure rate > 5%
|
||||
- Download error rate > 1%
|
||||
- Storage > 80% of budget
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
### Access Control
|
||||
|
||||
```
|
||||
Packages:
|
||||
- Write: API only (presigned URLs)
|
||||
- Read: Public (presigned URLs)
|
||||
|
||||
Assets:
|
||||
- Write: API only
|
||||
- Read: Public (direct CDN)
|
||||
|
||||
Temp:
|
||||
- Write: Presigned URLs (15 min)
|
||||
- Read: None (API internal only)
|
||||
```
|
||||
|
||||
### Signed URLs
|
||||
|
||||
```go
|
||||
// Presigned URL with expiration
|
||||
url := r2.PresignPut(key, 15*time.Minute, PutOptions{
|
||||
ContentType: "application/octet-stream",
|
||||
ContentLength: maxSize,
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] R2 bucket setup
|
||||
- [ ] Presigned URL generation
|
||||
- [ ] Upload flow implementation
|
||||
- [ ] Download URL generation
|
||||
- [ ] Icon/screenshot upload
|
||||
- [ ] Lifecycle rules for cleanup
|
||||
- [ ] Monitoring dashboard
|
||||
- [ ] Cost tracking
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Multi-region storage for lower latency?
|
||||
2. Package compression (gzip/brotli)?
|
||||
3. Delta updates storage structure?
|
||||
4. Screenshot requirements (dimensions, count)?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Cloudflare R2 Documentation](https://developers.cloudflare.com/r2/)
|
||||
- [S3 Presigned URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html)
|
||||
- [Backblaze B2 + Cloudflare](https://www.backblaze.com/b2/docs/cloud_to_cloud.html)
|
||||
503
DEV_PORTAL_M08_TELEMETRY.md
Normal file
503
DEV_PORTAL_M08_TELEMETRY.md
Normal file
@@ -0,0 +1,503 @@
|
||||
# Milestone 8: Telemetry System
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Collect app usage analytics and crash reports while respecting privacy.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Telemetry provides developers with insights into app usage, performance, and crashes. Must balance usefulness with user privacy.
|
||||
|
||||
---
|
||||
|
||||
## Privacy Principles
|
||||
|
||||
1. **Minimal collection** - Only what's necessary
|
||||
2. **No PII by default** - Anonymized device IDs
|
||||
3. **Transparency** - Users know what's collected
|
||||
4. **Opt-out available** - Users can disable
|
||||
5. **Data retention limits** - Auto-delete old data
|
||||
6. **GDPR compliance** - Export/delete on request
|
||||
|
||||
---
|
||||
|
||||
## Event Types
|
||||
|
||||
### Automatic Events (Default)
|
||||
|
||||
| Event | Description | Data |
|
||||
|-------|-------------|------|
|
||||
| `app_start` | App launched | version, mosis_version |
|
||||
| `app_stop` | App closed | duration_seconds |
|
||||
| `app_crash` | Unhandled error | crash_type, message |
|
||||
| `lua_error` | Lua runtime error | message, stack (no user data) |
|
||||
|
||||
### Performance Events (Default)
|
||||
|
||||
| Event | Description | Data |
|
||||
|-------|-------------|------|
|
||||
| `perf_frame` | Frame time (sampled) | avg_ms, p95_ms |
|
||||
| `perf_memory` | Memory usage | used_mb, limit_mb |
|
||||
| `perf_startup` | Startup time | duration_ms |
|
||||
|
||||
### Usage Events (Opt-in)
|
||||
|
||||
| Event | Description | Data |
|
||||
|-------|-------------|------|
|
||||
| `screen_view` | Screen navigation | screen_name |
|
||||
| `button_click` | UI interaction | element_id |
|
||||
| `feature_used` | Feature usage | feature_name |
|
||||
|
||||
---
|
||||
|
||||
## Data Schema
|
||||
|
||||
### Event Payload
|
||||
|
||||
```json
|
||||
{
|
||||
"app_id": "com.developer.myapp",
|
||||
"app_version": "1.2.0",
|
||||
"mosis_version": "1.0.0",
|
||||
"device_id": "sha256_hashed_id",
|
||||
"session_id": "uuid",
|
||||
"events": [
|
||||
{
|
||||
"type": "app_start",
|
||||
"timestamp": "2024-01-15T10:30:00Z",
|
||||
"data": {}
|
||||
},
|
||||
{
|
||||
"type": "screen_view",
|
||||
"timestamp": "2024-01-15T10:30:05Z",
|
||||
"data": {
|
||||
"screen_name": "home"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Crash Report Payload
|
||||
|
||||
```json
|
||||
{
|
||||
"app_id": "com.developer.myapp",
|
||||
"app_version": "1.2.0",
|
||||
"mosis_version": "1.0.0",
|
||||
"device_id": "sha256_hashed_id",
|
||||
"timestamp": "2024-01-15T10:35:00Z",
|
||||
"crash": {
|
||||
"type": "lua_error",
|
||||
"message": "attempt to index nil value 'user'",
|
||||
"stack_trace": "main.lua:42: in function 'loadUser'\nmain.lua:15: in main chunk",
|
||||
"context": {
|
||||
"screen": "profile.rml",
|
||||
"memory_mb": 45,
|
||||
"uptime_seconds": 300
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Device ID Hashing
|
||||
|
||||
```lua
|
||||
-- On device
|
||||
local raw_id = get_android_id() -- or similar
|
||||
local hashed = sha256(raw_id .. "mosis_salt_" .. app_id)
|
||||
-- Result: "a3f2b1c4d5e6..."
|
||||
|
||||
-- Cannot reverse to original device ID
|
||||
-- Different per app (can't track across apps)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Collection Architecture
|
||||
|
||||
```
|
||||
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
|
||||
│ Device │────►│ Batch │────►│ API │────►│ Storage │
|
||||
│ │ │ Queue │ │ │ │ │
|
||||
└──────────┘ └──────────┘ └──────────┘ └──────────┘
|
||||
│
|
||||
│ Every 60s or
|
||||
│ on app close
|
||||
▼
|
||||
┌──────────┐
|
||||
│ Upload │
|
||||
└──────────┘
|
||||
```
|
||||
|
||||
### Client-Side Batching
|
||||
|
||||
```lua
|
||||
-- TelemetryManager on device
|
||||
local events = {}
|
||||
local last_flush = os.time()
|
||||
|
||||
function track(event_type, data)
|
||||
if not telemetry_enabled then return end
|
||||
|
||||
table.insert(events, {
|
||||
type = event_type,
|
||||
timestamp = os.date("!%Y-%m-%dT%H:%M:%SZ"),
|
||||
data = data or {}
|
||||
})
|
||||
|
||||
-- Flush if batch is large or time elapsed
|
||||
if #events >= 50 or (os.time() - last_flush) > 60 then
|
||||
flush()
|
||||
end
|
||||
end
|
||||
|
||||
function flush()
|
||||
if #events == 0 then return end
|
||||
|
||||
local payload = {
|
||||
app_id = APP_ID,
|
||||
app_version = APP_VERSION,
|
||||
device_id = HASHED_DEVICE_ID,
|
||||
events = events
|
||||
}
|
||||
|
||||
-- Async HTTP POST
|
||||
http.post(TELEMETRY_URL, json.encode(payload))
|
||||
|
||||
events = {}
|
||||
last_flush = os.time()
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Storage Options
|
||||
|
||||
### Option A: PostgreSQL + TimescaleDB
|
||||
|
||||
```sql
|
||||
-- Hypertable for time-series data
|
||||
CREATE TABLE telemetry_events (
|
||||
time TIMESTAMPTZ NOT NULL,
|
||||
app_id TEXT NOT NULL,
|
||||
device_id TEXT NOT NULL,
|
||||
session_id TEXT,
|
||||
event_type TEXT NOT NULL,
|
||||
event_data JSONB,
|
||||
app_version TEXT,
|
||||
mosis_version TEXT
|
||||
);
|
||||
|
||||
SELECT create_hypertable('telemetry_events', 'time');
|
||||
|
||||
-- Continuous aggregate for daily stats
|
||||
CREATE MATERIALIZED VIEW daily_stats
|
||||
WITH (timescaledb.continuous) AS
|
||||
SELECT
|
||||
time_bucket('1 day', time) AS day,
|
||||
app_id,
|
||||
event_type,
|
||||
COUNT(*) as count,
|
||||
COUNT(DISTINCT device_id) as unique_devices
|
||||
FROM telemetry_events
|
||||
GROUP BY day, app_id, event_type;
|
||||
```
|
||||
|
||||
### Option B: ClickHouse
|
||||
|
||||
```sql
|
||||
CREATE TABLE telemetry_events (
|
||||
timestamp DateTime,
|
||||
app_id String,
|
||||
device_id String,
|
||||
session_id String,
|
||||
event_type String,
|
||||
event_data String, -- JSON
|
||||
app_version String,
|
||||
mosis_version String
|
||||
) ENGINE = MergeTree()
|
||||
PARTITION BY toYYYYMM(timestamp)
|
||||
ORDER BY (app_id, timestamp);
|
||||
```
|
||||
|
||||
### Option C: Custom + PostgreSQL
|
||||
|
||||
```
|
||||
Raw events → Write to append-only log
|
||||
Aggregator → Process hourly → Write to PostgreSQL
|
||||
Cleanup → Delete raw after 24h
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Aggregation
|
||||
|
||||
### Pre-computed Metrics
|
||||
|
||||
| Metric | Granularity | Retention |
|
||||
|--------|-------------|-----------|
|
||||
| Daily active users | Day | 2 years |
|
||||
| Event counts | Day | 1 year |
|
||||
| Crash counts | Day | 1 year |
|
||||
| Session duration | Day | 90 days |
|
||||
| Performance percentiles | Day | 90 days |
|
||||
|
||||
### Aggregation Queries
|
||||
|
||||
```sql
|
||||
-- Daily active users
|
||||
SELECT
|
||||
DATE_TRUNC('day', time) as day,
|
||||
COUNT(DISTINCT device_id) as dau
|
||||
FROM telemetry_events
|
||||
WHERE app_id = $1
|
||||
AND event_type = 'app_start'
|
||||
AND time > NOW() - INTERVAL '30 days'
|
||||
GROUP BY day
|
||||
ORDER BY day;
|
||||
|
||||
-- Crash rate by version
|
||||
SELECT
|
||||
app_version,
|
||||
COUNT(*) FILTER (WHERE event_type = 'app_crash') as crashes,
|
||||
COUNT(*) FILTER (WHERE event_type = 'app_start') as starts,
|
||||
ROUND(
|
||||
100.0 * COUNT(*) FILTER (WHERE event_type = 'app_crash') /
|
||||
NULLIF(COUNT(*) FILTER (WHERE event_type = 'app_start'), 0),
|
||||
2
|
||||
) as crash_rate
|
||||
FROM telemetry_events
|
||||
WHERE app_id = $1
|
||||
AND time > NOW() - INTERVAL '7 days'
|
||||
GROUP BY app_version;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Crash Grouping
|
||||
|
||||
### Stack Trace Fingerprinting
|
||||
|
||||
```go
|
||||
func fingerprintCrash(crash CrashReport) string {
|
||||
// Normalize stack trace
|
||||
normalized := normalizeStackTrace(crash.StackTrace)
|
||||
|
||||
// Hash key components
|
||||
key := fmt.Sprintf("%s:%s:%s",
|
||||
crash.CrashType,
|
||||
crash.Message,
|
||||
normalized,
|
||||
)
|
||||
|
||||
return sha256(key)[:16]
|
||||
}
|
||||
|
||||
func normalizeStackTrace(stack string) string {
|
||||
// Remove line numbers (they change with code updates)
|
||||
// Remove memory addresses
|
||||
// Keep function names and file names
|
||||
re := regexp.MustCompile(`:\d+:`)
|
||||
return re.ReplaceAllString(stack, ":?:")
|
||||
}
|
||||
```
|
||||
|
||||
### Crash Groups Table
|
||||
|
||||
```sql
|
||||
CREATE TABLE crash_groups (
|
||||
id UUID PRIMARY KEY,
|
||||
app_id TEXT NOT NULL,
|
||||
fingerprint TEXT NOT NULL,
|
||||
crash_type TEXT NOT NULL,
|
||||
message TEXT,
|
||||
sample_stack_trace TEXT,
|
||||
first_seen TIMESTAMPTZ NOT NULL,
|
||||
last_seen TIMESTAMPTZ NOT NULL,
|
||||
occurrence_count INT DEFAULT 1,
|
||||
affected_versions TEXT[],
|
||||
status TEXT DEFAULT 'open', -- open, resolved, ignored
|
||||
UNIQUE(app_id, fingerprint)
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Developer Dashboard
|
||||
|
||||
### Metrics View
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Analytics - My Calculator │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Date Range: [Last 30 days ▼] │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Daily Users │ │ Crashes │ │ Crash-free │ │
|
||||
│ │ 1,234 │ │ 23 │ │ 98.1% │ │
|
||||
│ │ ▲ +12% │ │ ▼ -45% │ │ ▲ +2% │ │
|
||||
│ └─────────────┘ └─────────────┘ └─────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────┐ │
|
||||
│ │ Daily Active Users │ │
|
||||
│ │ [Line chart showing DAU over time] │ │
|
||||
│ └────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────┐ │
|
||||
│ │ Version Distribution │ │
|
||||
│ │ [Pie chart: v1.2.0: 60%, v1.1.0: 30%, v1.0.0: 10%]│ │
|
||||
│ └────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Crashes View
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Crashes - My Calculator │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Filter: [All versions ▼] [Open ▼] │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ ● attempt to index nil value 'user' │ │
|
||||
│ │ lua_error • 156 occurrences • v1.2.0 │ │
|
||||
│ │ First: Jan 10 • Last: Jan 15 │ │
|
||||
│ │ [View] │ │
|
||||
│ ├──────────────────────────────────────────────────────┤ │
|
||||
│ │ ● memory limit exceeded │ │
|
||||
│ │ sandbox_error • 23 occurrences • v1.1.0, v1.2.0 │ │
|
||||
│ │ First: Jan 5 • Last: Jan 14 │ │
|
||||
│ │ [View] │ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
```yaml
|
||||
# Ingestion (from devices)
|
||||
POST /v1/telemetry/events:
|
||||
auth: device_token or api_key
|
||||
body: { app_id, device_id, events[] }
|
||||
response: { received: number }
|
||||
|
||||
POST /v1/telemetry/crash:
|
||||
auth: device_token or api_key
|
||||
body: { app_id, device_id, crash }
|
||||
response: { id: string }
|
||||
|
||||
# Dashboard (for developers)
|
||||
GET /v1/apps/:id/analytics/overview:
|
||||
auth: required
|
||||
query: { start_date, end_date }
|
||||
response: { dau, crashes, crash_free_rate, ... }
|
||||
|
||||
GET /v1/apps/:id/analytics/events:
|
||||
auth: required
|
||||
query: { start_date, end_date, event_type }
|
||||
response: { data: [{ date, count, unique_devices }] }
|
||||
|
||||
GET /v1/apps/:id/crashes:
|
||||
auth: required
|
||||
query: { version, status, page, limit }
|
||||
response: { crashes: CrashGroup[], total }
|
||||
|
||||
GET /v1/apps/:id/crashes/:fingerprint:
|
||||
auth: required
|
||||
response: { crash_group, recent_occurrences[] }
|
||||
|
||||
PATCH /v1/apps/:id/crashes/:fingerprint:
|
||||
auth: required
|
||||
body: { status: 'resolved' | 'ignored' }
|
||||
response: { crash_group }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Retention
|
||||
|
||||
| Data Type | Retention | Reason |
|
||||
|-----------|-----------|--------|
|
||||
| Raw events | 7 days | Debugging |
|
||||
| Daily aggregates | 2 years | Trends |
|
||||
| Crash reports | 90 days | Investigation |
|
||||
| Crash groups | Forever | Issue tracking |
|
||||
|
||||
### Cleanup Job
|
||||
|
||||
```sql
|
||||
-- Run daily
|
||||
DELETE FROM telemetry_events
|
||||
WHERE time < NOW() - INTERVAL '7 days';
|
||||
|
||||
DELETE FROM crash_reports
|
||||
WHERE timestamp < NOW() - INTERVAL '90 days';
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Privacy Controls
|
||||
|
||||
### User Settings
|
||||
|
||||
```
|
||||
Settings > Privacy > Analytics
|
||||
├── [✓] Send crash reports (helps developers fix bugs)
|
||||
├── [ ] Send usage analytics (how you use apps)
|
||||
└── [Request Data Deletion]
|
||||
```
|
||||
|
||||
### GDPR Endpoints
|
||||
|
||||
```yaml
|
||||
# User requests their data
|
||||
GET /v1/privacy/export:
|
||||
auth: user_token
|
||||
response: { download_url } # JSON export of all data
|
||||
|
||||
# User requests deletion
|
||||
DELETE /v1/privacy/data:
|
||||
auth: user_token
|
||||
response: { status: 'scheduled' } # Delete within 30 days
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] Event schema specification
|
||||
- [ ] Client-side SDK for batching
|
||||
- [ ] Ingestion API endpoints
|
||||
- [ ] Storage setup (TimescaleDB or ClickHouse)
|
||||
- [ ] Aggregation jobs
|
||||
- [ ] Crash grouping logic
|
||||
- [ ] Developer dashboard
|
||||
- [ ] Privacy controls
|
||||
- [ ] Data retention automation
|
||||
- [ ] GDPR export/delete
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Real-time crash alerts (email/Slack)?
|
||||
2. Sampling for high-volume apps?
|
||||
3. Custom events API for developers?
|
||||
4. Benchmarks/comparisons with similar apps?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [GDPR Requirements](https://gdpr.eu/)
|
||||
- [TimescaleDB Best Practices](https://docs.timescale.com/timescaledb/latest/)
|
||||
- [Sentry Crash Grouping](https://docs.sentry.io/product/data-management-settings/event-grouping/)
|
||||
474
DEV_PORTAL_M09_REVIEW.md
Normal file
474
DEV_PORTAL_M09_REVIEW.md
Normal file
@@ -0,0 +1,474 @@
|
||||
# Milestone 9: App Review System
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Automated and manual review process for app submissions.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The review system ensures apps meet quality and security standards before publication. Balances automation with manual review for edge cases.
|
||||
|
||||
---
|
||||
|
||||
## Review Pipeline
|
||||
|
||||
```
|
||||
┌─────────┐ ┌───────────┐ ┌─────────┐ ┌──────────┐ ┌───────────┐
|
||||
│ Submit │──►│ Automated │──►│ Manual │──►│ Approved │──►│ Published │
|
||||
│ │ │ Checks │ │ Review │ │ │ │ │
|
||||
└─────────┘ └───────────┘ └─────────┘ └──────────┘ └───────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────┐ ┌─────────┐
|
||||
│ Failed │ │Rejected │
|
||||
│(auto-fix)│ │(feedback)│
|
||||
└─────────┘ └─────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Automated Checks
|
||||
|
||||
### Tier 1: Package Validation (Blocking)
|
||||
|
||||
| Check | Description | Action on Fail |
|
||||
|-------|-------------|----------------|
|
||||
| Valid ZIP | Package is valid ZIP format | Reject |
|
||||
| Manifest exists | manifest.json at root | Reject |
|
||||
| Manifest valid | JSON parses, required fields | Reject |
|
||||
| Signature valid | Package signed with registered key | Reject |
|
||||
| Entry exists | Entry point file exists | Reject |
|
||||
| Size limits | Under max package size | Reject |
|
||||
| No forbidden files | No .exe, .dll, etc. | Reject |
|
||||
|
||||
### Tier 2: Content Validation (Blocking)
|
||||
|
||||
| Check | Description | Action on Fail |
|
||||
|-------|-------------|----------------|
|
||||
| RML valid | All RML files parse | Reject |
|
||||
| RCSS valid | All RCSS files parse | Reject |
|
||||
| Lua syntax | All Lua files parse | Reject |
|
||||
| Icons valid | Icons are valid images | Reject |
|
||||
| Icon sizes | Required icon sizes present | Reject |
|
||||
| Path safety | No path traversal attempts | Reject |
|
||||
|
||||
### Tier 3: Security Analysis (Warning/Flag)
|
||||
|
||||
| Check | Description | Action on Fail |
|
||||
|-------|-------------|----------------|
|
||||
| Dangerous patterns | Known malicious Lua patterns | Flag for review |
|
||||
| Excessive permissions | Unusual permission combos | Flag for review |
|
||||
| Obfuscated code | Heavily obfuscated Lua | Flag for review |
|
||||
| External URLs | Hardcoded external URLs | Flag for review |
|
||||
| Large assets | Unusually large files | Warning |
|
||||
|
||||
### Tier 4: Quality Checks (Warning)
|
||||
|
||||
| Check | Description | Action on Fail |
|
||||
|-------|-------------|----------------|
|
||||
| Description length | Meaningful description | Warning |
|
||||
| Release notes | Non-empty release notes | Warning |
|
||||
| Icon quality | Not placeholder/blank | Warning |
|
||||
| Localization | Locale files complete | Warning |
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
### Validation Worker
|
||||
|
||||
```go
|
||||
type ValidationResult struct {
|
||||
Passed bool
|
||||
Errors []ValidationError
|
||||
Warnings []ValidationWarning
|
||||
Flags []ReviewFlag
|
||||
}
|
||||
|
||||
type ValidationError struct {
|
||||
Code string
|
||||
Message string
|
||||
File string
|
||||
Line int
|
||||
}
|
||||
|
||||
func ValidatePackage(packagePath string) ValidationResult {
|
||||
result := ValidationResult{Passed: true}
|
||||
|
||||
// Tier 1: Package validation
|
||||
if err := validateZip(packagePath); err != nil {
|
||||
result.AddError("INVALID_ZIP", err.Error())
|
||||
return result
|
||||
}
|
||||
|
||||
manifest, err := extractManifest(packagePath)
|
||||
if err != nil {
|
||||
result.AddError("INVALID_MANIFEST", err.Error())
|
||||
return result
|
||||
}
|
||||
|
||||
if err := validateSignature(packagePath, manifest); err != nil {
|
||||
result.AddError("INVALID_SIGNATURE", err.Error())
|
||||
return result
|
||||
}
|
||||
|
||||
// Tier 2: Content validation
|
||||
files, _ := listFiles(packagePath)
|
||||
for _, file := range files {
|
||||
switch filepath.Ext(file) {
|
||||
case ".rml":
|
||||
if err := validateRML(file); err != nil {
|
||||
result.AddError("INVALID_RML", err.Error(), file)
|
||||
}
|
||||
case ".rcss":
|
||||
if err := validateRCSS(file); err != nil {
|
||||
result.AddError("INVALID_RCSS", err.Error(), file)
|
||||
}
|
||||
case ".lua":
|
||||
if err := validateLua(file); err != nil {
|
||||
result.AddError("INVALID_LUA", err.Error(), file)
|
||||
}
|
||||
if flags := analyzeLuaSecurity(file); len(flags) > 0 {
|
||||
result.Flags = append(result.Flags, flags...)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Tier 3: Security analysis
|
||||
if hasDangerousPatterns(files) {
|
||||
result.AddFlag("DANGEROUS_PATTERNS", "Code contains suspicious patterns")
|
||||
}
|
||||
|
||||
// Tier 4: Quality checks
|
||||
if len(manifest.Description) < 10 {
|
||||
result.AddWarning("SHORT_DESCRIPTION", "Description is very short")
|
||||
}
|
||||
|
||||
result.Passed = len(result.Errors) == 0
|
||||
return result
|
||||
}
|
||||
```
|
||||
|
||||
### Dangerous Pattern Detection
|
||||
|
||||
```go
|
||||
var dangerousPatterns = []struct {
|
||||
Pattern *regexp.Regexp
|
||||
Reason string
|
||||
}{
|
||||
{
|
||||
regexp.MustCompile(`loadstring\s*\(`),
|
||||
"Dynamic code execution",
|
||||
},
|
||||
{
|
||||
regexp.MustCompile(`debug\s*\.\s*\w+`),
|
||||
"Debug library usage",
|
||||
},
|
||||
{
|
||||
regexp.MustCompile(`os\s*\.\s*execute`),
|
||||
"OS command execution",
|
||||
},
|
||||
{
|
||||
regexp.MustCompile(`io\s*\.\s*\w+`),
|
||||
"Direct I/O operations",
|
||||
},
|
||||
{
|
||||
regexp.MustCompile(`ffi\s*\.\s*\w+`),
|
||||
"FFI usage",
|
||||
},
|
||||
{
|
||||
regexp.MustCompile(`package\s*\.\s*loadlib`),
|
||||
"Native library loading",
|
||||
},
|
||||
}
|
||||
|
||||
func analyzeLuaSecurity(content string) []ReviewFlag {
|
||||
var flags []ReviewFlag
|
||||
for _, dp := range dangerousPatterns {
|
||||
if dp.Pattern.MatchString(content) {
|
||||
flags = append(flags, ReviewFlag{
|
||||
Type: "SECURITY",
|
||||
Reason: dp.Reason,
|
||||
})
|
||||
}
|
||||
}
|
||||
return flags
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Manual Review
|
||||
|
||||
### When Required
|
||||
|
||||
| Trigger | Reason |
|
||||
|---------|--------|
|
||||
| New developer | First app submission |
|
||||
| Dangerous permissions | camera, microphone, contacts, location |
|
||||
| Security flags | Automated checks flagged concerns |
|
||||
| User reports | Existing app reported |
|
||||
| Appeal | Developer contests rejection |
|
||||
|
||||
### Review Queue UI
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Review Queue [14 pending] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Filter: [All ▼] [Flagged first ▼] │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ 🚩 Weather Pro v2.0.0 │ │
|
||||
│ │ com.newdev.weather • New developer │ │
|
||||
│ │ Permissions: location, network │ │
|
||||
│ │ Flags: First submission │ │
|
||||
│ │ Submitted: 2 hours ago │ │
|
||||
│ │ [Review] [Auto-approve]│ │
|
||||
│ ├──────────────────────────────────────────────────────┤ │
|
||||
│ │ 🚩 Photo Editor v1.5.0 │ │
|
||||
│ │ com.trusted.photos • Verified developer │ │
|
||||
│ │ Permissions: camera, storage │ │
|
||||
│ │ Flags: DANGEROUS_PATTERNS (1) │ │
|
||||
│ │ Submitted: 5 hours ago │ │
|
||||
│ │ [Review] [Auto-approve]│ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Review Detail View
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Review: Weather Pro v2.0.0 │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Package: com.newdev.weather │
|
||||
│ Developer: newdev@example.com (New - first app) │
|
||||
│ Submitted: Jan 15, 2024 10:30 AM │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ Automated Checks │ │
|
||||
│ │ ✓ Package validation passed │ │
|
||||
│ │ ✓ Content validation passed │ │
|
||||
│ │ ⚠ Warning: SHORT_DESCRIPTION │ │
|
||||
│ │ 🚩 Flag: First submission from new developer │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Permissions Requested: │
|
||||
│ • location (fine) - Used for weather forecasts │
|
||||
│ • network - Required for API calls │
|
||||
│ │
|
||||
│ Files: [Expand to browse] │
|
||||
│ ├── manifest.json │
|
||||
│ ├── assets/ │
|
||||
│ │ ├── main.rml │
|
||||
│ │ └── scripts/ │
|
||||
│ │ └── weather.lua [View source] │
|
||||
│ └── icons/ │
|
||||
│ │
|
||||
│ Reviewer Notes: │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [Approve] [Reject with feedback] [Request changes] │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Review States
|
||||
|
||||
```
|
||||
┌─────────┐
|
||||
│ Draft │
|
||||
└────┬────┘
|
||||
│ submit
|
||||
▼
|
||||
┌─────────┐
|
||||
┌───────│Uploaded │
|
||||
│ └────┬────┘
|
||||
│ │ validation
|
||||
│ ▼
|
||||
│ ┌──────────┐
|
||||
failed │ │Validating│
|
||||
│ └────┬─────┘
|
||||
│ │
|
||||
│ ┌───────┴───────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────┐ ┌─────────┐
|
||||
│ Failed │ │In Review│
|
||||
└────────┘ └────┬────┘
|
||||
│
|
||||
┌───────────┼───────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌────────┐ ┌──────────┐
|
||||
│ Approved │ │Rejected│ │ Changes │
|
||||
└────┬─────┘ └────────┘ │ Requested│
|
||||
│ └──────────┘
|
||||
│ publish
|
||||
▼
|
||||
┌──────────┐
|
||||
│Published │
|
||||
└──────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rejection Feedback
|
||||
|
||||
### Feedback Template
|
||||
|
||||
```json
|
||||
{
|
||||
"reason": "SECURITY_CONCERN",
|
||||
"message": "Your app contains patterns that raise security concerns.",
|
||||
"details": [
|
||||
{
|
||||
"file": "scripts/main.lua",
|
||||
"line": 42,
|
||||
"issue": "Usage of loadstring() is not allowed",
|
||||
"suggestion": "Use static Lua code instead of dynamic evaluation"
|
||||
}
|
||||
],
|
||||
"can_resubmit": true,
|
||||
"appeal_available": true
|
||||
}
|
||||
```
|
||||
|
||||
### Common Rejection Reasons
|
||||
|
||||
| Code | Description |
|
||||
|------|-------------|
|
||||
| `SECURITY_CONCERN` | Security issues found |
|
||||
| `QUALITY_ISSUE` | Doesn't meet quality standards |
|
||||
| `POLICY_VIOLATION` | Violates content policy |
|
||||
| `METADATA_ISSUE` | Incorrect/misleading metadata |
|
||||
| `PERMISSION_ABUSE` | Unnecessary permissions |
|
||||
| `COPYRIGHT` | Copyright/trademark issues |
|
||||
|
||||
---
|
||||
|
||||
## Appeal Process
|
||||
|
||||
```
|
||||
1. Developer receives rejection
|
||||
2. Developer clicks "Appeal" (within 14 days)
|
||||
3. Provides justification
|
||||
4. Different reviewer examines
|
||||
5. Final decision (approve/uphold rejection)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Review SLA
|
||||
|
||||
| Submission Type | Target Time |
|
||||
|-----------------|-------------|
|
||||
| Auto-approved | Instant |
|
||||
| New developer | 24 hours |
|
||||
| Flagged | 48 hours |
|
||||
| Appeal | 72 hours |
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
```yaml
|
||||
# Developer endpoints
|
||||
POST /v1/apps/:id/versions/:vid/submit:
|
||||
summary: Submit for review
|
||||
response: { version, estimated_review_time }
|
||||
|
||||
GET /v1/apps/:id/versions/:vid/review-status:
|
||||
summary: Get review status
|
||||
response: { status, feedback?, estimated_completion }
|
||||
|
||||
POST /v1/apps/:id/versions/:vid/appeal:
|
||||
summary: Appeal rejection
|
||||
body: { justification }
|
||||
response: { appeal_id, status }
|
||||
|
||||
# Internal review endpoints (admin only)
|
||||
GET /v1/admin/review-queue:
|
||||
summary: Get pending reviews
|
||||
response: { items[], total }
|
||||
|
||||
GET /v1/admin/review/:version_id:
|
||||
summary: Get review details
|
||||
response: { version, validation_result, flags }
|
||||
|
||||
POST /v1/admin/review/:version_id/approve:
|
||||
summary: Approve version
|
||||
body: { notes? }
|
||||
response: { version }
|
||||
|
||||
POST /v1/admin/review/:version_id/reject:
|
||||
summary: Reject version
|
||||
body: { reason, message, details[] }
|
||||
response: { version }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Metrics
|
||||
|
||||
### Review Performance
|
||||
|
||||
| Metric | Target |
|
||||
|--------|--------|
|
||||
| Auto-approval rate | > 70% |
|
||||
| Average review time | < 24 hours |
|
||||
| Rejection rate | < 20% |
|
||||
| Appeal overturn rate | < 10% |
|
||||
|
||||
### Dashboard
|
||||
|
||||
```sql
|
||||
-- Review stats
|
||||
SELECT
|
||||
DATE_TRUNC('week', submitted_at) as week,
|
||||
COUNT(*) as submissions,
|
||||
COUNT(*) FILTER (WHERE status = 'published') as approved,
|
||||
COUNT(*) FILTER (WHERE status = 'rejected') as rejected,
|
||||
AVG(EXTRACT(EPOCH FROM (reviewed_at - submitted_at))/3600) as avg_hours
|
||||
FROM app_versions
|
||||
WHERE submitted_at > NOW() - INTERVAL '30 days'
|
||||
GROUP BY week;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] Validation worker implementation
|
||||
- [ ] Dangerous pattern database
|
||||
- [ ] Review queue UI
|
||||
- [ ] Reviewer tools
|
||||
- [ ] Rejection feedback system
|
||||
- [ ] Appeal workflow
|
||||
- [ ] Review metrics dashboard
|
||||
- [ ] SLA monitoring
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Automated approval for trusted developers?
|
||||
2. Community moderators?
|
||||
3. Content policy document?
|
||||
4. Rate limiting resubmissions?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Apple App Review Guidelines](https://developer.apple.com/app-store/review/guidelines/)
|
||||
- [Google Play Policy](https://play.google.com/about/developer-content-policy/)
|
||||
603
DEV_PORTAL_M10_DEVICE.md
Normal file
603
DEV_PORTAL_M10_DEVICE.md
Normal file
@@ -0,0 +1,603 @@
|
||||
# Milestone 10: Device-Side App Management
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Install, update, and manage apps on Mosis devices.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Device-side app management handles the full lifecycle of apps on user devices: discovery, installation, updates, and removal.
|
||||
|
||||
---
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ MosisService │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
|
||||
│ │ AppManager │ │ UpdateService │ │ AppStore UI │ │
|
||||
│ │ (C++ class) │ │ (Background) │ │ (System App) │ │
|
||||
│ └───────────────┘ └───────────────┘ └───────────────┘ │
|
||||
│ │ │ │ │
|
||||
│ └───────────────────┼───────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌────────┴────────┐ │
|
||||
│ │ LuaSandboxManager│ │
|
||||
│ └─────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Storage Layout
|
||||
|
||||
```
|
||||
/data/mosis/
|
||||
├── config/
|
||||
│ ├── device.json # Device ID, settings
|
||||
│ └── apps.json # Installed apps registry
|
||||
├── apps/
|
||||
│ └── {package_id}/
|
||||
│ ├── package/ # Extracted app files
|
||||
│ │ ├── manifest.json
|
||||
│ │ └── assets/
|
||||
│ ├── data/ # App data (VirtualFS)
|
||||
│ ├── cache/ # App cache
|
||||
│ └── db/ # SQLite databases
|
||||
├── downloads/ # Temp download location
|
||||
└── backups/ # App data backups (before update)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## AppManager Class
|
||||
|
||||
### Interface
|
||||
|
||||
```cpp
|
||||
namespace mosis {
|
||||
|
||||
struct InstalledApp {
|
||||
std::string package_id;
|
||||
std::string name;
|
||||
std::string version_name;
|
||||
int version_code;
|
||||
std::string install_path;
|
||||
std::vector<std::string> permissions;
|
||||
std::chrono::system_clock::time_point installed_at;
|
||||
std::chrono::system_clock::time_point updated_at;
|
||||
int64_t package_size;
|
||||
int64_t data_size;
|
||||
};
|
||||
|
||||
struct InstallProgress {
|
||||
enum class Stage {
|
||||
Downloading,
|
||||
Verifying,
|
||||
Extracting,
|
||||
Registering,
|
||||
Complete,
|
||||
Failed
|
||||
};
|
||||
Stage stage;
|
||||
float progress; // 0.0 - 1.0
|
||||
std::string error;
|
||||
};
|
||||
|
||||
using ProgressCallback = std::function<void(const InstallProgress&)>;
|
||||
|
||||
class AppManager {
|
||||
public:
|
||||
explicit AppManager(const std::string& data_root);
|
||||
~AppManager();
|
||||
|
||||
// Installation
|
||||
bool Install(const std::string& package_url,
|
||||
const std::string& signature,
|
||||
ProgressCallback callback);
|
||||
bool InstallFromFile(const std::string& package_path,
|
||||
ProgressCallback callback);
|
||||
|
||||
// Uninstallation
|
||||
bool Uninstall(const std::string& package_id, bool keep_data = false);
|
||||
|
||||
// Updates
|
||||
bool Update(const std::string& package_id,
|
||||
const std::string& package_url,
|
||||
const std::string& signature,
|
||||
ProgressCallback callback);
|
||||
|
||||
// Queries
|
||||
std::vector<InstalledApp> GetInstalledApps() const;
|
||||
std::optional<InstalledApp> GetApp(const std::string& package_id) const;
|
||||
bool IsInstalled(const std::string& package_id) const;
|
||||
|
||||
// Data management
|
||||
int64_t GetAppDataSize(const std::string& package_id) const;
|
||||
bool ClearAppData(const std::string& package_id);
|
||||
bool ClearAppCache(const std::string& package_id);
|
||||
bool BackupAppData(const std::string& package_id);
|
||||
bool RestoreAppData(const std::string& package_id);
|
||||
|
||||
// Integration with sandbox
|
||||
void SetSandboxManager(LuaSandboxManager* manager);
|
||||
|
||||
private:
|
||||
std::string m_data_root;
|
||||
LuaSandboxManager* m_sandbox_manager = nullptr;
|
||||
mutable std::mutex m_mutex;
|
||||
std::map<std::string, InstalledApp> m_installed_apps;
|
||||
|
||||
bool VerifyPackage(const std::string& path, const std::string& signature);
|
||||
bool ExtractPackage(const std::string& path, const std::string& dest);
|
||||
void LoadInstalledApps();
|
||||
void SaveInstalledApps();
|
||||
};
|
||||
|
||||
} // namespace mosis
|
||||
```
|
||||
|
||||
### Implementation
|
||||
|
||||
```cpp
|
||||
bool AppManager::Install(const std::string& package_url,
|
||||
const std::string& signature,
|
||||
ProgressCallback callback) {
|
||||
callback({InstallProgress::Stage::Downloading, 0.0f, ""});
|
||||
|
||||
// Download package
|
||||
std::string download_path = m_data_root + "/downloads/" + GenerateUUID();
|
||||
if (!DownloadFile(package_url, download_path, [&](float p) {
|
||||
callback({InstallProgress::Stage::Downloading, p, ""});
|
||||
})) {
|
||||
callback({InstallProgress::Stage::Failed, 0.0f, "Download failed"});
|
||||
return false;
|
||||
}
|
||||
|
||||
callback({InstallProgress::Stage::Verifying, 0.0f, ""});
|
||||
|
||||
// Verify signature
|
||||
if (!VerifyPackage(download_path, signature)) {
|
||||
std::filesystem::remove(download_path);
|
||||
callback({InstallProgress::Stage::Failed, 0.0f, "Signature verification failed"});
|
||||
return false;
|
||||
}
|
||||
|
||||
// Extract manifest to get package_id
|
||||
auto manifest = ExtractManifest(download_path);
|
||||
if (!manifest) {
|
||||
std::filesystem::remove(download_path);
|
||||
callback({InstallProgress::Stage::Failed, 0.0f, "Invalid manifest"});
|
||||
return false;
|
||||
}
|
||||
|
||||
callback({InstallProgress::Stage::Extracting, 0.0f, ""});
|
||||
|
||||
// Check if already installed (update path)
|
||||
std::string install_path = m_data_root + "/apps/" + manifest->package_id;
|
||||
if (std::filesystem::exists(install_path + "/package")) {
|
||||
// Backup existing data
|
||||
BackupAppData(manifest->package_id);
|
||||
// Remove old package
|
||||
std::filesystem::remove_all(install_path + "/package");
|
||||
}
|
||||
|
||||
// Extract package
|
||||
std::filesystem::create_directories(install_path + "/package");
|
||||
if (!ExtractPackage(download_path, install_path + "/package")) {
|
||||
callback({InstallProgress::Stage::Failed, 0.0f, "Extraction failed"});
|
||||
return false;
|
||||
}
|
||||
|
||||
// Clean up download
|
||||
std::filesystem::remove(download_path);
|
||||
|
||||
callback({InstallProgress::Stage::Registering, 0.0f, ""});
|
||||
|
||||
// Create data directories
|
||||
std::filesystem::create_directories(install_path + "/data");
|
||||
std::filesystem::create_directories(install_path + "/cache");
|
||||
std::filesystem::create_directories(install_path + "/db");
|
||||
|
||||
// Register app
|
||||
InstalledApp app{
|
||||
.package_id = manifest->package_id,
|
||||
.name = manifest->name,
|
||||
.version_name = manifest->version,
|
||||
.version_code = manifest->version_code,
|
||||
.install_path = install_path,
|
||||
.permissions = manifest->permissions,
|
||||
.installed_at = std::chrono::system_clock::now(),
|
||||
.updated_at = std::chrono::system_clock::now(),
|
||||
.package_size = std::filesystem::file_size(download_path)
|
||||
};
|
||||
|
||||
{
|
||||
std::lock_guard<std::mutex> lock(m_mutex);
|
||||
m_installed_apps[manifest->package_id] = app;
|
||||
SaveInstalledApps();
|
||||
}
|
||||
|
||||
callback({InstallProgress::Stage::Complete, 1.0f, ""});
|
||||
return true;
|
||||
}
|
||||
|
||||
bool AppManager::Uninstall(const std::string& package_id, bool keep_data) {
|
||||
std::lock_guard<std::mutex> lock(m_mutex);
|
||||
|
||||
auto it = m_installed_apps.find(package_id);
|
||||
if (it == m_installed_apps.end()) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Stop app if running
|
||||
if (m_sandbox_manager && m_sandbox_manager->IsAppRunning(package_id)) {
|
||||
m_sandbox_manager->StopApp(package_id);
|
||||
}
|
||||
|
||||
// Remove files
|
||||
std::string install_path = it->second.install_path;
|
||||
std::filesystem::remove_all(install_path + "/package");
|
||||
std::filesystem::remove_all(install_path + "/cache");
|
||||
|
||||
if (!keep_data) {
|
||||
std::filesystem::remove_all(install_path + "/data");
|
||||
std::filesystem::remove_all(install_path + "/db");
|
||||
std::filesystem::remove_all(install_path);
|
||||
}
|
||||
|
||||
// Unregister
|
||||
m_installed_apps.erase(it);
|
||||
SaveInstalledApps();
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Update Service
|
||||
|
||||
### Background Update Checker
|
||||
|
||||
```cpp
|
||||
class UpdateService {
|
||||
public:
|
||||
UpdateService(AppManager* app_manager, const std::string& api_base);
|
||||
|
||||
// Start background checking
|
||||
void Start(std::chrono::hours interval = std::chrono::hours(24));
|
||||
void Stop();
|
||||
|
||||
// Manual check
|
||||
std::vector<UpdateInfo> CheckForUpdates();
|
||||
|
||||
// Download and install update
|
||||
bool ApplyUpdate(const std::string& package_id, ProgressCallback callback);
|
||||
|
||||
// Settings
|
||||
void SetAutoUpdate(bool enabled);
|
||||
void SetWifiOnly(bool wifi_only);
|
||||
|
||||
private:
|
||||
void CheckLoop();
|
||||
|
||||
AppManager* m_app_manager;
|
||||
std::string m_api_base;
|
||||
std::thread m_check_thread;
|
||||
std::atomic<bool> m_running{false};
|
||||
bool m_auto_update = false;
|
||||
bool m_wifi_only = true;
|
||||
};
|
||||
```
|
||||
|
||||
### Update Check Flow
|
||||
|
||||
```
|
||||
1. Get list of installed apps
|
||||
2. Call API: GET /store/apps/updates?packages=com.a,com.b,com.c
|
||||
3. API returns list of available updates
|
||||
4. If auto-update enabled and on WiFi:
|
||||
- Download and install in background
|
||||
- Notify user of completed updates
|
||||
5. If manual:
|
||||
- Show notification with update count
|
||||
- User opens App Store to review
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## App Store System App
|
||||
|
||||
### UI Screens
|
||||
|
||||
```
|
||||
Home
|
||||
├── Featured apps
|
||||
├── Categories
|
||||
├── Search
|
||||
└── My Apps
|
||||
├── Installed
|
||||
├── Updates available
|
||||
└── Previously installed
|
||||
|
||||
App Detail
|
||||
├── Icon, name, developer
|
||||
├── Screenshots
|
||||
├── Description
|
||||
├── Permissions
|
||||
├── Reviews (future)
|
||||
└── [Install] / [Update] / [Open]
|
||||
|
||||
Settings
|
||||
├── Auto-update (WiFi only)
|
||||
├── Storage usage
|
||||
└── Clear all caches
|
||||
```
|
||||
|
||||
### RML Structure
|
||||
|
||||
```xml
|
||||
<!-- app_store/main.rml -->
|
||||
<rml>
|
||||
<head>
|
||||
<link type="text/rcss" href="../../ui/theme.rcss"/>
|
||||
<link type="text/rcss" href="store.rcss"/>
|
||||
<script src="store.lua"/>
|
||||
</head>
|
||||
<body>
|
||||
<div class="app-bar">
|
||||
<h1>App Store</h1>
|
||||
<div class="search-icon" onclick="showSearch()"/>
|
||||
</div>
|
||||
|
||||
<div id="content">
|
||||
<!-- Dynamic content loaded by Lua -->
|
||||
</div>
|
||||
|
||||
<div class="bottom-nav">
|
||||
<div class="nav-item active" onclick="showHome()">
|
||||
<img src="icons/home.tga"/>
|
||||
<span>Home</span>
|
||||
</div>
|
||||
<div class="nav-item" onclick="showCategories()">
|
||||
<img src="icons/category.tga"/>
|
||||
<span>Categories</span>
|
||||
</div>
|
||||
<div class="nav-item" onclick="showMyApps()">
|
||||
<img src="icons/apps.tga"/>
|
||||
<span>My Apps</span>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</rml>
|
||||
```
|
||||
|
||||
### Lua Store Logic
|
||||
|
||||
```lua
|
||||
-- store.lua
|
||||
local api = require("store_api")
|
||||
local ui = require("ui")
|
||||
|
||||
local state = {
|
||||
screen = "home",
|
||||
featured = {},
|
||||
categories = {},
|
||||
installed = {},
|
||||
updates = {}
|
||||
}
|
||||
|
||||
function init()
|
||||
-- Load installed apps
|
||||
state.installed = mosis.apps.getInstalled()
|
||||
|
||||
-- Fetch featured apps
|
||||
api.getFeatured(function(apps)
|
||||
state.featured = apps
|
||||
render()
|
||||
end)
|
||||
|
||||
-- Check for updates
|
||||
checkUpdates()
|
||||
end
|
||||
|
||||
function checkUpdates()
|
||||
local package_ids = {}
|
||||
for _, app in ipairs(state.installed) do
|
||||
table.insert(package_ids, app.package_id)
|
||||
end
|
||||
|
||||
api.checkUpdates(package_ids, function(updates)
|
||||
state.updates = updates
|
||||
render()
|
||||
end)
|
||||
end
|
||||
|
||||
function installApp(package_id)
|
||||
local app = findApp(package_id)
|
||||
if not app then return end
|
||||
|
||||
showProgress(app.name)
|
||||
|
||||
mosis.apps.install(app.download_url, app.signature, function(progress)
|
||||
updateProgress(progress.stage, progress.progress)
|
||||
|
||||
if progress.stage == "complete" then
|
||||
hideProgress()
|
||||
showToast(app.name .. " installed")
|
||||
state.installed = mosis.apps.getInstalled()
|
||||
render()
|
||||
elseif progress.stage == "failed" then
|
||||
hideProgress()
|
||||
showError("Installation failed: " .. progress.error)
|
||||
end
|
||||
end)
|
||||
end
|
||||
|
||||
function openApp(package_id)
|
||||
mosis.apps.launch(package_id)
|
||||
end
|
||||
|
||||
function uninstallApp(package_id)
|
||||
showConfirm("Uninstall " .. getAppName(package_id) .. "?", function(confirmed)
|
||||
if confirmed then
|
||||
mosis.apps.uninstall(package_id)
|
||||
state.installed = mosis.apps.getInstalled()
|
||||
render()
|
||||
end
|
||||
end)
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Lua API for Apps
|
||||
|
||||
### Exposed to System Apps
|
||||
|
||||
```lua
|
||||
-- mosis.apps namespace (system apps only)
|
||||
|
||||
-- Get installed apps
|
||||
local apps = mosis.apps.getInstalled()
|
||||
-- Returns: [{package_id, name, version_name, version_code, installed_at}]
|
||||
|
||||
-- Install from store
|
||||
mosis.apps.install(url, signature, callback)
|
||||
-- callback(progress): {stage, progress, error}
|
||||
|
||||
-- Uninstall
|
||||
mosis.apps.uninstall(package_id)
|
||||
|
||||
-- Launch app
|
||||
mosis.apps.launch(package_id)
|
||||
|
||||
-- Get app info
|
||||
local info = mosis.apps.getInfo(package_id)
|
||||
|
||||
-- Storage management
|
||||
local size = mosis.apps.getDataSize(package_id)
|
||||
mosis.apps.clearCache(package_id)
|
||||
mosis.apps.clearData(package_id)
|
||||
```
|
||||
|
||||
### Exposed to All Apps
|
||||
|
||||
```lua
|
||||
-- mosis.app namespace (current app only)
|
||||
|
||||
-- Get own package info
|
||||
local info = mosis.app.info()
|
||||
-- Returns: {package_id, name, version_name, version_code}
|
||||
|
||||
-- Check for update
|
||||
mosis.app.checkUpdate(function(available, new_version)
|
||||
if available then
|
||||
showUpdatePrompt(new_version)
|
||||
end
|
||||
end)
|
||||
|
||||
-- Open store page for self
|
||||
mosis.app.openStorePage()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Permissions for App Management
|
||||
|
||||
```lua
|
||||
-- Required permission to use mosis.apps.*
|
||||
permissions = {"system.app_management"}
|
||||
|
||||
-- Only granted to:
|
||||
-- - App Store system app
|
||||
-- - Settings system app
|
||||
-- - Other OEM system apps
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Installation Intents
|
||||
|
||||
### From Deep Links
|
||||
|
||||
```
|
||||
mosis://store/app/com.developer.myapp
|
||||
mosis://store/install?url=...&sig=...
|
||||
```
|
||||
|
||||
### From App Store
|
||||
|
||||
```lua
|
||||
-- User taps Install button
|
||||
installApp("com.developer.myapp")
|
||||
```
|
||||
|
||||
### From ADB (Development)
|
||||
|
||||
```bash
|
||||
adb shell am broadcast -a com.omixlab.mosis.INSTALL_APP \
|
||||
--es package_path "/sdcard/myapp.mosis"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
### Package Verification
|
||||
|
||||
1. Verify ZIP integrity
|
||||
2. Verify Ed25519 signature
|
||||
3. Verify signer is registered developer
|
||||
4. Verify app not in blocklist
|
||||
5. Verify permissions are declared
|
||||
|
||||
### Installation Sources
|
||||
|
||||
| Source | Allowed |
|
||||
|--------|---------|
|
||||
| Official store | Always |
|
||||
| Developer sideload | If enabled in settings |
|
||||
| Unknown APK | Never (MosisService only) |
|
||||
|
||||
### Sandboxing
|
||||
|
||||
- All apps run in LuaSandbox
|
||||
- File access limited to app's data directory
|
||||
- Network access requires permission
|
||||
- Hardware access requires permission + user gesture
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] AppManager C++ class
|
||||
- [ ] UpdateService background checker
|
||||
- [ ] App Store system app
|
||||
- [ ] Lua API bindings (mosis.apps, mosis.app)
|
||||
- [ ] Installation progress UI
|
||||
- [ ] Uninstall confirmation UI
|
||||
- [ ] Storage management UI
|
||||
- [ ] Deep link handling
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. App backup to cloud?
|
||||
2. Family sharing / multiple devices?
|
||||
3. Enterprise MDM integration?
|
||||
4. Sideloading policy?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Android PackageManager](https://developer.android.com/reference/android/content/pm/PackageManager)
|
||||
- [iOS App Installation](https://developer.apple.com/documentation/devicemanagement/installing_apps)
|
||||
609
DEV_PORTAL_M11_CLI.md
Normal file
609
DEV_PORTAL_M11_CLI.md
Normal file
@@ -0,0 +1,609 @@
|
||||
# Milestone 11: Developer CLI Tool
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Command-line tool for app development workflow.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The CLI tool (`mosis`) streamlines the developer workflow: project creation, building, testing, signing, and publishing.
|
||||
|
||||
---
|
||||
|
||||
## Commands Overview
|
||||
|
||||
```
|
||||
mosis
|
||||
├── init Create new app project
|
||||
├── validate Validate manifest and assets
|
||||
├── build Create .mosis package
|
||||
├── sign Sign package with developer key
|
||||
├── run Run in local designer/emulator
|
||||
├── test Run automated tests
|
||||
├── login Authenticate with portal
|
||||
├── logout Clear authentication
|
||||
├── publish Upload and submit for review
|
||||
├── status Check review status
|
||||
├── keys
|
||||
│ ├── generate Generate signing keypair
|
||||
│ ├── list List registered keys
|
||||
│ └── register Upload public key to portal
|
||||
└── config
|
||||
├── get Get config value
|
||||
└── set Set config value
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Command Details
|
||||
|
||||
### `mosis init`
|
||||
|
||||
Create a new app project with boilerplate.
|
||||
|
||||
```bash
|
||||
$ mosis init
|
||||
|
||||
? App name: My Calculator
|
||||
? Package ID: com.myname.calculator
|
||||
? Description: A simple calculator app
|
||||
? Author name: John Doe
|
||||
? Author email: john@example.com
|
||||
|
||||
Creating project structure...
|
||||
|
||||
✓ Created manifest.json
|
||||
✓ Created assets/main.rml
|
||||
✓ Created assets/styles/theme.rcss
|
||||
✓ Created assets/scripts/app.lua
|
||||
✓ Created icons/ (placeholder icons)
|
||||
|
||||
Project created! Next steps:
|
||||
cd my-calculator
|
||||
mosis run # Preview in designer
|
||||
mosis build # Create package
|
||||
mosis publish # Submit to store
|
||||
```
|
||||
|
||||
#### Generated Structure
|
||||
|
||||
```
|
||||
my-calculator/
|
||||
├── manifest.json
|
||||
├── assets/
|
||||
│ ├── main.rml
|
||||
│ ├── styles/
|
||||
│ │ └── theme.rcss
|
||||
│ └── scripts/
|
||||
│ └── app.lua
|
||||
├── icons/
|
||||
│ ├── icon-32.png
|
||||
│ ├── icon-64.png
|
||||
│ └── icon-128.png
|
||||
└── .mosisignore # Files to exclude from package
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `mosis validate`
|
||||
|
||||
Validate project without building.
|
||||
|
||||
```bash
|
||||
$ mosis validate
|
||||
|
||||
Validating manifest.json...
|
||||
✓ Required fields present
|
||||
✓ Package ID format valid
|
||||
✓ Version format valid
|
||||
|
||||
Validating assets...
|
||||
✓ Entry point exists: assets/main.rml
|
||||
✓ All RML files valid (3 files)
|
||||
✓ All RCSS files valid (2 files)
|
||||
✓ All Lua files valid (4 files)
|
||||
|
||||
Validating icons...
|
||||
✓ icon-32.png (32x32)
|
||||
✓ icon-64.png (64x64)
|
||||
✓ icon-128.png (128x128)
|
||||
|
||||
Checking permissions...
|
||||
✓ Permissions declared: storage, network
|
||||
|
||||
All validations passed!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `mosis build`
|
||||
|
||||
Create a .mosis package.
|
||||
|
||||
```bash
|
||||
$ mosis build
|
||||
|
||||
Reading manifest.json...
|
||||
Package: com.myname.calculator v1.0.0 (1)
|
||||
|
||||
Collecting files...
|
||||
✓ manifest.json
|
||||
✓ assets/main.rml
|
||||
✓ assets/styles/theme.rcss
|
||||
✓ assets/scripts/app.lua
|
||||
✓ icons/icon-32.png
|
||||
✓ icons/icon-64.png
|
||||
✓ icons/icon-128.png
|
||||
|
||||
Creating package...
|
||||
✓ Package created: dist/com.myname.calculator-1.0.0.mosis (45.2 KB)
|
||||
|
||||
⚠ Package is unsigned. Run 'mosis sign' before publishing.
|
||||
```
|
||||
|
||||
#### Options
|
||||
|
||||
```bash
|
||||
mosis build [options]
|
||||
|
||||
Options:
|
||||
-o, --output <path> Output path (default: dist/)
|
||||
--no-compress Skip compression
|
||||
--include-source Include .lua source maps
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `mosis sign`
|
||||
|
||||
Sign a package with developer key.
|
||||
|
||||
```bash
|
||||
$ mosis sign dist/com.myname.calculator-1.0.0.mosis
|
||||
|
||||
Using key: ~/.mosis/signing_key.pem
|
||||
Fingerprint: SHA256:abc123...
|
||||
|
||||
Generating file hashes...
|
||||
Signing MANIFEST.MF...
|
||||
|
||||
✓ Package signed: dist/com.myname.calculator-1.0.0.mosis
|
||||
|
||||
Signature details:
|
||||
Algorithm: Ed25519
|
||||
Key fingerprint: SHA256:abc123...
|
||||
Files signed: 7
|
||||
```
|
||||
|
||||
#### Options
|
||||
|
||||
```bash
|
||||
mosis sign <package> [options]
|
||||
|
||||
Options:
|
||||
-k, --key <path> Path to private key (default: ~/.mosis/signing_key.pem)
|
||||
--verify Verify existing signature
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `mosis run`
|
||||
|
||||
Launch in desktop designer for testing.
|
||||
|
||||
```bash
|
||||
$ mosis run
|
||||
|
||||
Starting Mosis Designer...
|
||||
Loading: assets/main.rml
|
||||
|
||||
Designer running at http://localhost:8080
|
||||
Press Ctrl+C to stop
|
||||
|
||||
[Hot reload enabled - changes auto-refresh]
|
||||
```
|
||||
|
||||
#### Options
|
||||
|
||||
```bash
|
||||
mosis run [options]
|
||||
|
||||
Options:
|
||||
--entry <file> Override entry point
|
||||
--port <number> Designer port (default: 8080)
|
||||
--no-hot-reload Disable hot reload
|
||||
--device <name> Emulate device (phone, tablet)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `mosis login`
|
||||
|
||||
Authenticate with developer portal.
|
||||
|
||||
```bash
|
||||
$ mosis login
|
||||
|
||||
Opening browser for authentication...
|
||||
Waiting for authorization...
|
||||
|
||||
✓ Logged in as john@example.com
|
||||
API key stored in ~/.mosis/credentials
|
||||
```
|
||||
|
||||
#### Options
|
||||
|
||||
```bash
|
||||
mosis login [options]
|
||||
|
||||
Options:
|
||||
--api-key <key> Use API key instead of browser auth
|
||||
--portal <url> Portal URL (default: https://portal.mosis.dev)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `mosis publish`
|
||||
|
||||
Upload and submit for review.
|
||||
|
||||
```bash
|
||||
$ mosis publish
|
||||
|
||||
Checking authentication...
|
||||
✓ Logged in as john@example.com
|
||||
|
||||
Building package...
|
||||
✓ Package created: dist/com.myname.calculator-1.0.0.mosis
|
||||
|
||||
Signing package...
|
||||
✓ Package signed
|
||||
|
||||
Uploading...
|
||||
████████████████████████████████ 100%
|
||||
|
||||
Submitting for review...
|
||||
✓ Version 1.0.0 submitted
|
||||
|
||||
Review status: In Review
|
||||
Estimated review time: 24 hours
|
||||
|
||||
Track status: mosis status
|
||||
```
|
||||
|
||||
#### Options
|
||||
|
||||
```bash
|
||||
mosis publish [options]
|
||||
|
||||
Options:
|
||||
--package <path> Use existing package
|
||||
--notes <text> Release notes
|
||||
--notes-file <path> Release notes from file
|
||||
--draft Upload without submitting for review
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `mosis status`
|
||||
|
||||
Check app/version status.
|
||||
|
||||
```bash
|
||||
$ mosis status
|
||||
|
||||
App: My Calculator (com.myname.calculator)
|
||||
|
||||
Versions:
|
||||
v1.0.0 (1) Published Jan 10, 2024 1,234 downloads
|
||||
v1.1.0 (2) In Review Jan 15, 2024 Submitted 2h ago
|
||||
|
||||
Latest review:
|
||||
Status: In Review
|
||||
Submitted: Jan 15, 2024 10:30 AM
|
||||
Estimated completion: Jan 16, 2024
|
||||
|
||||
Run 'mosis status --watch' to monitor in real-time.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `mosis keys generate`
|
||||
|
||||
Generate Ed25519 signing keypair.
|
||||
|
||||
```bash
|
||||
$ mosis keys generate
|
||||
|
||||
Generating Ed25519 keypair...
|
||||
|
||||
Private key saved to: ~/.mosis/signing_key.pem
|
||||
Public key saved to: ~/.mosis/signing_key.pub
|
||||
|
||||
Fingerprint: SHA256:abc123def456...
|
||||
|
||||
⚠ IMPORTANT: Keep your private key secure!
|
||||
- Never share or commit signing_key.pem
|
||||
- Back it up securely
|
||||
- If compromised, revoke immediately
|
||||
|
||||
Next step: Register your public key
|
||||
mosis keys register
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### `mosis keys register`
|
||||
|
||||
Upload public key to portal.
|
||||
|
||||
```bash
|
||||
$ mosis keys register
|
||||
|
||||
Reading public key from ~/.mosis/signing_key.pub
|
||||
Fingerprint: SHA256:abc123def456...
|
||||
|
||||
? Key name: MacBook Pro 2024
|
||||
|
||||
Uploading to portal...
|
||||
✓ Key registered successfully
|
||||
|
||||
Your signing key is now active. Packages signed with this
|
||||
key will be accepted for review.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Config File Location
|
||||
|
||||
```
|
||||
~/.mosis/
|
||||
├── config.json # CLI configuration
|
||||
├── credentials # Auth tokens (encrypted)
|
||||
├── signing_key.pem # Private key
|
||||
└── signing_key.pub # Public key
|
||||
```
|
||||
|
||||
### Config Options
|
||||
|
||||
```json
|
||||
{
|
||||
"portal_url": "https://portal.mosis.dev",
|
||||
"api_url": "https://api.mosis.dev",
|
||||
"designer_path": "/usr/local/bin/mosis-designer",
|
||||
"default_author": {
|
||||
"name": "John Doe",
|
||||
"email": "john@example.com"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
### Tech Stack Options
|
||||
|
||||
#### Option A: Go
|
||||
|
||||
```go
|
||||
// Using cobra for CLI framework
|
||||
package main
|
||||
|
||||
import (
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
func main() {
|
||||
rootCmd := &cobra.Command{
|
||||
Use: "mosis",
|
||||
Short: "Mosis app development CLI",
|
||||
}
|
||||
|
||||
rootCmd.AddCommand(initCmd())
|
||||
rootCmd.AddCommand(buildCmd())
|
||||
rootCmd.AddCommand(signCmd())
|
||||
rootCmd.AddCommand(publishCmd())
|
||||
// ...
|
||||
|
||||
rootCmd.Execute()
|
||||
}
|
||||
```
|
||||
|
||||
**Pros**: Single binary, fast, cross-platform
|
||||
**Cons**: More code to write
|
||||
|
||||
#### Option B: Node.js (oclif)
|
||||
|
||||
```typescript
|
||||
// Using oclif framework
|
||||
import { Command } from '@oclif/core'
|
||||
|
||||
export default class Build extends Command {
|
||||
static description = 'Build .mosis package'
|
||||
|
||||
async run() {
|
||||
const manifest = await this.readManifest()
|
||||
const files = await this.collectFiles()
|
||||
const package = await this.createPackage(files)
|
||||
this.log(`✓ Package created: ${package.path}`)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Pros**: Fast development, npm distribution
|
||||
**Cons**: Requires Node.js runtime
|
||||
|
||||
#### Option C: Rust (clap)
|
||||
|
||||
```rust
|
||||
use clap::{Parser, Subcommand};
|
||||
|
||||
#[derive(Parser)]
|
||||
#[command(name = "mosis")]
|
||||
struct Cli {
|
||||
#[command(subcommand)]
|
||||
command: Commands,
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum Commands {
|
||||
Init { name: Option<String> },
|
||||
Build { output: Option<PathBuf> },
|
||||
Sign { package: PathBuf },
|
||||
Publish,
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let cli = Cli::parse();
|
||||
match cli.command {
|
||||
Commands::Init { name } => init::run(name),
|
||||
Commands::Build { output } => build::run(output),
|
||||
// ...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Pros**: Single binary, very fast
|
||||
**Cons**: Slower development
|
||||
|
||||
---
|
||||
|
||||
## Distribution
|
||||
|
||||
### npm (Node.js version)
|
||||
|
||||
```bash
|
||||
npm install -g @mosis/cli
|
||||
```
|
||||
|
||||
### Homebrew (macOS)
|
||||
|
||||
```bash
|
||||
brew tap mosis/tap
|
||||
brew install mosis
|
||||
```
|
||||
|
||||
### Direct Download
|
||||
|
||||
```bash
|
||||
# Linux/macOS
|
||||
curl -fsSL https://mosis.dev/install.sh | sh
|
||||
|
||||
# Windows
|
||||
irm https://mosis.dev/install.ps1 | iex
|
||||
```
|
||||
|
||||
### Package Managers
|
||||
|
||||
| Platform | Package Manager | Command |
|
||||
|----------|-----------------|---------|
|
||||
| macOS | Homebrew | `brew install mosis` |
|
||||
| Windows | Scoop | `scoop install mosis` |
|
||||
| Linux | apt (deb) | `apt install mosis` |
|
||||
| Any | npm | `npm install -g @mosis/cli` |
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### User-Friendly Errors
|
||||
|
||||
```bash
|
||||
$ mosis build
|
||||
|
||||
Error: manifest.json not found
|
||||
|
||||
Are you in a Mosis project directory?
|
||||
Run 'mosis init' to create a new project.
|
||||
```
|
||||
|
||||
### Verbose Mode
|
||||
|
||||
```bash
|
||||
$ mosis build --verbose
|
||||
|
||||
[DEBUG] Reading manifest from ./manifest.json
|
||||
[DEBUG] Manifest parsed: {id: "com.example.app", ...}
|
||||
[DEBUG] Collecting files from ./assets
|
||||
[DEBUG] Found 7 files
|
||||
[DEBUG] Creating ZIP archive
|
||||
[DEBUG] Writing to dist/com.example.app-1.0.0.mosis
|
||||
[DEBUG] Package size: 45234 bytes
|
||||
✓ Package created
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
```yaml
|
||||
name: Build and Publish
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*'
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Mosis CLI
|
||||
run: npm install -g @mosis/cli
|
||||
|
||||
- name: Build and Sign
|
||||
env:
|
||||
MOSIS_SIGNING_KEY: ${{ secrets.MOSIS_SIGNING_KEY }}
|
||||
run: |
|
||||
echo "$MOSIS_SIGNING_KEY" > signing_key.pem
|
||||
mosis build
|
||||
mosis sign dist/*.mosis --key signing_key.pem
|
||||
|
||||
- name: Publish
|
||||
env:
|
||||
MOSIS_API_KEY: ${{ secrets.MOSIS_API_KEY }}
|
||||
run: mosis publish --api-key "$MOSIS_API_KEY"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] CLI framework selection
|
||||
- [ ] `init` command
|
||||
- [ ] `validate` command
|
||||
- [ ] `build` command
|
||||
- [ ] `sign` command
|
||||
- [ ] `run` command (designer integration)
|
||||
- [ ] `login/logout` commands
|
||||
- [ ] `publish` command
|
||||
- [ ] `status` command
|
||||
- [ ] `keys` subcommands
|
||||
- [ ] Configuration management
|
||||
- [ ] Distribution packages
|
||||
- [ ] CI/CD examples
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Should CLI auto-update itself?
|
||||
2. Offline mode for build/sign?
|
||||
3. Plugin system for custom commands?
|
||||
4. IDE integrations (VS Code extension)?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Cobra CLI Framework](https://cobra.dev/)
|
||||
- [oclif Framework](https://oclif.io/)
|
||||
- [Clap for Rust](https://docs.rs/clap/)
|
||||
617
DEV_PORTAL_M12_DOCS.md
Normal file
617
DEV_PORTAL_M12_DOCS.md
Normal file
@@ -0,0 +1,617 @@
|
||||
# Milestone 12: Documentation Site
|
||||
|
||||
**Status**: Planning
|
||||
**Goal**: Comprehensive documentation for Mosis app developers.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The documentation site is the primary resource for developers learning to build Mosis apps. Must be clear, searchable, and up-to-date.
|
||||
|
||||
---
|
||||
|
||||
## Information Architecture
|
||||
|
||||
```
|
||||
docs.mosis.dev/
|
||||
├── Getting Started
|
||||
│ ├── Introduction
|
||||
│ ├── Quick Start (5 min)
|
||||
│ ├── Your First App
|
||||
│ └── Project Structure
|
||||
├── Guides
|
||||
│ ├── UI Design
|
||||
│ │ ├── RML Basics
|
||||
│ │ ├── Styling with RCSS
|
||||
│ │ ├── Layouts
|
||||
│ │ └── Components
|
||||
│ ├── Lua Scripting
|
||||
│ │ ├── Basics
|
||||
│ │ ├── Event Handling
|
||||
│ │ ├── State Management
|
||||
│ │ └── Async Operations
|
||||
│ ├── Data & Storage
|
||||
│ │ ├── Local Storage
|
||||
│ │ ├── SQLite Database
|
||||
│ │ └── Files
|
||||
│ ├── Networking
|
||||
│ │ ├── HTTP Requests
|
||||
│ │ └── WebSockets
|
||||
│ ├── Hardware
|
||||
│ │ ├── Camera
|
||||
│ │ ├── Microphone
|
||||
│ │ ├── Location
|
||||
│ │ └── Sensors
|
||||
│ ├── Permissions
|
||||
│ │ ├── Permission Model
|
||||
│ │ ├── Requesting Permissions
|
||||
│ │ └── User Gestures
|
||||
│ └── Publishing
|
||||
│ ├── Preparing for Release
|
||||
│ ├── App Signing
|
||||
│ └── Store Guidelines
|
||||
├── API Reference
|
||||
│ ├── Lua APIs
|
||||
│ │ ├── mosis.storage
|
||||
│ │ ├── mosis.db
|
||||
│ │ ├── mosis.http
|
||||
│ │ ├── mosis.ws
|
||||
│ │ ├── mosis.camera
|
||||
│ │ ├── mosis.microphone
|
||||
│ │ ├── mosis.location
|
||||
│ │ ├── mosis.sensors
|
||||
│ │ └── ...
|
||||
│ ├── RML Elements
|
||||
│ ├── RCSS Properties
|
||||
│ └── Manifest Schema
|
||||
├── CLI Reference
|
||||
│ ├── mosis init
|
||||
│ ├── mosis build
|
||||
│ ├── mosis sign
|
||||
│ ├── mosis publish
|
||||
│ └── ...
|
||||
├── Best Practices
|
||||
│ ├── Performance
|
||||
│ ├── Security
|
||||
│ ├── UX Guidelines
|
||||
│ └── Accessibility
|
||||
├── Troubleshooting
|
||||
│ ├── Common Errors
|
||||
│ ├── Debugging Tips
|
||||
│ └── FAQ
|
||||
└── Changelog
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Content Types
|
||||
|
||||
### Tutorials (Step-by-step)
|
||||
|
||||
```markdown
|
||||
# Build a Weather App
|
||||
|
||||
In this tutorial, you'll build a weather app that:
|
||||
- Fetches weather data from an API
|
||||
- Displays current conditions
|
||||
- Shows a 5-day forecast
|
||||
- Requests location permission
|
||||
|
||||
**Time:** 30 minutes
|
||||
**Prerequisites:** Completed Quick Start
|
||||
|
||||
## Step 1: Create the Project
|
||||
|
||||
```bash
|
||||
mosis init weather-app
|
||||
cd weather-app
|
||||
```
|
||||
|
||||
## Step 2: Design the UI
|
||||
|
||||
Open `assets/main.rml` and add...
|
||||
```
|
||||
|
||||
### Guides (Conceptual)
|
||||
|
||||
```markdown
|
||||
# Understanding Permissions
|
||||
|
||||
Mosis uses a permission system to protect user privacy.
|
||||
Apps must declare permissions in their manifest and
|
||||
request them at runtime.
|
||||
|
||||
## Permission Categories
|
||||
|
||||
| Category | Description | Examples |
|
||||
|----------|-------------|----------|
|
||||
| Normal | Low risk, auto-granted | storage, network |
|
||||
| Dangerous | User data, requires prompt | camera, location |
|
||||
| Signature | System only | app_management |
|
||||
|
||||
## When Permissions Are Checked
|
||||
|
||||
Permissions are checked when your app calls...
|
||||
```
|
||||
|
||||
### API Reference (Technical)
|
||||
|
||||
```markdown
|
||||
# mosis.http
|
||||
|
||||
HTTP client for making network requests.
|
||||
|
||||
## Functions
|
||||
|
||||
### `mosis.http.get(url, options)`
|
||||
|
||||
Make a GET request.
|
||||
|
||||
**Parameters:**
|
||||
- `url` (string): The URL to fetch
|
||||
- `options` (table, optional):
|
||||
- `headers` (table): Request headers
|
||||
- `timeout` (number): Timeout in ms (default: 30000)
|
||||
|
||||
**Returns:** Promise that resolves to Response
|
||||
|
||||
**Example:**
|
||||
```lua
|
||||
local response = mosis.http.get("https://api.example.com/data")
|
||||
if response.ok then
|
||||
local data = json.decode(response.body)
|
||||
print(data.name)
|
||||
end
|
||||
```
|
||||
|
||||
**Errors:**
|
||||
- `NETWORK_ERROR`: Network unavailable
|
||||
- `TIMEOUT`: Request timed out
|
||||
- `INVALID_URL`: Malformed URL
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack Options
|
||||
|
||||
### Option A: Docusaurus
|
||||
|
||||
```
|
||||
Framework: Docusaurus 3
|
||||
Language: MDX (Markdown + React)
|
||||
Search: Algolia DocSearch
|
||||
Deploy: Vercel/Cloudflare Pages
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Versioning built-in | React knowledge needed |
|
||||
| Great search | Can be heavy |
|
||||
| Plugin ecosystem | |
|
||||
| Used by many OSS | |
|
||||
|
||||
### Option B: VitePress
|
||||
|
||||
```
|
||||
Framework: VitePress
|
||||
Language: Markdown + Vue
|
||||
Search: Built-in local search
|
||||
Deploy: Any static host
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Very fast | Fewer features |
|
||||
| Vue-powered | Less ecosystem |
|
||||
| Simple setup | |
|
||||
|
||||
### Option C: Astro Starlight
|
||||
|
||||
```
|
||||
Framework: Astro + Starlight
|
||||
Language: MDX
|
||||
Search: Pagefind (local)
|
||||
Deploy: Any static host
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Very fast | Newer |
|
||||
| Great defaults | Less customizable |
|
||||
| Built-in i18n | |
|
||||
|
||||
### Option D: MkDocs Material
|
||||
|
||||
```
|
||||
Framework: MkDocs
|
||||
Language: Markdown
|
||||
Search: Built-in
|
||||
Deploy: Any static host
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Simple | Less interactive |
|
||||
| Great theme | Python-based |
|
||||
| Fast builds | |
|
||||
|
||||
---
|
||||
|
||||
## Features Required
|
||||
|
||||
### Must Have
|
||||
|
||||
- [ ] Full-text search
|
||||
- [ ] Syntax highlighting
|
||||
- [ ] Mobile responsive
|
||||
- [ ] Dark mode
|
||||
- [ ] Version selector
|
||||
- [ ] Edit on GitHub links
|
||||
- [ ] Copy code buttons
|
||||
- [ ] Table of contents
|
||||
- [ ] Previous/Next navigation
|
||||
|
||||
### Nice to Have
|
||||
|
||||
- [ ] API playground
|
||||
- [ ] Interactive examples
|
||||
- [ ] Video tutorials
|
||||
- [ ] Community translations
|
||||
- [ ] Feedback widget
|
||||
|
||||
---
|
||||
|
||||
## Code Examples
|
||||
|
||||
### Runnable Examples
|
||||
|
||||
```html
|
||||
<!-- Interactive code block -->
|
||||
<CodePlayground language="lua">
|
||||
local greeting = "Hello, Mosis!"
|
||||
print(greeting)
|
||||
</CodePlayground>
|
||||
```
|
||||
|
||||
### Multi-file Examples
|
||||
|
||||
```markdown
|
||||
:::code-group
|
||||
|
||||
```rml [main.rml]
|
||||
<div id="counter">
|
||||
<span id="count">0</span>
|
||||
<button onclick="increment()">+</button>
|
||||
</div>
|
||||
```
|
||||
|
||||
```lua [app.lua]
|
||||
local count = 0
|
||||
|
||||
function increment()
|
||||
count = count + 1
|
||||
document:GetElementById("count").inner_rml = tostring(count)
|
||||
end
|
||||
```
|
||||
|
||||
```rcss [styles.rcss]
|
||||
#counter {
|
||||
display: flex;
|
||||
gap: 10px;
|
||||
}
|
||||
```
|
||||
|
||||
:::
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Documentation Generation
|
||||
|
||||
### From Lua Annotations
|
||||
|
||||
```lua
|
||||
--- Make an HTTP GET request.
|
||||
--- @param url string The URL to fetch
|
||||
--- @param options? HttpOptions Request options
|
||||
--- @return HttpResponse response The response object
|
||||
--- @example
|
||||
--- local resp = mosis.http.get("https://api.example.com/data")
|
||||
--- print(resp.body)
|
||||
function mosis.http.get(url, options)
|
||||
-- implementation
|
||||
end
|
||||
```
|
||||
|
||||
### Generated Output
|
||||
|
||||
```markdown
|
||||
## `mosis.http.get(url, options?)`
|
||||
|
||||
Make an HTTP GET request.
|
||||
|
||||
### Parameters
|
||||
|
||||
| Name | Type | Description |
|
||||
|------|------|-------------|
|
||||
| url | string | The URL to fetch |
|
||||
| options | HttpOptions? | Request options |
|
||||
|
||||
### Returns
|
||||
|
||||
`HttpResponse` - The response object
|
||||
|
||||
### Example
|
||||
|
||||
```lua
|
||||
local resp = mosis.http.get("https://api.example.com/data")
|
||||
print(resp.body)
|
||||
```
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Versioning
|
||||
|
||||
### URL Structure
|
||||
|
||||
```
|
||||
docs.mosis.dev/ # Latest stable
|
||||
docs.mosis.dev/v1.0/ # Version 1.0
|
||||
docs.mosis.dev/v1.1/ # Version 1.1
|
||||
docs.mosis.dev/next/ # Development (unreleased)
|
||||
```
|
||||
|
||||
### Version Dropdown
|
||||
|
||||
```
|
||||
┌────────────────────┐
|
||||
│ Version: 1.1 (latest) ▼ │
|
||||
├────────────────────┤
|
||||
│ 1.1 (latest) │
|
||||
│ 1.0 │
|
||||
│ next (unreleased) │
|
||||
└────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Search
|
||||
|
||||
### Algolia DocSearch (Free for OSS)
|
||||
|
||||
```javascript
|
||||
// docusaurus.config.js
|
||||
themeConfig: {
|
||||
algolia: {
|
||||
appId: 'YOUR_APP_ID',
|
||||
apiKey: 'YOUR_SEARCH_KEY',
|
||||
indexName: 'mosis',
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### Local Search (Pagefind)
|
||||
|
||||
```javascript
|
||||
// For Astro/VitePress
|
||||
// Indexes at build time, searches client-side
|
||||
// No external service needed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Internationalization
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
docs/
|
||||
├── en/
|
||||
│ ├── getting-started/
|
||||
│ └── guides/
|
||||
├── es/
|
||||
│ ├── getting-started/
|
||||
│ └── guides/
|
||||
└── zh/
|
||||
├── getting-started/
|
||||
└── guides/
|
||||
```
|
||||
|
||||
### Language Switcher
|
||||
|
||||
```
|
||||
┌──────────┐
|
||||
│ 🌐 EN ▼ │
|
||||
├──────────┤
|
||||
│ English │
|
||||
│ Español │
|
||||
│ 中文 │
|
||||
└──────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Content Guidelines
|
||||
|
||||
### Writing Style
|
||||
|
||||
1. **Be concise** - Get to the point quickly
|
||||
2. **Use active voice** - "The function returns..." not "A value is returned..."
|
||||
3. **Show, don't tell** - Code examples over explanations
|
||||
4. **Assume beginner** - Don't assume prior knowledge
|
||||
5. **Test all examples** - Every code block must work
|
||||
|
||||
### Code Style
|
||||
|
||||
```lua
|
||||
-- Good: Clear, commented
|
||||
local response = mosis.http.get(API_URL)
|
||||
if response.ok then
|
||||
local data = json.decode(response.body)
|
||||
updateUI(data)
|
||||
else
|
||||
showError("Failed to load data")
|
||||
end
|
||||
|
||||
-- Bad: Unclear, no error handling
|
||||
local d = json.decode(mosis.http.get(u).body)
|
||||
```
|
||||
|
||||
### Screenshots
|
||||
|
||||
- Use consistent device frame
|
||||
- Show relevant UI only
|
||||
- Add callouts for important areas
|
||||
- Keep file sizes small (WebP)
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
### CI/CD Pipeline
|
||||
|
||||
```yaml
|
||||
# .github/workflows/docs.yml
|
||||
name: Deploy Docs
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'docs/**'
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
|
||||
- name: Install and Build
|
||||
run: |
|
||||
cd docs
|
||||
npm install
|
||||
npm run build
|
||||
|
||||
- name: Deploy to Cloudflare Pages
|
||||
uses: cloudflare/pages-action@v1
|
||||
with:
|
||||
apiToken: ${{ secrets.CF_API_TOKEN }}
|
||||
accountId: ${{ secrets.CF_ACCOUNT_ID }}
|
||||
projectName: mosis-docs
|
||||
directory: docs/build
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Analytics
|
||||
|
||||
### Track (Privacy-Friendly)
|
||||
|
||||
- Page views (which docs are popular)
|
||||
- Search queries (what are people looking for)
|
||||
- 404 pages (missing content)
|
||||
- Time on page (engagement)
|
||||
|
||||
### Tools
|
||||
|
||||
- Plausible (privacy-focused)
|
||||
- Simple Analytics
|
||||
- Cloudflare Analytics (free)
|
||||
|
||||
---
|
||||
|
||||
## Feedback
|
||||
|
||||
### Per-Page Feedback
|
||||
|
||||
```
|
||||
┌────────────────────────────────────┐
|
||||
│ Was this page helpful? │
|
||||
│ │
|
||||
│ [👍 Yes] [👎 No] │
|
||||
│ │
|
||||
│ [Edit this page on GitHub] │
|
||||
└────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Feedback Collection
|
||||
|
||||
```javascript
|
||||
// Send to analytics or issue tracker
|
||||
function submitFeedback(page, helpful, comment) {
|
||||
fetch('/api/feedback', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ page, helpful, comment })
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
- [ ] Framework selection
|
||||
- [ ] Information architecture
|
||||
- [ ] Getting Started content
|
||||
- [ ] UI design guides
|
||||
- [ ] Lua scripting guides
|
||||
- [ ] API reference (all namespaces)
|
||||
- [ ] CLI reference
|
||||
- [ ] Best practices
|
||||
- [ ] Search integration
|
||||
- [ ] Version selector
|
||||
- [ ] Deploy pipeline
|
||||
- [ ] Feedback system
|
||||
|
||||
---
|
||||
|
||||
## Content Prioritization
|
||||
|
||||
### Phase 1 (Launch)
|
||||
|
||||
1. Quick Start
|
||||
2. Your First App
|
||||
3. Project Structure
|
||||
4. RML Basics
|
||||
5. Lua Basics
|
||||
6. API Reference (core APIs)
|
||||
7. CLI Reference
|
||||
|
||||
### Phase 2
|
||||
|
||||
1. Complete API Reference
|
||||
2. All hardware guides
|
||||
3. Best practices
|
||||
4. Troubleshooting
|
||||
|
||||
### Phase 3
|
||||
|
||||
1. Advanced guides
|
||||
2. Video tutorials
|
||||
3. Translations
|
||||
4. Community contributions
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Host docs separately or under main domain?
|
||||
2. Community wiki/contributions?
|
||||
3. Video tutorial platform (YouTube, embedded)?
|
||||
4. Glossary/terminology page?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Docusaurus](https://docusaurus.io/)
|
||||
- [VitePress](https://vitepress.dev/)
|
||||
- [Astro Starlight](https://starlight.astro.build/)
|
||||
- [Divio Documentation System](https://documentation.divio.com/)
|
||||
714
DEV_PORTAL_MILESTONES.md
Normal file
714
DEV_PORTAL_MILESTONES.md
Normal file
@@ -0,0 +1,714 @@
|
||||
# Developer Portal & App Ecosystem Milestones
|
||||
|
||||
Planning document for the Mosis app development, distribution, and monitoring ecosystem.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
```
|
||||
Developer Mosis Platform User Device
|
||||
─────────────────────────────────────────────────────────────────────────────────
|
||||
Register Account ──────────────► Developer Portal
|
||||
│
|
||||
Create App Project ────────────► App Registry
|
||||
│
|
||||
Design UI (RML/RCSS/Lua) ──────► Designer Tool (local)
|
||||
│
|
||||
Test Locally ──────────────────► Desktop Designer / Emulator
|
||||
│
|
||||
Submit for Review ─────────────► App Store Backend
|
||||
│
|
||||
Publish ───────────────────────► CDN / Distribution
|
||||
│
|
||||
Install ◄─────────────────── User Request
|
||||
│
|
||||
Run in Sandbox ◄──────────── Launch App
|
||||
│
|
||||
Telemetry/Crashes ──────────► Analytics Backend
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Decision Areas
|
||||
|
||||
| Area | Options | Status |
|
||||
|------|---------|--------|
|
||||
| Web Stack | Node/Express, Go, Rust/Axum, .NET | TBD |
|
||||
| Database | PostgreSQL, SQLite, MongoDB | TBD |
|
||||
| Auth | OAuth2/OIDC, API keys, JWT | TBD |
|
||||
| CDN/Storage | S3, Cloudflare R2, self-hosted | TBD |
|
||||
| Telemetry | Custom, PostHog, Plausible | TBD |
|
||||
| Crash Reports | Sentry, custom, Crashlytics | TBD |
|
||||
| App Format | ZIP, custom package, signed | TBD |
|
||||
|
||||
---
|
||||
|
||||
## Milestone 1: App Package Format
|
||||
|
||||
**Goal**: Define how apps are bundled, signed, and validated.
|
||||
|
||||
### Questions to Answer
|
||||
|
||||
1. What files comprise an app package?
|
||||
2. How is the manifest structured?
|
||||
3. How are apps signed for integrity?
|
||||
4. How are updates handled (full vs delta)?
|
||||
5. What metadata is required (name, version, permissions, icons)?
|
||||
|
||||
### Considerations
|
||||
|
||||
| Approach | Pros | Cons |
|
||||
|----------|------|------|
|
||||
| **ZIP archive** | Simple, standard tooling | No built-in signing |
|
||||
| **Custom format (.mosis)** | Can embed signature, metadata | Custom tooling needed |
|
||||
| **Signed ZIP** | Best of both, detached signature | Slightly more complex |
|
||||
|
||||
### Proposed Structure
|
||||
|
||||
```
|
||||
myapp.mosis/
|
||||
├── manifest.json # App metadata, permissions, entry point
|
||||
├── signature.sig # Detached signature (optional for dev)
|
||||
├── icon.png # App icon (multiple sizes?)
|
||||
├── assets/
|
||||
│ ├── main.rml # Entry point UI
|
||||
│ ├── styles.rcss # Stylesheets
|
||||
│ └── scripts/
|
||||
│ └── app.lua # Lua code
|
||||
└── locales/ # i18n (optional)
|
||||
├── en.json
|
||||
└── es.json
|
||||
```
|
||||
|
||||
### Manifest Schema (Draft)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "com.developer.appname",
|
||||
"name": "My App",
|
||||
"version": "1.0.0",
|
||||
"version_code": 1,
|
||||
"entry": "assets/main.rml",
|
||||
"permissions": ["storage", "network"],
|
||||
"min_mosis_version": "1.0.0",
|
||||
"author": {
|
||||
"name": "Developer Name",
|
||||
"email": "dev@example.com"
|
||||
},
|
||||
"icons": {
|
||||
"32": "icon-32.png",
|
||||
"64": "icon-64.png",
|
||||
"128": "icon-128.png"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] Manifest JSON schema specification
|
||||
- [ ] Package format specification
|
||||
- [ ] Signing mechanism (key format, algorithm)
|
||||
- [ ] Package validation tool (CLI)
|
||||
- [ ] Package creation tool (CLI or integrated in designer)
|
||||
|
||||
---
|
||||
|
||||
## Milestone 2: Web Stack Selection
|
||||
|
||||
**Goal**: Choose backend technologies for the developer portal and app store.
|
||||
|
||||
### Options Analysis
|
||||
|
||||
#### Option A: Node.js + Express/Fastify
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Language | TypeScript |
|
||||
| Framework | Express, Fastify, or Hono |
|
||||
| Pros | Large ecosystem, easy hiring, fast development |
|
||||
| Cons | Single-threaded, callback complexity |
|
||||
| Hosting | Vercel, Railway, any VPS |
|
||||
|
||||
#### Option B: Go
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Language | Go |
|
||||
| Framework | Gin, Echo, or Chi |
|
||||
| Pros | Fast, low memory, single binary deployment |
|
||||
| Cons | Smaller ecosystem, verbose error handling |
|
||||
| Hosting | Any VPS, Cloud Run |
|
||||
|
||||
#### Option C: Rust + Axum
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Language | Rust |
|
||||
| Framework | Axum, Actix-web |
|
||||
| Pros | Maximum performance, memory safety |
|
||||
| Cons | Steep learning curve, slower development |
|
||||
| Hosting | Any VPS, Fly.io |
|
||||
|
||||
#### Option D: .NET
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Language | C# |
|
||||
| Framework | ASP.NET Core |
|
||||
| Pros | Enterprise-ready, great tooling, fast |
|
||||
| Cons | Heavier runtime, Microsoft ecosystem |
|
||||
| Hosting | Azure, any VPS |
|
||||
|
||||
### Evaluation Criteria
|
||||
|
||||
1. **Development speed** - How fast can we iterate?
|
||||
2. **Performance** - Can it handle scale?
|
||||
3. **Hosting cost** - Monthly infrastructure cost
|
||||
4. **Team familiarity** - Learning curve
|
||||
5. **Ecosystem** - Libraries for auth, storage, etc.
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] Prototype API in top 2 candidates
|
||||
- [ ] Benchmark comparison
|
||||
- [ ] Final selection with rationale
|
||||
|
||||
---
|
||||
|
||||
## Milestone 3: Database Selection
|
||||
|
||||
**Goal**: Choose database for developer accounts, app metadata, analytics.
|
||||
|
||||
### Options Analysis
|
||||
|
||||
#### Option A: PostgreSQL
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Type | Relational |
|
||||
| Pros | ACID, JSON support, mature, scalable |
|
||||
| Cons | Requires management, connection pooling |
|
||||
| Hosting | Supabase, Neon, RDS, self-hosted |
|
||||
|
||||
#### Option B: SQLite + Litestream
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Type | Embedded relational |
|
||||
| Pros | Zero ops, fast reads, simple backup |
|
||||
| Cons | Single-writer, limited concurrency |
|
||||
| Hosting | Embedded in app server |
|
||||
|
||||
#### Option C: MongoDB
|
||||
|
||||
| Aspect | Details |
|
||||
|--------|---------|
|
||||
| Type | Document |
|
||||
| Pros | Flexible schema, easy start |
|
||||
| Cons | Less ACID, can get expensive |
|
||||
| Hosting | Atlas, self-hosted |
|
||||
|
||||
### Data Models Preview
|
||||
|
||||
```
|
||||
developers
|
||||
├── id (UUID)
|
||||
├── email
|
||||
├── name
|
||||
├── api_keys[]
|
||||
├── created_at
|
||||
└── verified
|
||||
|
||||
apps
|
||||
├── id (UUID)
|
||||
├── developer_id (FK)
|
||||
├── package_id (com.dev.app)
|
||||
├── name
|
||||
├── description
|
||||
├── versions[]
|
||||
├── status (draft/review/published/suspended)
|
||||
├── created_at
|
||||
└── updated_at
|
||||
|
||||
app_versions
|
||||
├── id (UUID)
|
||||
├── app_id (FK)
|
||||
├── version_code
|
||||
├── version_name
|
||||
├── package_url
|
||||
├── signature
|
||||
├── release_notes
|
||||
├── status
|
||||
└── published_at
|
||||
|
||||
telemetry_events
|
||||
├── id
|
||||
├── app_id
|
||||
├── device_id (anonymized)
|
||||
├── event_type
|
||||
├── event_data (JSON)
|
||||
├── timestamp
|
||||
└── mosis_version
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] Schema design for all tables
|
||||
- [ ] Migration strategy
|
||||
- [ ] Backup/restore plan
|
||||
- [ ] Final selection with rationale
|
||||
|
||||
---
|
||||
|
||||
## Milestone 4: Authentication System
|
||||
|
||||
**Goal**: Secure developer authentication and app signing.
|
||||
|
||||
### Developer Authentication
|
||||
|
||||
| Method | Use Case |
|
||||
|--------|----------|
|
||||
| OAuth2 (GitHub/Google) | Portal login |
|
||||
| Email + Password | Alternative login |
|
||||
| API Keys | CLI tools, CI/CD |
|
||||
| JWT | Session tokens |
|
||||
|
||||
### App Signing
|
||||
|
||||
| Approach | Details |
|
||||
|----------|---------|
|
||||
| **Developer keypair** | Dev signs with private key, we verify with public |
|
||||
| **Platform signing** | We sign after review (like iOS) |
|
||||
| **Both** | Dev signs, we countersign after review |
|
||||
|
||||
### Key Management
|
||||
|
||||
- Developer generates keypair locally
|
||||
- Public key uploaded to portal
|
||||
- Private key never leaves developer machine
|
||||
- Key rotation supported
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] OAuth2 integration (GitHub, Google)
|
||||
- [ ] API key generation and management
|
||||
- [ ] Developer keypair registration
|
||||
- [ ] App signature verification
|
||||
- [ ] JWT token handling
|
||||
|
||||
---
|
||||
|
||||
## Milestone 5: Developer Portal Frontend
|
||||
|
||||
**Goal**: Web interface for developer account and app management.
|
||||
|
||||
### Pages Required
|
||||
|
||||
| Page | Features |
|
||||
|------|----------|
|
||||
| **Landing** | Sign up, sign in, overview |
|
||||
| **Dashboard** | App list, quick stats |
|
||||
| **App Details** | Versions, analytics, settings |
|
||||
| **Create App** | Wizard for new app |
|
||||
| **Submit Version** | Upload, release notes, submit |
|
||||
| **API Keys** | Generate, revoke keys |
|
||||
| **Profile** | Account settings, keys |
|
||||
| **Docs** | SDK docs, API reference |
|
||||
|
||||
### Tech Options
|
||||
|
||||
| Option | Pros | Cons |
|
||||
|--------|------|------|
|
||||
| Next.js | SSR, React, full-stack | Complexity |
|
||||
| SvelteKit | Fast, simple, SSR | Smaller ecosystem |
|
||||
| Astro + React | Static + islands | Newer |
|
||||
| Plain HTML + htmx | Simple, fast | Limited interactivity |
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] Framework selection
|
||||
- [ ] UI component library selection
|
||||
- [ ] Page wireframes
|
||||
- [ ] Implementation
|
||||
|
||||
---
|
||||
|
||||
## Milestone 6: App Store Backend API
|
||||
|
||||
**Goal**: REST/GraphQL API for app submission, review, and distribution.
|
||||
|
||||
### API Endpoints (Draft)
|
||||
|
||||
```
|
||||
Auth
|
||||
├── POST /auth/register
|
||||
├── POST /auth/login
|
||||
├── POST /auth/logout
|
||||
├── GET /auth/me
|
||||
|
||||
Apps
|
||||
├── GET /apps # List developer's apps
|
||||
├── POST /apps # Create new app
|
||||
├── GET /apps/:id # Get app details
|
||||
├── PATCH /apps/:id # Update app metadata
|
||||
├── DELETE /apps/:id # Delete app (if no published versions)
|
||||
|
||||
Versions
|
||||
├── GET /apps/:id/versions # List versions
|
||||
├── POST /apps/:id/versions # Upload new version
|
||||
├── GET /apps/:id/versions/:vid # Get version details
|
||||
├── POST /apps/:id/versions/:vid/submit # Submit for review
|
||||
├── POST /apps/:id/versions/:vid/publish # Publish (after review)
|
||||
|
||||
Public (App Store)
|
||||
├── GET /store/apps # Browse/search apps
|
||||
├── GET /store/apps/:id # App store listing
|
||||
├── GET /store/apps/:id/download # Download latest version
|
||||
|
||||
API Keys
|
||||
├── GET /keys # List API keys
|
||||
├── POST /keys # Generate new key
|
||||
├── DELETE /keys/:id # Revoke key
|
||||
|
||||
Telemetry (device → server)
|
||||
├── POST /telemetry/events # Batch event upload
|
||||
├── POST /telemetry/crash # Crash report
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] OpenAPI specification
|
||||
- [ ] Rate limiting strategy
|
||||
- [ ] Authentication middleware
|
||||
- [ ] Implementation
|
||||
|
||||
---
|
||||
|
||||
## Milestone 7: CDN & Storage
|
||||
|
||||
**Goal**: Scalable storage for app packages and assets.
|
||||
|
||||
### Requirements
|
||||
|
||||
1. Store app packages (.mosis files)
|
||||
2. Serve downloads globally with low latency
|
||||
3. Handle icons and screenshots
|
||||
4. Version retention policy
|
||||
5. Bandwidth cost management
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Pros | Cons |
|
||||
|--------|------|------|
|
||||
| **Cloudflare R2** | No egress fees, global | Newer service |
|
||||
| **AWS S3 + CloudFront** | Mature, reliable | Egress costs |
|
||||
| **Backblaze B2 + Cloudflare** | Cheap storage, free egress via CF | More setup |
|
||||
| **Self-hosted MinIO** | Full control | Ops burden |
|
||||
|
||||
### Storage Structure
|
||||
|
||||
```
|
||||
/packages/
|
||||
/{app_id}/
|
||||
/{version_code}/
|
||||
package.mosis
|
||||
signature.sig
|
||||
|
||||
/assets/
|
||||
/{app_id}/
|
||||
icon-32.png
|
||||
icon-64.png
|
||||
icon-128.png
|
||||
screenshots/
|
||||
1.png
|
||||
2.png
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] Storage provider selection
|
||||
- [ ] CDN configuration
|
||||
- [ ] Upload flow (presigned URLs vs direct)
|
||||
- [ ] Download URL generation
|
||||
- [ ] Retention/cleanup policy
|
||||
|
||||
---
|
||||
|
||||
## Milestone 8: Telemetry System
|
||||
|
||||
**Goal**: Collect app usage analytics and crash reports.
|
||||
|
||||
### Event Types
|
||||
|
||||
| Category | Events |
|
||||
|----------|--------|
|
||||
| **Lifecycle** | app_start, app_stop, app_crash |
|
||||
| **Performance** | frame_time, memory_usage, lua_errors |
|
||||
| **Usage** | screen_view, button_click (opt-in) |
|
||||
| **System** | mosis_version, device_info |
|
||||
|
||||
### Privacy Considerations
|
||||
|
||||
1. **No PII by default** - Device ID is hashed, no user data
|
||||
2. **Opt-in for detailed analytics** - User consent required
|
||||
3. **Data retention** - Auto-delete after X days
|
||||
4. **GDPR compliance** - Export/delete on request
|
||||
5. **Aggregation** - Store aggregates, drop raw after processing
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Pros | Cons |
|
||||
|--------|------|------|
|
||||
| **Custom** | Full control, no vendor lock | Build everything |
|
||||
| **PostHog** | Self-hostable, feature-rich | Can be heavy |
|
||||
| **Plausible** | Privacy-focused, simple | Limited features |
|
||||
| **Segment + warehouse** | Flexible routing | Complex, costly |
|
||||
|
||||
### Crash Report Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"app_id": "com.dev.app",
|
||||
"app_version": "1.0.0",
|
||||
"mosis_version": "1.0.0",
|
||||
"device_id": "hashed",
|
||||
"timestamp": "2024-01-15T10:30:00Z",
|
||||
"crash_type": "lua_error",
|
||||
"message": "attempt to index nil value",
|
||||
"stack_trace": "...",
|
||||
"context": {
|
||||
"screen": "main.rml",
|
||||
"memory_mb": 45,
|
||||
"uptime_seconds": 120
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] Event schema specification
|
||||
- [ ] Collection endpoint
|
||||
- [ ] Storage strategy (time-series DB?)
|
||||
- [ ] Dashboard for developers
|
||||
- [ ] Privacy controls
|
||||
|
||||
---
|
||||
|
||||
## Milestone 9: App Review System
|
||||
|
||||
**Goal**: Automated and manual review process for app submissions.
|
||||
|
||||
### Automated Checks
|
||||
|
||||
| Check | Description |
|
||||
|-------|-------------|
|
||||
| **Manifest validation** | Required fields, valid permissions |
|
||||
| **Package integrity** | Signature verification |
|
||||
| **Static analysis** | Dangerous Lua patterns |
|
||||
| **Asset validation** | Icons present, correct sizes |
|
||||
| **Size limits** | Package under max size |
|
||||
| **Duplicate detection** | Same app ID collision |
|
||||
|
||||
### Manual Review (Optional)
|
||||
|
||||
- Flag for manual review based on:
|
||||
- New developer (first app)
|
||||
- Dangerous permissions requested
|
||||
- Automated check warnings
|
||||
- User reports
|
||||
|
||||
### Review States
|
||||
|
||||
```
|
||||
draft → submitted → in_review → approved → published
|
||||
↘ rejected (with feedback)
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] Automated validation pipeline
|
||||
- [ ] Review queue UI
|
||||
- [ ] Rejection feedback system
|
||||
- [ ] Appeal process
|
||||
|
||||
---
|
||||
|
||||
## Milestone 10: Device-Side App Management
|
||||
|
||||
**Goal**: Install, update, and manage apps on Mosis devices.
|
||||
|
||||
### Components
|
||||
|
||||
| Component | Location | Purpose |
|
||||
|-----------|----------|---------|
|
||||
| App Manager | MosisService | Install/uninstall/update apps |
|
||||
| App Store Client | System app | Browse, search, install UI |
|
||||
| Update Checker | Background service | Check for updates |
|
||||
|
||||
### Installation Flow
|
||||
|
||||
```
|
||||
1. User taps "Install" in App Store
|
||||
2. Download package from CDN
|
||||
3. Verify signature
|
||||
4. Extract to app directory
|
||||
5. Register with LuaSandboxManager
|
||||
6. Add to home screen
|
||||
```
|
||||
|
||||
### Update Flow
|
||||
|
||||
```
|
||||
1. Background check for updates (daily?)
|
||||
2. Notify user of available updates
|
||||
3. Download new version
|
||||
4. Verify signature
|
||||
5. Replace app files (atomic swap)
|
||||
6. Restart app if running
|
||||
```
|
||||
|
||||
### Storage Layout
|
||||
|
||||
```
|
||||
/data/mosis/
|
||||
/apps/
|
||||
/com.dev.app/
|
||||
/package/ # Extracted app files
|
||||
/data/ # App data (VirtualFS)
|
||||
/cache/ # App cache
|
||||
/db/ # SQLite databases
|
||||
```
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] AppManager class in MosisService
|
||||
- [ ] App Store system app
|
||||
- [ ] Update checking service
|
||||
- [ ] Uninstall with data cleanup
|
||||
|
||||
---
|
||||
|
||||
## Milestone 11: Developer CLI Tool
|
||||
|
||||
**Goal**: Command-line tool for app development workflow.
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
# Project management
|
||||
mosis init # Create new app project
|
||||
mosis validate # Validate manifest and assets
|
||||
|
||||
# Packaging
|
||||
mosis build # Create .mosis package
|
||||
mosis sign # Sign package with developer key
|
||||
|
||||
# Testing
|
||||
mosis run # Run in local designer/emulator
|
||||
mosis test # Run automated tests
|
||||
|
||||
# Publishing
|
||||
mosis login # Authenticate with portal
|
||||
mosis publish # Upload and submit for review
|
||||
mosis status # Check review status
|
||||
|
||||
# Keys
|
||||
mosis keys generate # Generate signing keypair
|
||||
mosis keys register # Upload public key to portal
|
||||
```
|
||||
|
||||
### Implementation Options
|
||||
|
||||
| Option | Pros | Cons |
|
||||
|--------|------|------|
|
||||
| Node.js (oclif) | Easy to build, npm distribution | Requires Node |
|
||||
| Go | Single binary, fast | Slower development |
|
||||
| Rust (clap) | Single binary, fast | Slower development |
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] CLI framework selection
|
||||
- [ ] Core commands implementation
|
||||
- [ ] Distribution (npm, homebrew, direct download)
|
||||
- [ ] Documentation
|
||||
|
||||
---
|
||||
|
||||
## Milestone 12: Documentation Site
|
||||
|
||||
**Goal**: Comprehensive docs for developers.
|
||||
|
||||
### Sections
|
||||
|
||||
| Section | Content |
|
||||
|---------|---------|
|
||||
| **Getting Started** | Quick start, first app tutorial |
|
||||
| **Guides** | UI design, Lua scripting, permissions |
|
||||
| **API Reference** | All Lua APIs, manifest schema |
|
||||
| **CLI Reference** | All commands and options |
|
||||
| **Best Practices** | Performance, security, UX |
|
||||
| **Troubleshooting** | Common issues, FAQ |
|
||||
|
||||
### Tech Options
|
||||
|
||||
| Option | Pros | Cons |
|
||||
|--------|------|------|
|
||||
| Docusaurus | React-based, versioning | Heavy |
|
||||
| VitePress | Vue-based, fast | Less features |
|
||||
| Astro Starlight | Fast, modern | Newer |
|
||||
| MkDocs | Python, simple | Less customizable |
|
||||
|
||||
### Deliverables
|
||||
|
||||
- [ ] Framework selection
|
||||
- [ ] Information architecture
|
||||
- [ ] Content writing
|
||||
- [ ] API docs generation from code
|
||||
- [ ] Search integration
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Phase | Milestones | Description |
|
||||
|-------|------------|-------------|
|
||||
| **Foundation** | 1-4 | Package format, web stack, database, auth |
|
||||
| **Portal** | 5-6 | Developer portal frontend and API |
|
||||
| **Distribution** | 7, 10 | CDN/storage, device-side app management |
|
||||
| **Quality** | 8-9 | Telemetry, crash reports, review system |
|
||||
| **Tooling** | 11-12 | CLI tool, documentation |
|
||||
|
||||
### Recommended Order
|
||||
|
||||
1. **Milestone 1** - Package format (needed by everything)
|
||||
2. **Milestone 2** - Web stack selection
|
||||
3. **Milestone 3** - Database selection
|
||||
4. **Milestone 4** - Authentication
|
||||
5. **Milestone 6** - Backend API
|
||||
6. **Milestone 5** - Portal frontend
|
||||
7. **Milestone 7** - CDN/storage
|
||||
8. **Milestone 10** - Device-side management
|
||||
9. **Milestone 11** - CLI tool
|
||||
10. **Milestone 9** - Review system
|
||||
11. **Milestone 8** - Telemetry
|
||||
12. **Milestone 12** - Documentation
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Monetization model?** - Free only, paid apps, subscriptions?
|
||||
2. **Enterprise/self-hosted?** - Can companies run private app stores?
|
||||
3. **App categories?** - Predefined or free-form tags?
|
||||
4. **Rating/reviews?** - User reviews for apps?
|
||||
5. **Analytics dashboard?** - What metrics do developers see?
|
||||
6. **Localization?** - Multi-language portal and apps?
|
||||
7. **Beta testing?** - TestFlight-like distribution?
|
||||
8. **Team accounts?** - Multiple developers per app?
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
Begin with Milestone 1 (App Package Format) to establish the foundation, then proceed with technology selections in Milestones 2-4 before building the portal.
|
||||
Reference in New Issue
Block a user