omar/MosisService
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b86ee549343b7da305afef6918fd4eea4627d52f
MosisService/DEV_PORTAL_M07_STORAGE.md

9.3 KiB
Raw Blame History

Milestone 7: CDN & Storage

Status: Planning Goal: Scalable storage for app packages and assets with global distribution.

Overview

Storage handles app packages, icons, screenshots, and serves downloads to devices worldwide. Must be cost-effective, fast, and reliable.

Requirements

Functional

  • Store app packages (.mosis files)
  • Store icons (multiple sizes)
  • Store screenshots (optional)
  • Serve downloads globally
  • Support presigned upload URLs
  • Version retention (keep old versions)

Non-Functional

Requirement Target
Upload speed < 30s for 50MB
Download latency < 100ms (p50)
Availability 99.9%
Durability 99.999999999% (11 nines)
Cost Minimize egress fees

Storage Structure

/packages/
  /{developer_id}/
    /{app_id}/
      /{version_code}/
        package.mosis
        manifest.json (extracted for quick access)

/assets/
  /{app_id}/
    icon-32.png
    icon-64.png
    icon-128.png
    screenshots/
      1.png
      2.png
      3.png

/temp/
  /{upload_id}/
    package.mosis (pending validation)

Options Analysis

Option A: Cloudflare R2

Storage:    Object storage (S3-compatible)
CDN:        Cloudflare network (automatic)
Egress:     FREE (zero egress fees)
Pricing:    $0.015/GB storage, $4.50/million requests
Pros Cons
No egress fees Newer service
Global CDN included Fewer regions than S3
S3-compatible API Less tooling
Workers integration

Cost Estimate (10K apps, 100GB)

Component Monthly Cost
Storage (100GB) $1.50
Requests (1M) $4.50
Egress $0
Total ~$6

Option B: AWS S3 + CloudFront

Storage:    S3 Standard
CDN:        CloudFront
Egress:     $0.085-0.12/GB (varies by region)
Pricing:    $0.023/GB storage
Pros Cons
Most mature Egress costs add up
Best tooling Complex pricing
All regions Need CloudFront config
IAM integration

Cost Estimate (10K apps, 100GB, 1TB egress)

Component Monthly Cost
Storage (100GB) $2.30
CloudFront (1TB) $85
Requests ~$5
Total ~$92

Option C: Backblaze B2 + Cloudflare

Storage:    Backblaze B2
CDN:        Cloudflare (free egress via Bandwidth Alliance)
Egress:     FREE (through Cloudflare)
Pricing:    $0.005/GB storage
Pros Cons
Cheapest storage Two services to manage
Free egress via CF B2 API slightly different
Good reliability Need CF proxy setup

Cost Estimate (10K apps, 100GB)

Component Monthly Cost
Storage (100GB) $0.50
Egress (via CF) $0
Requests ~$0.40
Total ~$1

Option D: Self-hosted MinIO

Storage:    MinIO on VPS
CDN:        Cloudflare proxy
Egress:     VPS bandwidth
Pricing:    VPS cost only
Pros Cons
Full control Ops overhead
S3-compatible Need to manage
Predictable cost Scaling complexity

Cost Estimate

Component Monthly Cost
VPS (500GB SSD) $20-50
Cloudflare $0 (free tier)
Total ~$20-50

Recommendation

Primary: Cloudflare R2

  • Zero egress fees (biggest cost saver)
  • S3-compatible (easy migration)
  • Built-in global CDN
  • Good enough tooling

Fallback: Backblaze B2 + Cloudflare

  • If R2 has issues
  • Even cheaper storage
  • Slightly more setup

Upload Flow

Presigned URL Approach

┌────────┐    ┌─────────┐    ┌─────────┐
│ Client │───►│   API   │───►│   R2    │
└────────┘    └─────────┘    └─────────┘
     │              │              │
     │  1. Request upload URL      │
     │◄─────────────┤              │
     │  (presigned URL)            │
     │                             │
     │  2. Upload directly         │
     │────────────────────────────►│
     │                             │
     │  3. Notify complete         │
     │─────────────►│              │
     │              │  4. Validate │
     │              │─────────────►│
     │              │              │
     │  5. Confirm  │              │
     │◄─────────────┤              │

API Implementation

// 1. Request upload URL
func CreateVersion(c *gin.Context) {
    // Create version record
    version := Version{
        AppID:       appID,
        VersionCode: req.VersionCode,
        Status:      "uploading",
    }
    db.Create(&version)

    // Generate presigned URL
    key := fmt.Sprintf("temp/%s/package.mosis", version.ID)
    url, err := r2.PresignPut(key, 15*time.Minute)

    c.JSON(200, gin.H{
        "version":    version,
        "upload_url": url,
        "expires":    time.Now().Add(15 * time.Minute),
    })
}

// 3. Upload complete notification
func UploadComplete(c *gin.Context) {
    // Move from temp to final location
    tempKey := fmt.Sprintf("temp/%s/package.mosis", versionID)
    finalKey := fmt.Sprintf("packages/%s/%s/%d/package.mosis",
        developerID, appID, versionCode)

    r2.Copy(tempKey, finalKey)
    r2.Delete(tempKey)

    // Update version status
    version.Status = "validating"
    version.PackageURL = finalKey

    // Trigger async validation
    queue.Publish("validate-package", version.ID)
}

Download Flow

Public Downloads

// Get download URL (short-lived)
func GetDownloadURL(c *gin.Context) {
    version := getLatestPublishedVersion(packageID)

    // Generate presigned download URL (1 hour)
    url, _ := r2.PresignGet(version.PackageURL, 1*time.Hour)

    c.JSON(200, gin.H{
        "download_url": url,
        "version":      version.VersionName,
        "size":         version.PackageSize,
        "signature":    version.Signature,
    })
}

CDN Caching

Cache-Control: public, max-age=86400
  • Packages are immutable (version code = unique)
  • Cache aggressively at edge
  • Invalidate only if package is pulled

Icon/Screenshot Handling

Upload

// Icons uploaded with app creation/update
func UploadIcon(c *gin.Context) {
    file, _ := c.FormFile("icon")

    // Validate dimensions
    img, _ := png.Decode(file)
    if img.Bounds().Dx() != img.Bounds().Dy() {
        return error("Icon must be square")
    }

    // Generate multiple sizes
    sizes := []int{32, 64, 128}
    for _, size := range sizes {
        resized := resize(img, size, size)
        key := fmt.Sprintf("assets/%s/icon-%d.png", appID, size)
        r2.Put(key, resized)
    }
}

Serving

https://cdn.mosis.dev/assets/{app_id}/icon-64.png
  • Public read access
  • Long cache TTL (icons rarely change)
  • Cloudflare image optimization (optional)

Retention Policy

Packages

Status Retention
Published (current) Forever
Published (old) 1 year
Draft 30 days
Rejected 7 days
Failed validation 24 hours

Temp Uploads

  • Delete after 1 hour if not completed
  • Lifecycle rule on /temp/ prefix

Implementation

# R2 Lifecycle Rules
rules:
  - prefix: "temp/"
    expiration_days: 1

  - prefix: "packages/"
    tags:
      status: draft
    expiration_days: 30

Backup Strategy

R2 Built-in

  • 11 nines durability
  • Automatic replication within region
  • No additional backup needed for packages

Metadata Backup

  • Package metadata in PostgreSQL
  • PostgreSQL backups cover this
  • Can regenerate URLs from DB

Monitoring

Metrics to Track

Metric Source
Upload success rate API logs
Download latency Cloudflare analytics
Storage usage R2 dashboard
Bandwidth R2 dashboard
Error rate API logs

Alerts

  • Upload failure rate > 5%
  • Download error rate > 1%
  • Storage > 80% of budget

Security

Access Control

Packages:
  - Write: API only (presigned URLs)
  - Read: Public (presigned URLs)

Assets:
  - Write: API only
  - Read: Public (direct CDN)

Temp:
  - Write: Presigned URLs (15 min)
  - Read: None (API internal only)

Signed URLs

// Presigned URL with expiration
url := r2.PresignPut(key, 15*time.Minute, PutOptions{
    ContentType:   "application/octet-stream",
    ContentLength: maxSize,
})

Deliverables

  • R2 bucket setup
  • Presigned URL generation
  • Upload flow implementation
  • Download URL generation
  • Icon/screenshot upload
  • Lifecycle rules for cleanup
  • Monitoring dashboard
  • Cost tracking

Open Questions

  1. Multi-region storage for lower latency?
  2. Package compression (gzip/brotli)?
  3. Delta updates storage structure?
  4. Screenshot requirements (dimensions, count)?

References

Reference in New Issue View Git Blame Copy Permalink