# Milestone 7: CDN & Storage

**Status**: Planning
**Goal**: Scalable storage for app packages and assets with global distribution.

---

## Overview

Storage handles app packages, icons, screenshots, and serves downloads to devices worldwide. Must be cost-effective, fast, and reliable.

---

## Requirements

### Functional

- Store app packages (.mosis files)
- Store icons (multiple sizes)
- Store screenshots (optional)
- Serve downloads globally
- Support presigned upload URLs
- Version retention (keep old versions)

### Non-Functional

| Requirement | Target |
|-------------|--------|
| Upload speed | < 30s for 50MB |
| Download latency | < 100ms (p50) |
| Availability | 99.9% |
| Durability | 99.999999999% (11 nines) |
| Cost | Minimize egress fees |

---

## Storage Structure

```
/packages/
  /{developer_id}/
    /{app_id}/
      /{version_code}/
        package.mosis
        manifest.json (extracted for quick access)

/assets/
  /{app_id}/
    icon-32.png
    icon-64.png
    icon-128.png
    screenshots/
      1.png
      2.png
      3.png

/temp/
  /{upload_id}/
    package.mosis (pending validation)
```

---

## Options Analysis

### Option A: Cloudflare R2

```
Storage:    Object storage (S3-compatible)
CDN:        Cloudflare network (automatic)
Egress:     FREE (zero egress fees)
Pricing:    $0.015/GB storage, $4.50/million requests
```

| Pros | Cons |
|------|------|
| No egress fees | Newer service |
| Global CDN included | Fewer regions than S3 |
| S3-compatible API | Less tooling |
| Workers integration | |

#### Cost Estimate (10K apps, 100GB)

| Component | Monthly Cost |
|-----------|--------------|
| Storage (100GB) | $1.50 |
| Requests (1M) | $4.50 |
| Egress | $0 |
| **Total** | **~$6** |

---

### Option B: AWS S3 + CloudFront

```
Storage:    S3 Standard
CDN:        CloudFront
Egress:     $0.085-0.12/GB (varies by region)
Pricing:    $0.023/GB storage
```

| Pros | Cons |
|------|------|
| Most mature | Egress costs add up |
| Best tooling | Complex pricing |
| All regions | Need CloudFront config |
| IAM integration | |

#### Cost Estimate (10K apps, 100GB, 1TB egress)

| Component | Monthly Cost |
|-----------|--------------|
| Storage (100GB) | $2.30 |
| CloudFront (1TB) | $85 |
| Requests | ~$5 |
| **Total** | **~$92** |

---

### Option C: Backblaze B2 + Cloudflare

```
Storage:    Backblaze B2
CDN:        Cloudflare (free egress via Bandwidth Alliance)
Egress:     FREE (through Cloudflare)
Pricing:    $0.005/GB storage
```

| Pros | Cons |
|------|------|
| Cheapest storage | Two services to manage |
| Free egress via CF | B2 API slightly different |
| Good reliability | Need CF proxy setup |

#### Cost Estimate (10K apps, 100GB)

| Component | Monthly Cost |
|-----------|--------------|
| Storage (100GB) | $0.50 |
| Egress (via CF) | $0 |
| Requests | ~$0.40 |
| **Total** | **~$1** |

---

### Option D: Self-hosted MinIO

```
Storage:    MinIO on VPS
CDN:        Cloudflare proxy
Egress:     VPS bandwidth
Pricing:    VPS cost only
```

| Pros | Cons |
|------|------|
| Full control | Ops overhead |
| S3-compatible | Need to manage |
| Predictable cost | Scaling complexity |

#### Cost Estimate

| Component | Monthly Cost |
|-----------|--------------|
| VPS (500GB SSD) | $20-50 |
| Cloudflare | $0 (free tier) |
| **Total** | **~$20-50** |

---

## Recommendation

**Primary: Cloudflare R2**
- Zero egress fees (biggest cost saver)
- S3-compatible (easy migration)
- Built-in global CDN
- Good enough tooling

**Fallback: Backblaze B2 + Cloudflare**
- If R2 has issues
- Even cheaper storage
- Slightly more setup

---

## Upload Flow

### Presigned URL Approach

```
┌────────┐    ┌─────────┐    ┌─────────┐
│ Client │───►│   API   │───►│   R2    │
└────────┘    └─────────┘    └─────────┘
     │              │              │
     │  1. Request upload URL      │
     │◄─────────────┤              │
     │  (presigned URL)            │
     │                             │
     │  2. Upload directly         │
     │────────────────────────────►│
     │                             │
     │  3. Notify complete         │
     │─────────────►│              │
     │              │  4. Validate │
     │              │─────────────►│
     │              │              │
     │  5. Confirm  │              │
     │◄─────────────┤              │
```

### API Implementation

```go
// 1. Request upload URL
func CreateVersion(c *gin.Context) {
    // Create version record
    version := Version{
        AppID:       appID,
        VersionCode: req.VersionCode,
        Status:      "uploading",
    }
    db.Create(&version)

    // Generate presigned URL
    key := fmt.Sprintf("temp/%s/package.mosis", version.ID)
    url, err := r2.PresignPut(key, 15*time.Minute)

    c.JSON(200, gin.H{
        "version":    version,
        "upload_url": url,
        "expires":    time.Now().Add(15 * time.Minute),
    })
}

// 3. Upload complete notification
func UploadComplete(c *gin.Context) {
    // Move from temp to final location
    tempKey := fmt.Sprintf("temp/%s/package.mosis", versionID)
    finalKey := fmt.Sprintf("packages/%s/%s/%d/package.mosis",
        developerID, appID, versionCode)

    r2.Copy(tempKey, finalKey)
    r2.Delete(tempKey)

    // Update version status
    version.Status = "validating"
    version.PackageURL = finalKey

    // Trigger async validation
    queue.Publish("validate-package", version.ID)
}
```

---

## Download Flow

### Public Downloads

```go
// Get download URL (short-lived)
func GetDownloadURL(c *gin.Context) {
    version := getLatestPublishedVersion(packageID)

    // Generate presigned download URL (1 hour)
    url, _ := r2.PresignGet(version.PackageURL, 1*time.Hour)

    c.JSON(200, gin.H{
        "download_url": url,
        "version":      version.VersionName,
        "size":         version.PackageSize,
        "signature":    version.Signature,
    })
}
```

### CDN Caching

```
Cache-Control: public, max-age=86400
```

- Packages are immutable (version code = unique)
- Cache aggressively at edge
- Invalidate only if package is pulled

---

## Icon/Screenshot Handling

### Upload

```go
// Icons uploaded with app creation/update
func UploadIcon(c *gin.Context) {
    file, _ := c.FormFile("icon")

    // Validate dimensions
    img, _ := png.Decode(file)
    if img.Bounds().Dx() != img.Bounds().Dy() {
        return error("Icon must be square")
    }

    // Generate multiple sizes
    sizes := []int{32, 64, 128}
    for _, size := range sizes {
        resized := resize(img, size, size)
        key := fmt.Sprintf("assets/%s/icon-%d.png", appID, size)
        r2.Put(key, resized)
    }
}
```

### Serving

```
https://cdn.mosis.dev/assets/{app_id}/icon-64.png
```

- Public read access
- Long cache TTL (icons rarely change)
- Cloudflare image optimization (optional)

---

## Retention Policy

### Packages

| Status | Retention |
|--------|-----------|
| Published (current) | Forever |
| Published (old) | 1 year |
| Draft | 30 days |
| Rejected | 7 days |
| Failed validation | 24 hours |

### Temp Uploads

- Delete after 1 hour if not completed
- Lifecycle rule on `/temp/` prefix

### Implementation

```yaml
# R2 Lifecycle Rules
rules:
  - prefix: "temp/"
    expiration_days: 1

  - prefix: "packages/"
    tags:
      status: draft
    expiration_days: 30
```

---

## Backup Strategy

### R2 Built-in

- 11 nines durability
- Automatic replication within region
- No additional backup needed for packages

### Metadata Backup

- Package metadata in PostgreSQL
- PostgreSQL backups cover this
- Can regenerate URLs from DB

---

## Monitoring

### Metrics to Track

| Metric | Source |
|--------|--------|
| Upload success rate | API logs |
| Download latency | Cloudflare analytics |
| Storage usage | R2 dashboard |
| Bandwidth | R2 dashboard |
| Error rate | API logs |

### Alerts

- Upload failure rate > 5%
- Download error rate > 1%
- Storage > 80% of budget

---

## Security

### Access Control

```
Packages:
  - Write: API only (presigned URLs)
  - Read: Public (presigned URLs)

Assets:
  - Write: API only
  - Read: Public (direct CDN)

Temp:
  - Write: Presigned URLs (15 min)
  - Read: None (API internal only)
```

### Signed URLs

```go
// Presigned URL with expiration
url := r2.PresignPut(key, 15*time.Minute, PutOptions{
    ContentType:   "application/octet-stream",
    ContentLength: maxSize,
})
```

---

## Deliverables

- [ ] R2 bucket setup
- [ ] Presigned URL generation
- [ ] Upload flow implementation
- [ ] Download URL generation
- [ ] Icon/screenshot upload
- [ ] Lifecycle rules for cleanup
- [ ] Monitoring dashboard
- [ ] Cost tracking

---

## Open Questions

1. Multi-region storage for lower latency?
2. Package compression (gzip/brotli)?
3. Delta updates storage structure?
4. Screenshot requirements (dimensions, count)?

---

## References

- [Cloudflare R2 Documentation](https://developers.cloudflare.com/r2/)
- [S3 Presigned URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html)
- [Backblaze B2 + Cloudflare](https://www.backblaze.com/b2/docs/cloud_to_cloud.html)