The Challenge
High-resolution media processing is a resource hog. Scaling a platform that handles concurrent 4K video transcoding and massive PDF compression requires more than just fast code - it requires surgical resource management and an architecture that doesn't melt the CPU under load.
Technical Implementation
I wrote Lattice in Rust because media processing is heavily CPU-bound, requiring predictable performance and streaming uploads. I built everything from authentication and billing to object storage, keeping memory usage low. The system orchestrates external tools like FFmpeg and Ghostscript for specialized file formats.
#
Processing Engine
- Image Optimization: Resizing, compression, and conversion to WebP and AVIF with EXIF metadata stripping.•Video Transcoding: Web-optimized transcoding (VP9, H.264, H.265, AV1) with configurable bitrates and resolutions.
•PDF Compression: High-fidelity compression using four quality presets (screen, ebook, printer, prepress) via Ghostscript.
•HTML Conversion: Capturing URLs or raw HTML as PDF, DOCX, or PNG screenshots using a Playwright-based pipeline.
#
Platform Architecture
- Rust & Axum: for async execution and extremely low memory footprint.•MinIO & S3-compatible storage: for isolated per-account file storage with secure, presigned URL delivery.
•HMAC-Verified Webhooks: with exponential backoff for reliable event notifications.
#
Auth & Billing
- Scoped API Keys: for REST endpoints with rate-limiting and usage tracking.•Stripe Integration: for usage-based billing, automated checkout, and subscription lifecycle management.
Key Features
•Asynchronous Upload Streaming: handle files up to 100MB without loading them entirely into memory.
•Native Ecosystem Plugins: built-in support for WordPress and Drupal media library optimization.
•Automated Delivery: capability to automatically push processed files to any S3-compatible bucket or SFTP server.
•Interactive API Docs: live-rendered documentation for seamless developer onboarding.
By the numbers
•90% size reduction for PDF compression.
•100MB maximum file size limit.
•100ms average processing time for standard payloads.