Signed URL File Upload API for Conversations in Sparkforge

Signed URL File Upload streams files directly from the client to Google Cloud Storage (GCS) using secure, time-limited URLs. This eliminates server-side memory pressure, speeds up transfers, and supports larger media for messaging.

What it does

Instead of routing file bytes through application servers, the client uploads to a short-lived signed URL issued by Sparkforge. The architecture reduces out-of-memory risk, speeds transfers, and validates content at multiple stages.

Key benefits

• Stability: files never buffer in server memory, eliminating OOM crashes.
• Larger uploads: up to 100MB for WhatsApp, up to 5MB on other channels.
• Performance: direct client-to-GCS transfer reduces latency.
• Reliability: URLs expire after 15 minutes with clear error messages.
• Security: path validation, access control, and content-type verification.

Three-step upload flow

1. Initiate — POST /conversations/messages/upload/initiate. Send file metadata to receive a signed URL and object path.
2. Upload — Client PUTs the file directly to the signed URL on GCS before it expires.
3. Complete — POST /conversations/messages/upload/complete. Sparkforge verifies the stored object and returns the public URL for use in messages.

Channel size limits

• WhatsApp: up to 100MB.
• All other channels: up to 5MB.
• Oversized files are rejected at Initiate before any bytes transfer.

Content-type validation

MIME types are checked at two stages to block spoofed or mismatched files.

• On Initiate: validates the declared contentType matches the file extension.
• On Complete: verifies the stored contentType in GCS matches the filename.

Security guards

• Confirms the conversation exists and belongs to the specified location.
• Verifies the user has access to the location.
• Validates the object path matches the expected format.
• Signed URLs expire after 15 minutes.

Error handling

• URL expired: request a fresh URL by repeating Initiate.
• Oversized file: shrink the file or switch to a channel with a higher limit.
• Content-type mismatch: ensure filename extension and Content-Type agree.
• Permission/path errors: confirm location access and conversation IDs.

Implementation tips

• Initiate close to the actual upload time to maximize the 15-minute window.
• Always set an explicit Content-Type on the PUT request.
• Log the object path returned from Initiate for troubleshooting.
• For large WhatsApp videos, re-initiate instead of retrying near expiry.
• Validate size and MIME client-side before Initiate so users see errors fast.