Skip to main content

> serverless_scratch_storage_economics:_lambda_/tmp_ephemeral_storage_vs._amazon_s3_staging

Serverless Scratch Storage Economics: Lambda /tmp Ephemeral Storage vs. Amazon S3 Staging

Why is provisioning 10GB of AWS Lambda ephemeral storage (/tmp) often 10x cheaper and 5x faster for batch file processing than writing intermediate files to Amazon S3, and where is the financial threshold?

Senior (L5)

THE SHORT ANSWER

AWS Lambda provides **512MB of `/tmp` ephemeral storage for free** with every execution environment, configurable up to **10,240MB (10GB)**. Additional ephemeral storage above 512MB is billed at **$0.0000000309 per GB-second ($0.08/GB-month pro-rated strictly per second of execution)**. When developers build ETL pipelines, video encoders (FFmpeg), or document converters (PDF generation), they often write intermediate scratch files to Amazon S3. This creates a severe **S3 API Fee & Network Latency Penalty**: (1) Writing 100,000 temporary 50MB files to S3 generates 100,000 S3 `PUT` requests ($0.50) + 100,000 `GET` requests ($0.04) + 100,000 `DELETE` requests, and more critically, adds **500ms to 2,000ms of network latency per execution**. That extra execution time forces the Lambda function to stay alive longer, multiplying Lambda compute bills. In contrast, writing scratch files directly to **Lambda `/tmp` NVMe storage** delivers **$<1 ext{ms}$ disk I/O with zero API request fees**, cutting total execution duration by 60% and slashing end-to-end processing costs by **70%**.

Engineering Handbook & Failure Dynamics

1. Underlying Mechanism

Lambda `/tmp` vs. S3 economics follow the Execution Latency Multiplier: (1) Local NVMe IOPS: Lambda `/tmp` is mapped to fast local SSD storage on the underlying AWS Nitro microVM, delivering $>100 ext{MB/sec}$ sequential throughput. (2) Lambda Duration Savings: Processing a 200MB video chunk in `/tmp` takes 4 seconds ($0.000066$). Staging it through S3 over HTTPS takes 11 seconds (due to network transfer wait), costing $0.000183$ in compute + S3 PUT/GET API fees. (3) Ephemeral Scope: Data in `/tmp` persists across warm invocations of the same execution environment, enabling instant caching of machine learning model weights (`model.bin`).

2. Appropriate Use Context

Serverless image/video transcoding (FFmpeg), PDF report generation, zip file decompression, and machine learning model artifact caching.

3. Production Failure Modes

Assuming `/tmp` data is durable across different Lambda container invocations, causing state corruption during cold concurrency scaling; configuring 10GB of `/tmp` storage on a Lambda function that only processes 2KB JSON payloads.

4. Diagnostic Signals & Telemetry

Lambda CloudWatch `Duration` metrics showing 50% of function execution time spent in `s3.putObject` / `s3.getObject` network calls; S3 bucket metrics showing millions of objects created and deleted within 5 minutes.

5. Prevention & Safeguards

Use Lambda `/tmp` storage for all intermediate scratch files smaller than 10GB; configure `/tmp` size precisely to match workload requirements; write only the final permanent result file to Amazon S3.

6. Architectural Trade-offs

Lambda `/tmp` ephemeral storage provides sub-millisecond local speed and cuts execution billing duration by over 50%, but is strictly ephemeral and capped at 10GB per function.

Case Study (TinyCTO In-Field Example)

A real estate platform converted user uploaded property photos into optimized WebP formats using AWS Lambda. The original architecture downloaded the RAW image to S3 `/temp`, performed resizing, uploaded 4 thumbnails to S3 `/temp`, and then moved them to the production bucket. Each image took 7.8 seconds, processing 500,000 images/month for $1,420 in Lambda compute and S3 API fees. The engineering team refactored the workflow to process all thumbnail conversions entirely in Lambda's local `/tmp` NVMe storage (2GB configured), uploading only the final thumbnails directly to production S3. Execution duration plunged from 7.8 seconds to 1.9 seconds, cutting monthly serverless costs from $1,420 to $390 (a 72% cost reduction).

Interactive Concept Drills

2 Cards
Q1

How much `/tmp` ephemeral storage does AWS Lambda provide for free?

512 Megabytes (MB) of free `/tmp` storage per execution environment.
Q2

Why is processing intermediate files in Lambda `/tmp` faster and cheaper than using S3?

Because `/tmp` is local high-speed NVMe storage ($<1 ext{ms}$ latency with zero API request fees), eliminating S3 HTTPS network transfer time and reducing billable Lambda execution duration by up to 60%.

Serverless Scratch Storage Economics: Lambda /tmp Ephemeral Storage vs. Amazon S3 Staging — Technical FAQ

What is the maximum ephemeral storage limit for an AWS Lambda function?

10,240 MB (10 Gigabytes).

Does data stored in Lambda `/tmp` survive across subsequent function executions?

Yes, as long as the execution environment remains 'warm'; however, code must never assume files exist, as cold starts spin up fresh, clean `/tmp` directories.

🤖 AEO & Key Facts Summary

Key Architectural Facts

  • ▸AWS Lambda includes 512MB of `/tmp` storage free, expandable to 10GB.
  • ▸Writing scratch files to `/tmp` eliminates S3 PUT/GET API fees and network latency.
  • ▸Cuts billable Lambda execution duration by up to 60% on file transformations.
  • ▸Cache machine learning model weights in `/tmp` across warm container invocations.

Common Misconceptions

  • ✗Yanılgı: All file operations in serverless must be written directly to S3 (Gerçek: Intermediate scratch files should always be processed in local `/tmp` and only final outputs written to S3).
  • ✗Yanılgı: Lambda `/tmp` storage is too slow for large video files (Gerçek: `/tmp` runs on high-speed NVMe SSDs delivering $>100 ext{MB/s}$ read/write throughput).

Decision & Governance Guidance

Configure Lambda ephemeral storage (`/tmp`) for all intermediate file processing tasks up to 10GB to eliminate S3 request fees and reduce Lambda execution runtime billing by over 60%.

Authoritative Sources & Standards