Serverless Cold Starts and Concurrency Limits: A Load Testing Checklist

Serverless looks elastic until cold starts and concurrency caps turn a traffic step into 503s. Average latency charts hide bimodal distributions: warm invocations at 80ms, cold at 2s, throttled at error. Load tests must include idle-then-burst patterns and respect account-scoped limits (AWS Lambda concurrency).

This checklist covers what to test beyond flat RPS, a k6 ramp that simulates quiet-then-flash-sale traffic, and how to distinguish throttle responses from cold latency in reports. Pair with spike vs load literacy when product describes "sudden traffic."

What to test beyond average latency

Serverless surfaces differ from always-on containers:

• Cold start after idle window—first requests after quiet period pay init cost (runtime, VPC ENI, secret fetch).
• Account/region concurrency saturation—reserved vs unreserved pools; burst limits per function.
• Provisioned vs on-demand behavior—provisioned concurrency removes cold path for N warm instances; test both modes if you use them.
• Downstream dependency timeouts amplified by slow invocations—API Gateway 504 may be symptom, not root cause.
• 429/503 throttle signatures distinct from 500 application errors—tag and threshold separately in analysis.

Think of serverless capacity as tokens and warm instances, not CPU percent on a fixed fleet.

Staging fidelity warnings

Staging functions often have higher memory, no VPC cold penalty, or different concurrency reservations. Document gaps like any staging vs prod test—serverless gaps are larger than usual.

k6: idle gap then spike

Illustrative—not production-ready. Tune stages to your traffic story; coordinate with cloud team on max concurrency before running 80 req/s against shared staging account.

What this demonstrates:

• ramping-arrival-rate with low initial stage simulates idle traffic, sharp ramp simulates flash sale or webhook burst.
• Tags surface:serverless segment metrics from containerized routes in same gateway.
• Relaxed error threshold on spike phase optional—document whether test seeks SLO proof or limit mapping.
• Short sleep keeps connection reuse realistic without zero think time (VU sanity).

import http from 'k6/http';
import { check, sleep } from 'k6';

const BASE = __ENV.API_BASE || 'https://staging.example.com';

export const options = {
  scenarios: {
    burst_after_quiet: {
      executor: 'ramping-arrival-rate',
      startRate: 1,
      timeUnit: '1s',
      preAllocatedVUs: 50,
      maxVUs: 200,
      stages: [
        { duration: '3m', target: 2 },
        { duration: '30s', target: 80 },
        { duration: '2m', target: 80 },
        { duration: '1m', target: 2 },
      ],
      tags: { surface: 'serverless' },
    },
  },
  thresholds: {
    'http_req_duration{surface:serverless}': ['p(95)<2000'],
    'http_req_failed{surface:serverless}': ['rate<0.05'],
  },
};

export default function () {
  const res = http.get(`${BASE}/fn/invoke`, { tags: { route: 'invoke', surface: 'serverless' } });
  check(res, {
    ok: (r) => r.status < 500,
    not_throttled: (r) => r.status !== 429 && r.status !== 503,
  });
  sleep(0.1);
}

Patterns that work

• Log cold vs warm when platform exposes x-cold-start, X-Amz-Executed-Version, or similar—custom Trend in k6 if header present.
• Coordinate with account limits—not unbounded VUs against shared staging payer account.
• Run spike during staffed window—abort if throttle rate exceeds agreed exploration bound.
• Compare provisioned on/off in two exports—leadership sees cost vs latency trade-off.

Anti-patterns to avoid

• Flat constant-arrival-rate only—never exercises cold path after idle.
• Treating all 503 as application bugs—may be concurrency throttle.
• Stressing production Lambda without change control and concurrency reservation plan.
• Ignoring downstream DB connection limits when functions scale out fast.

Pro tip (example command): break out status codes in summary for throttle analysis.

k6 run serverless-burst.js --summary-trend-stats="p(95),p(99)" --tag surface=serverless

What this command demonstrates: combine with post-run filter on http_req_duration where status=503 in your analysis tool—separates cold latency from hard throttle.

Decision table: pattern vs executor

Use spike-shaped ramp before marketing events—not another steady load rerun.

Use steady ARR only after warm pool confirmed—otherwise cold skews p95.

Observability and pre-run checklist

• Log cold vs warm if platform exposes diagnostic headers—note in report.
• Coordinate max concurrency with cloud team—document account/region limits.
• Compare spike vs load test type with product language.
• Separate throttle vs application 5xx in checks or post-processing.
• Note provisioned concurrency count and memory settings in report footer.
• Schedule during window when on-call can disable test via kill switch.

How Performate supports serverless load testing

Below is a concrete workflow example for invoke URL burst testing—adapt stages to your flash-sale profile.

Example: cold-start spike template

1. Import invoke URL from Postman—headers and auth preserved. Problem solved: no retyping API Gateway paths under launch pressure.
2. Build ramp stages in UI—3m quiet, 30s spike, 2m hold. Problem solved: visual stage editor reduces YAML mistakes in high-stress weeks.
3. Run burst on staging with cloud team on Slack. Problem solved: desktop abort faster than waiting for CI job cancel.
4. Filter errors by status in report—429/503 vs 500. Problem solved: capacity ticket gets throttle evidence, not generic "errors up."
5. Export for capacity ticket with stage diagram screenshot. Problem solved: finance/provisioned concurrency debate uses one artifact.
6. Save template for next launch—retag surface:serverless only. Problem solved: repeatability without rebuilding from memory.

That workflow maps directly to the cta in this post: runnable scenarios and shareable reports without glue-code panic before launch.

Closing takeaway

Serverless load testing is idle + burst + limits, not flat RPS. Schedule a cold-start spike before the next flash sale; tag throttles separately; document concurrency reservations in the same ticket as the k6 export.

Coordinate with your cloud team, run quiet-then-spike once on staging, and record where 429s begin—that number belongs in the launch runbook.

If your platform exposes provisioned concurrency, run the same script twice in one week—off vs on—and attach both exports to the capacity ticket so cost conversations use identical traffic shape.

Try Performate free | Book a demo | k6 executors