---
title: "Kubernetes API and Microservice Load Testing: Ingress, Timeouts, and HPA Signals"
description: "Kubernetes load testing: ingress timeouts, HPA, mesh overhead—k6 against real cluster hostnames with Kubernetes.io networking context."
publishDate: "2026-09-14"
draft: false
keyword: "kubernetes load testing"
intent: "MOFU"
cta: "Use Performate to turn this playbook into runnable k6 scenarios, thresholds, and shareable reports without losing days to glue code."
tags: ["k6","load-testing","api-performance","kubernetes"]
---

Your microservices pass k6 against `localhost:8080`—then production misses SLOs because ingress idle timeouts clip long polls, HPA adds pods sixty seconds after the queue backs up, and the mesh sidecar adds twelve milliseconds you never modeled. Kubernetes shifts failure modes; load tests must target **customer-visible hostnames** and correlate ramps with **cluster signals**, not pod IPs in a dev laptop port-forward.

[Kubernetes](https://kubernetes.io/docs/home/) adds layers—Ingress or Gateway API, Services, Endpoints, HPAs, and optional service mesh sidecars—each with timeouts, connection limits, and scaling lag. k6 remains the execution engine; the discipline is testing through the same front door users hit while watching `kubectl top`, HPA events, and saturation metrics alongside [`http_req_duration`](https://k6.io/docs/using-k6/metrics/).

In this guide you will learn which K8s-specific bottlenecks dominate API latency stories, how to structure k6 scenarios against ingress hostnames, and which observables should block a release before you scale traffic into shared staging clusters.

## Why Kubernetes changes performance—not just deployment

Functionally, your API behaves the same in a pod as on a VM. Under load through the cluster path, different limits appear:

- **Ingress and gateway timeouts** (`proxy-read-timeout`, `proxy-send-timeout`) terminate requests that outlive upstream slowness—k6 sees 502/504 bursts, not gradual latency growth.
- **Connection reuse and keep-alive** through L7 load balancers differ from direct pod tests—TLS handshakes and SNIs add cost at scale.
- **HPA lag** scales on CPU or custom metrics minutes after queue depth spikes—ramps that jump instantly hide autoscaling debt ([capacity planning](/en/blog/capacity-planning-from-k6-results)).
- **Service mesh sidecars** add CPU, memory, and hop latency—compare tagged runs with `mesh:on` vs `mesh:off` ([observability tags](/en/blog/k6-observability-metrics-tags)).
- **Noisy neighbors and limits** on shared staging nodes skew results if you never record node pool and resource requests during the test.

Think of pod-local k6 as testing an engine on a lift; ingress-targeted k6 tests the whole car on the highway—including toll booths and on-ramps.

### When in-cluster smoke passes but ingress fails at peak

A port-forward to one pod proves container health. It skips DNS, TLS termination, WAF rules, and cross-AZ routing. Teams discovered this gap when [queue/worker pipelines](/en/blog/queue-worker-async-pipeline-load-testing) looked healthy at the API while ingress returned 504s during fan-out—because the test never exceeded the controller's connection table.

Pair ingress tests with [mock vs real dependencies](/en/blog/mock-services-vs-real-dependencies-load-tests) consciously: mocks hide downstream saturation that HPA cannot fix.

## Practical k6 implementation: ingress hostnames and ramp discipline

Target the external hostname and TLS config customers use. Ramp gradually so HPA and cluster autoscaler have time to react—unless you explicitly test cold-start failure modes. Below is an illustrative script; adapt hostnames, paths, and SLO numbers to your environment.

**What this example demonstrates:**

- **External BASE URL:** hits ingress hostname, not `127.0.0.1:8080`.
- **Gradual ramp:** `ramping-arrival-rate` exposes HPA lag vs instant spikes.
- **Mesh comparison tags:** `mesh:enabled` env toggles headers or routes for A/B runs.
- **Separate error and tail thresholds:** ingress 502 bursts fail the run before average latency looks acceptable.

```javascript
import http from 'k6/http';
import { check, sleep } from 'k6';

const BASE = __ENV.API_BASE || 'https://api.staging.example.com';
const MESH = __ENV.MESH_ENABLED === 'true' ? 'on' : 'off';

export const options = {
  scenarios: {
    ingress_ramp: {
      executor: 'ramping-arrival-rate',
      startRate: 5,
      timeUnit: '1s',
      preAllocatedVUs: 30,
      maxVUs: 200,
      stages: [
        { duration: '3m', target: 30 },
        { duration: '10m', target: 30 },
        { duration: '2m', target: 0 },
      ],
      tags: { route: 'search', mesh: MESH, target: 'ingress' },
      exec: 'search',
    },
  },
  thresholds: {
    'http_req_duration{route:search}': ['p(95)<500', 'p(99)<900'],
    'http_req_failed{route:search}': ['rate<0.02'],
  },
};

export function search() {
  const res = http.get(`${BASE}/v1/search?q=load-test`, {
    headers: {
      Authorization: `Bearer ${__ENV.TOKEN}`,
      'X-Mesh-Enabled': MESH === 'on' ? 'true' : 'false',
    },
    tags: { route: 'search', mesh: MESH, target: 'ingress' },
    timeout: '120s',
  });
  check(res, {
    'search not gateway timeout': (r) => r.status !== 504 && r.status !== 502,
    'search 2xx': (r) => r.status >= 200 && r.status < 300,
  });
  sleep(0.2);
}
```

**Patterns that work**

- **Match ingress timeout annotations** to k6 `timeout` values—know whether slowness or proxy kill fired.
- **Watch HPA during steady stage:** `kubectl get hpa -w` and correlate scale events with k6 RPS.
- **Record node pool, pod count, and CPU throttling** in the test runbook—not just k6 summaries.
- **Run mesh on/off pairs** at identical RPS before blaming application code ([HTTP/2 and HTTP/3 tuning](/en/blog/http2-http3-load-testing-latency)).

**Anti-patterns to avoid**

- Load-testing pod IPs while production traffic uses ingress—misleading SLO sign-off.
- Instant max RPS ramps on autoscaling workloads without documenting intentional cruelty.
- Ignoring staging cluster quotas—saturating shared ingress hurts neighboring teams.
- Single aggregate latency threshold when 502 bursts hide behind fast 200s from cache hits.

**Pro tip (example command):** tag runs with cluster context for postmortems that cite identical conditions.

```bash
k6 run ingress-search.js --tag cluster=staging-eu1 --tag git_sha=$CI_COMMIT_SHA --summary-export=summary.json
```

**What this command demonstrates:** exported summaries tie k6 results to cluster metadata so regressions compare the same ingress, mesh mode, and release.

## Decision framework: what to test through Kubernetes layers

| Situation | Recommended action |
|:---|:---|
| New service before prod cutover | Ramp through ingress hostname; watch HPA and pod count |
| 502/504 spikes in prod | Re-run with k6 timeout ≥ ingress `proxy-read-timeout`; capture gateway logs |
| Mesh rollout | Identical RPS with `mesh:on/off` tags; compare p99 delta |
| Worker/queue backed APIs | Pair API ramp with [async pipeline tests](/en/blog/queue-worker-async-pipeline-load-testing) |
| Staging ≠ prod topology | Document gaps; avoid prod SLO sign-off from undersized staging only |
| Cold-start sensitivity | Step ramp vs instant spike scenarios—explicitly labeled |

**Test through ingress if** customers never touch pod networks directly—which is almost always.

**Test HPA reaction if** autoscaling metrics drive your capacity story—hold steady RPS long enough for scale-out.

**Test mesh variants if** sidecars ship mTLS and retry policies that change tail latency.

## Observability, documentation, and next steps

Kubernetes load tests fail postmortems when cluster state was not captured. Before you scale traffic:

- [ ] Document ingress hostname, TLS cert source, timeout annotations, and WAF rules in the runbook.
- [ ] Record start/end pod counts, HPA min/max, and CPU/memory limits for the target Deployment.
- [ ] Correlate k6 tags with APM and service mesh dashboards during the steady stage.
- [ ] Snapshot `kubectl top pods` and node pressure at peak RPS.
- [ ] Archive k6 summary JSON with cluster name, mesh mode, and git SHA per release.

## How Performate simplifies Kubernetes-targeted load tests

Teams lose days wiring auth, hostnames, and export metadata every time ingress changes. Below is a **concrete workflow example** for the search API through staging ingress.

**Example: ingress-realistic search scenario without port-forward hacks**

1. **Import OpenAPI or Postman** for `GET /v1/search` with production-like query params. *Problem solved:* URLs and headers match what ingress routes—not localhost shortcuts.
2. **Set `API_BASE` to the staging ingress hostname** (`https://api.staging.example.com`) in the environment panel. *Problem solved:* TLS and DNS paths match customer traffic.
3. **Create a ramping-arrival-rate scenario**—5→30 req/s over fifteen minutes. *Problem solved:* visual ramp tuning while you watch HPA in another terminal.
4. **Apply tags** `route:search`, `target:ingress`, and `mesh:on/off` from env. *Problem solved:* reports isolate mesh overhead debates with data, not anecdotes.
5. **Set thresholds** on `p99` and gateway error rate (`502/504` checks in script). *Problem solved:* release gates catch proxy kills average latency hides.
6. **Export k6 script and summary** with scenario metadata for the postmortem folder. *Problem solved:* cluster snapshots and traffic mixes cite the same artifact.

That workflow maps directly to the `cta` in this post: runnable scenarios and shareable reports without losing days to glue code.

## Closing takeaway

Kubernetes load testing is an **ingress, timeout, and autoscaling** problem—not a pod health check. Target external hostnames, ramp honestly, tag mesh modes, and correlate k6 ramps with HPA events before you sign off on capacity.

Run this week's search ramp through staging ingress—not port-forward—and note whether latency tails or gateway errors appear first when pods scale.

[Try Performate free](https://performate.app) | [Book a demo](/demo) | [k6 running in Kubernetes](https://k6.io/docs/misc/fine-tuning-os/)