---
title: "Baseline vs Regression: Building a Repeatable Performance Benchmark for APIs"
description: "Lock baselines with identical scenarios, detect regressions with k6 thresholds, and compare builds without noisy apples-to-oranges charts."
publishDate: "2026-07-23"
draft: false
keyword: "api performance baseline"
intent: "MOFU"
cta: "Use Performate to store comparable runs and threshold outcomes between releases."
tags: ["k6","load-testing","api-performance","performance","baseline"]
---

Release 412 “looked fine” in QA until `p99` checkout crept 180ms over the envelope your team agreed on in March—because nobody reran the **same** k6 scenario with the **same** data and tags. A **baseline** is the accepted latency and error envelope for a scenario under a known environment contract. A **regression** is meaningful drift from that envelope after a change. Without frozen scenario DNA, you compare fiction.

In this guide you will learn what to lock in a baseline bundle, how to encode gates with k6 [thresholds](https://k6.io/docs/using-k6/thresholds/), when to reset baselines after infra change, and how to store comparable runs so engineer turnover does not erase memory.

## Why baselines fail before thresholds do

Teams often copy a script, tweak VUs “to be safe,” and wonder why nightly charts disagree. Small drift invalidates comparisons:

- **Executor mismatch** (`constant-vus` vs arrival-rate) changes who waits in queues.
- **Payload or auth drift** alters CPU and payload size without a code change in the service under test.
- **Environment fingerprint change** (DB snapshot, cache warm, feature flags) shifts tails independently of your merge.

Industry practice ties service levels to SLIs users feel ([Google SRE workbook](https://sre.google/workbook/implementing-slos/)); k6 encodes them as tagged thresholds on [`http_req_duration`](https://k6.io/docs/using-k6/metrics/) and `http_req_failed`. Review [common load testing mistakes](/en/blog/common-load-testing-mistakes) before you trust a red threshold—and wire smoke gates into [load testing in CI/CD](/en/blog/load-testing-in-ci-cd) once the baseline is stable.

Pair baseline discipline with [k6 thresholds examples](/en/blog/k6-thresholds-examples) when you add routes or dependencies.

## Practical k6 implementation: frozen DNA, build tags, and gates

Treat the baseline script as a versioned artifact: scenario parameters, env vars, datasets, and threshold lines travel together.

**Example script (illustrative—not production-ready).** Fictional checkout API and envelope numbers.

**What this example demonstrates:**

- **Env-driven fingerprint** `BUILD_ID` on tags for report comparison across CI runs.
- **Single golden scenario** with constant arrival rate—stable request pressure week to week.
- **Thresholds** that fail the run when tails or errors breach the recorded baseline envelope.
- **Abort-friendly duration** for nightly smoke; extend duration for quarterly re-baselining.

```javascript
import http from 'k6/http';
import { check, sleep } from 'k6';

const BASE = __ENV.API_BASE || 'https://staging.example.com';
const BUILD = __ENV.BUILD_ID || 'local';
const body = JSON.stringify({ sku: 'SKU-100', qty: 1, currency: 'USD' });

export const options = {
  scenarios: {
    baseline_checkout: {
      executor: 'constant-arrival-rate',
      rate: Number(__ENV.BASELINE_RPS || 25),
      timeUnit: '1s',
      duration: __ENV.BASELINE_DURATION || '10m',
      preAllocatedVUs: 25,
      maxVUs: 100,
      tags: { route: 'checkout', profile: 'baseline' },
      exec: 'checkout',
    },
  },
  thresholds: {
    // Envelope captured from build 408 green run — adjust only via explicit baseline renewal
    'http_req_duration{route:checkout}': ['p(95)<750', 'p(99)<1100'],
    http_req_failed: ['rate<0.008'],
  },
};

export function checkout() {
  const res = http.post(`${BASE}/v2/checkout`, body, {
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${__ENV.TOKEN}`,
    },
    tags: { route: 'checkout', build: BUILD, profile: 'baseline' },
  });
  check(res, { 'checkout 2xx': (r) => r.status >= 200 && r.status < 300 });
  sleep(0.35);
}
```

**Patterns that work**

- Store raw summary JSON + git SHA + env fingerprint beside every green baseline capture.
- Fail merges on threshold breach; use softer `p95` delta review on weeklies with enough sample size—short smokes are noisy.
- Tag `build` on requests so Performate or k6 Cloud comparisons align runs ([k6 observability metrics and tags](/en/blog/k6-observability-metrics-tags)).
- Document **baseline renewal** in the changelog when infra or intentional latency trades move the envelope.

**Anti-patterns to avoid**

- Changing RPS and duration in the same week you compare builds—pick one dimension per experiment.
- Resetting thresholds silently after a failed run “to go green.”
- Comparing staging baseline to production traffic shape without annotation.

**Pro tip (example command):** export JSON for archival and diff tooling.

```bash
k6 run baseline-checkout.js -e BUILD_ID=$CI_COMMIT_SHA -o baseline-summary.json
```

**What this command demonstrates:** machine-readable output attaches to the build record so regressions are diffable, not screenshot debates.

## Decision framework: baseline capture vs regression gate vs reset

| Situation | Recommended action |
|:---|:---|
| New critical API route in baseline journey | Extend script; re-capture envelope; update thresholds in one PR |
| Weekly release candidate | Same scenario DNA; fail CI on threshold breach |
| Infra upgrade (DB major, cache topology) | Schedule baseline renewal; annotate dashboards with effective date |
| Intentional latency trade (heavier validation) | Product sign-off + new envelope; do not silently widen thresholds |
| Noisy short smoke in CI | Lower duration/RPS but keep tags and relative thresholds; avoid changing both |

**Capture a new baseline if** you changed scenario DNA intentionally and stakeholders accept a new envelope.

**Treat as regression if** DNA is unchanged and tagged tails or errors cross thresholds—investigate before widening gates.

**Reset dashboards only after** explicit renewal notes—otherwise on-call chases ghosts from March numbers on July infra.

## Observability, documentation, and next steps

Before the next release train:

- [ ] Freeze scenario DNA document: executor, duration, RPS, payload mix, think time, dataset version.
- [ ] Record environment fingerprint (DB snapshot id, feature flags, cache state procedure).
- [ ] Archive green summary JSON with git SHA when baselines are captured.
- [ ] Wire threshold failure to merge blocking in CI for the baseline scenario.
- [ ] Compare `p95`/`p99` deltas between nightly runs with minimum sample rules.
- [ ] Log baseline renewal events with owner and reason when infra or product changes the envelope.

## How Performate keeps baselines comparable across releases

Scenario definitions scattered across laptops do not survive turnover. Centralize the golden path and exports.

**Example: checkout baseline tracked per build**

1. **Import the golden Postman flow** (or OpenAPI operation) used for the March baseline capture. *Problem solved:* one definition for local, CI, and review meetings.
2. **Set constant arrival rate** to the recorded `BASELINE_RPS` (e.g. 25 req/s) and duration to 10m in the editor. *Problem solved:* scenario DNA lives in one workspace, not five forks.
3. **Apply tags** `route:checkout`, `profile:baseline`, and pass `BUILD_ID` from CI env in exported scripts. *Problem solved:* comparison views filter the same dimensions every run.
4. **Enter thresholds** matching the archived envelope (`p95`, `p99`, error rate). *Problem solved:* visual editing of gates without drift between hand-edited script copies.
5. **Run on each release candidate** and open historical comparison in the integrated report. *Problem solved:* product and engineering debate one export per build.
6. **Export k6** for pipeline smoke at reduced duration/RPS with **unchanged tags and threshold expressions**—scale time, not shape.

That workflow delivers the post `cta`: store comparable runs and threshold outcomes between releases without apples-to-oranges charts.

## Closing takeaway

**API performance baseline** discipline is frozen scenario DNA plus honest thresholds. **Regression** detection is only trustworthy when traffic shape, data, tags, and environment fingerprint match the captured green run—then let k6 fail the build when tails move.

Re-run your golden checkout scenario on the current candidate with `BUILD_ID` set—note whether you need a fix or a documented baseline renewal.

[Try Performate free](https://performate.app) | [Book a demo](/demo) | [k6 results output](https://k6.io/docs/getting-started/results-output/)