---
title: "Soak Tests and Long Runs: Using AI to Spot Patterns in Noisy Time Series"
description: "Long-running k6 soak tests produce noisy charts; use AI to highlight drift and anomalies—then verify with engineering judgment and infra metrics."
publishDate: "2026-10-20"
draft: false
keyword: "ai soak test analysis"
intent: "MOFU"
cta: "Use Performate’s desktop workflow—imports, k6 runs, and AI-assisted analysis where your plan allows—to ship faster without skipping validation."
tags: ["k6","load-testing","ai","performate","soak","test"]
---

Your soak run finished at 3 a.m. with a flat error rate and a `p95` line that barely moved—yet production still melted down on Tuesday from a connection pool that never released handles. Humans glaze over six hours of nearly identical charts; models can flag **slope changes** and **periodic teeth** in structured time series—if you export the right buckets and refuse hand-wavy summaries.

**AI soak test analysis** is not a replacement for heap dumps or query traces. It is a triage accelerator: turn long k6 evidence into candidate windows, then prove causes with infra metrics and controlled follow-up runs. In this guide you will learn what to export from soak tests, how to script k6 for interval-friendly metrics, which patterns models hunt well (and where they hallucinate), and how to close the loop with engineering tickets.

## Why soak failures hide in noise—not spikes

Short load tests catch saturation and obvious misconfiguration. Soak tests catch **slow modes**: memory creep, pool churn, log-induced CPU, cache inefficiency, and replication lag that only appears after hours of steady traffic.

Under a long run, symptoms look boring:

- **Gradual `p99` lift** with flat `http_req_failed`—queueing, GC, or fragmentation, not “the API is down.”
- **Sawtooth latency** aligned with cron, cache eviction, or batch jobs on shared staging.
- **Error bursts** that match connection pool timeouts or broker backpressure minutes after traffic looked healthy.

Pair soak discipline with the [soak testing playbook](/en/blog/soak-testing-playbook-memory-leaks) for duration and abort rules. Use AI commentary ([Gemini load test analysis](/en/blog/gemini-ai-load-test-analysis)) only on **aggregated intervals**—models need timestamps and values, not vague screenshots.

### Define success before you start the clock

Document upfront: max error rate, max `p99` growth vs hour-one baseline (or vs steady-state start at T+X), and when to abort so shared staging is not collateral damage. Hour one is often distorted by JIT warmups and cache fills—either exclude warm-up from SLO evaluation or mark “steady state starts at T+45m” on the run sheet.

## Practical k6 implementation: soak shape, buckets, and drift tags

Soak scripts should hold **arrival rate** steady (not ramp-to-failure) and emit metrics you can bucket for AI and dashboards.

**Example script (illustrative—not production-ready).** Uses fictional URLs and SLO numbers. Adapt duration, rates, and thresholds to your risk profile.

**What this example demonstrates:**

- **Long steady load** with `constant-arrival-rate` so VU count does not mask pool exhaustion.
- **Route tags** on every request for per-endpoint drift in summaries.
- **Custom trend metric** `soak_drift_ms` sampled each iteration—optional hook for external aggregators; primary analysis still uses built-in `http_req_duration` percentiles per interval export.
- **Strict error ceiling** plus relaxed duration thresholds suited to multi-hour runs (tighten for smoke soaks).

```javascript
import http from 'k6/http';
import { check, sleep, Trend } from 'k6';

const BASE = __ENV.API_BASE || 'https://staging.example.com';
const soakDrift = new Trend('soak_drift_ms', true);

export const options = {
  scenarios: {
    soak_checkout: {
      executor: 'constant-arrival-rate',
      rate: Number(__ENV.SOAK_RPS || 8),
      timeUnit: '1s',
      duration: __ENV.SOAK_DURATION || '4h',
      preAllocatedVUs: 15,
      maxVUs: 60,
      tags: { scenario: 'soak', route: 'checkout' },
      exec: 'checkout',
    },
  },
  thresholds: {
    http_req_failed: ['rate<0.005'],
    // Soak SLO: tail may drift—gate on errors first; compare p99 buckets offline
    'http_req_duration{route:checkout}': ['p(99)<2000'],
  },
};

export function checkout() {
  const start = Date.now();
  const res = http.post(`${BASE}/v2/checkout`, JSON.stringify({ sku: 'SKU-100', qty: 1 }), {
    headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${__ENV.TOKEN}` },
    tags: { route: 'checkout', scenario: 'soak' },
  });
  soakDrift.add(Date.now() - start);
  check(res, { 'checkout 2xx': (r) => r.status >= 200 && r.status < 300 });
  sleep(Number(__ENV.THINK_SEC || 0.5));
}
```

**Patterns that work**

- Export **5-minute buckets**: requests, errors, `p95`/`p99`, active VUs—small enough for AI, large enough to smooth noise.
- Attach **infra overlays** when policy allows: heap, GC time, DB connections—soak truth is rarely client-only.
- Annotate **deployment markers** (cache warm, config reload) in the run sheet so models do not blame your service for someone else’s batch job.
- Compare **same-duration windows** across builds with `build_id` columns in CSV exports.

**Anti-patterns to avoid**

- Feeding raw per-request CSVs with millions of rows—aggregate first, drill second.
- Treating a single spike on shared staging as your regression without checking diurnal noise ([staging vs production load tests](/en/blog/staging-vs-production-load-tests)).
- Accepting AI text that says “degradation seems present” without cited timestamps and metric values.

**Pro tip (example command):** surface percentile trends in the summary for post-soak review.

```bash
k6 run soak-checkout.js --summary-trend-stats="p(95),p(99),avg" -o soak-summary.json
```

**What this command demonstrates:** JSON plus percentile stats give you a portable artifact to bucket externally or paste into bounded AI prompts alongside interval CSVs.

## Decision framework: duration, exports, and AI vs manual review

| Situation | Recommended action |
|:---|:---|
| Suspected memory leak or handle leak | 4–8h soak; heap/GC overlays; compare hour-4 vs hour-1 `p99` buckets |
| Connection pool churn | 90–120m soak at realistic RPS; watch error bursts + pool metrics |
| Shared staging with overnight ETL | Segment intervals or annotate cron; do not let AI blame your API for ETL spikes |
| First soak for a new scenario | Establish steady-state start time; archive hour-1 as reference only |
| AI summary for leadership | Require cited windows; open engineering tickets per flagged interval |

**Use AI triage if** you have structured interval exports and a human owner who will run heap dumps, thread dumps, or query traces on flagged windows.

**Use manual-only review if** the run is short, the environment is too noisy, or you lack infra overlays—models will overfit to spikes.

**Abort early if** error rate crosses a red line or saturation breaks unrelated teams—soaks are learning tools, not denial-of-service drills.

## Observability, documentation, and next steps

Before the next overnight soak:

- [ ] Document duration rationale beside the scenario name (why 4h vs 8h).
- [ ] Export 5-minute aggregates with `build_id`, git SHA, and environment fingerprint.
- [ ] Define steady-state start (exclude warm-up from SLO comparison).
- [ ] Post start/stop times to ops channels on shared clusters.
- [ ] Turn each AI-flagged window into a ticket: metric snippet, suspected subsystem, next experiment.
- [ ] Overlay last green soak vs candidate for the same window before merging perf fixes.

When long runs expose database lag, cross-check [read replicas and eventual consistency load testing](/en/blog/read-replicas-eventual-consistency-load-testing).

## How Performate simplifies soak review with AI assistance

Long runs generate large artifacts; desktop workflow keeps imports, execution, and commentary in one place.

**Example: four-hour checkout soak with AI-assisted pattern review**

1. **Import the Postman collection** (or OpenAPI) for the checkout path you will soak. *Problem solved:* one source of truth for headers and payloads across reruns.
2. **Create a `constant-arrival-rate` scenario** at production-like RPS (e.g. 8 req/s) and set duration to 4h in the visual editor. *Problem solved:* no hand-editing executor blocks when product changes duration.
3. **Apply tags** `scenario:soak` and `route:checkout` on the scenario. *Problem solved:* reports slice latency the same way as the k6 example above.
4. **Run the soak locally or against staging** and capture the integrated report export. *Problem solved:* comparable summaries without juggling three spreadsheets.
5. **Where your plan allows, use AI-assisted analysis** on bounded summaries—require timestamps and values in the narrative, reject vague degradation language. *Problem solved:* faster candidate windows; you still verify with infra metrics.
6. **Export the k6 script** for CI nightly soaks at reduced rate/duration with the same tags and thresholds.

That workflow matches the post `cta`: ship faster on post-soak review without skipping validation or root-cause proof.

## Closing takeaway

**AI soak test analysis** pairs disciplined k6 soaks with structured exports—models highlight candidate drift; engineers prove causes. Flat `p95` lines can hide climbing tails; feed interval buckets, infra overlays, and build labels, then verify every AI-flagged window with experiments—not opinions.

Schedule this week’s soak with documented abort rules and steady-state boundaries—note which metric bucket diverged first from your last green run.

[Try Performate free](https://performate.app) | [Book a demo](/demo) | [k6 scenarios](https://k6.io/docs/using-k6/scenarios/)