JMeter Works in GUI Fails in CLI

Script works in GUI, fails in CLI

Symptom

• Green samples in the GUI with a few threads

• Same .jmx fails under:

  jmeter -n -t plan.jmx -l results.jtl

• Failures often show missing files, wrong host, plugin/class errors, or connection errors only on the CI agent

Official stance: use GUI to build and debug; use non-GUI for real load (best practices). CLI is required in CI/CD and containers.

Common causes

Cause	Detail
Working directory	Relative CSV, keystore, or payload paths resolve from different cwd
Properties	GUI used defaults or GUI-set props; CLI missing -J / -q
Plugins missing	CI image or server install lacks lib/ext JARs (plugins)
Headless network	Agent cannot reach SUT (ConnectException)
Different user.properties	Local custom props not present on the agent
Output paths	-o report dir already exists / not writable
Java version skew	Different JVM behaviour or TLS defaults

Fix (ordered)

1. Run CLI on the same machine as the successful GUI run first:

   cd /path/to/plan-dir
   jmeter -n -t plan.jmx -l results.jtl -j jmeter.log -e -o report/

2. Open jmeter.log for file-not-found, class-not-found, or SSL lines.

3. Make CSV and resource paths stable (relative to plan location or absolute). CSV Data Set is the documented multi-user pattern (best practices).

4. Parameterize hosts and threads with `\${__P(name,default)}` and pass the same -J flags you expect in CI (functions).

5. Mirror plugins into the CLI environment; restart after installs.

6. In Docker/K8s, mount plans and outputs explicitly (Docker guide); use qainsights/jmeter or your pinned image consistently.

7. Only after local CLI is green, run the same command in CI with the same properties and network access.

Related tools and topics

Resource	Use when
CI/CD load testing	Pipeline patterns
Plugins essentials	Missing elements in CLI
ConnectException	Network only in CI
Properties cheat sheet	Env differences

Frequently asked questions

Is GUI success enough to ship a plan?

No. Always prove non-GUI once. Production and CI never use the GUI for load.

Why relative CSV paths break in CLI?

GUI often starts with a cwd near the plan or last open file. CLI cwd is wherever you launched the process. Prefer plan-relative layout and document the launch directory.

Does -n change sampler behaviour?

Same engine, different resource profile (no GUI listeners). Functional failures are usually path/property/plugin/network differences, not -n itself.

Continue Learning

→

Next Practical Step

From the plan directory, run jmeter -n once with -j jmeter.log and fix the first path or property error in that log.

📖

Related Reference

• Best practices
• CI/CD
• Functions and variables