Usage

The agent optimize→measure loop — "did the fix work, what changed" — is a single flag. -b / --baseline diffs <file.jfr> (the after) against a baseline (the before) and names what moved, instead of making you eyeball two summaries:

It diffs the totals (incl. total allocation bytes — the absolute memory anchor), hot paths, allocation sites, locks, and every extended section. Each row is anchored on its absolute weight (bytes / ms / samples) with the direction (▲/▼) and the share change as secondary context, ranked by the size of the absolute change; NEW / GONE are called out, and rows that barely moved drop. Anchoring on absolute matters: in an optimize loop the total shrinks, so a site whose absolute bytes fell can show a rising share — share alone would mislabel a real win as a regression. -a/-x scoping applies to both recordings.

When a fix extracts a helper method, one allocation drop splits across two rows (the caller falls, a NEW callee appears) — which can read like a regression at a glance. The diff adds an Allocation by type (rollup) block that sums sites by their declaring type, so the net is one line (e.g. GoFmt.* — 7.6 GB → 5.8 GB (▼ 24%) [2 methods]); it shows only types with ≥2 contributing methods and a real net change.

Optimizing a JVM library is almost always a JMH loop, and JMH’s -prof jfr writes one .jfr per fork (often all named profile.jfr in per-benchmark subdirs). Point analyze (and --baseline) at a directory and jvmlens finds every .jfr under it and merges the forks into one summary, so the signal isn’t split:

(JMH records the whole fork including warm-up; for measurement-only signal, configure JMH to delay the recording past warm-up, compare steady-state runs, or pass --skip-warmup <ms> to analyze — it drops the first <ms> of each recording, measured per file from that file’s earliest event, so each fork’s own warm-up is trimmed and hot paths reflect steady state.)

Or skip the separate step entirely with the JMH profiler plugin — it records the fork and prints the summary inline when the trial ends. Put jvmlens-jmh.jar (a tiny engine+profiler jar, no Spring/picocli/jmh) on the benchmark’s classpath and:

appPackage (+-separate several) scopes to your code; report focuses the summary. An unknown option key is a hard error with a did-you-mean (no silent misconfiguration). Socket I/O events are dropped by default (on a JMH fork the only socket traffic is the harness control socket — a rotating localhost:<port> that would otherwise show up as External I/O and skew the correlation); pass socketio=true to keep them. If the run also enables JMH’s GC profiler (-prof gc), the summary is headed with JMH’s exact gc.alloc.rate.norm (bytes/op) next to jvmlens’s sampled per-site bytes, so the measured rate and the attribution sit together. The plugin runs in JMH’s host process and reuses the same engine as analyze.

For a before→after diff entirely inside JMH — no separate analyze — keep the fork’s recording with keep=<path> (so it can seed the next run’s baseline) and diff against a prior one with baseline=<prev.jfr> (the profiler prints the change report instead of the summary):

Add JMH’s GC profiler (-prof gc) to both runs and the baseline= run prints a measured allocation A/B verdict above the sampled diff — JMH’s exact gc.alloc.rate.norm before→after with a significance call (the change is SIGNIFICANT only if the relative Δ exceeds the combined noise band and the confidence intervals don’t overlap), and a ⚠ when the sampled total Δ disagrees with the measured one (JIT elision / sampling redistribution). Use -f 2+ so the cross-fork variance is real — a single fork badly understates it. This gates the optimize loop on measured significance instead of a sampled delta that may be a phantom.

The same verdict adds a dispersion line: a genuine structural allocation removal doesn’t just lower the mean, it eliminates the GC-sampling variance that was that allocation, so the cross-fork band collapses (±17,200 → ±35 /op, variance collapsed ~500× is a strong real-win signal the mean alone can’t give). And once bytes/op goes near-deterministic it prints a stop signal — further allocation tuning has diminishing returns; the residual is intrinsic floor, pivot off allocation. The diff also hedges a CPU hot-path ▲ row whose absolute samples rose while the total exec samples fell (possible sampling redistribution — total samples fell N%), so a share that climbed only because the rest of the profile shrank isn’t misread as a regression.

A measured throughput A/B verdict sits next to the allocation one — the CPU analog — gating on JMH’s exact primary score (ops/s, us/op, …) with the same significance test. Its job is to stop a sampled hot-path swing being read as a speedup: if a hot-path share moved a lot but wall-clock throughput is flat, it says so explicitly (the sampled hot-path share moved (executeArguments 37%→18%) but throughput is flat — a CPU-share shift is not a speedup; the win is in allocation/GC). Because the inline baseline= matches a single benchmark method, a baseline JFR recorded for a different method is detected and the measured verdict is skipped with a warning rather than emitting confident wrong numbers — record a per-benchmark baseline.

The CPU hot-path diff applies the same redistribution hedge as allocation: a ▲ row whose absolute samples rose while the total fell, rose while the total stayed ~flat, or outpaced a modestly-rising total (its share climbed) is annotated (possible sampling redistribution …), because under a fixed-duration capture a now-faster workload runs more iterations — an unchanged frame then accrues more samples and gains share without its per-op cost changing. Whenever hot-path shares shift more than the total moved, the diff notes that absolute exec-sample deltas conflate per-op cost with throughput and points to a fixed-iteration bench A/B for a clean per-op comparison.

--hints appends a hedged ## Likely fix directions [possible] section that maps recognized hot-frame / allocation shapes to a one-line direction, each grounded in the row that triggered it — e.g. DoubleToDecimal/formatUnsignedInt → "number→string formatting", a LinkedList$ListItr.<init> self-time → "per-iteration iterator allocation", AbstractStringBuilder.ensureCapacity → "presize the buffer", and per-call regex compilation (java.util.regex.Pattern.* leaves, or a String.replaceAll(String,…)/matches/split source line) → "hoist the Pattern to a static final field and reuse Matcher`". It is off by default (the report stays clean data) and every line is tagged `[possible] — a direction to investigate, never an assertion.

The semantic dimensions feed --hints too, each with its own mechanical lever, grounded in the app call-site the row is anchored to (· at <Class>:line): db — N+1 → batch (WHERE id IN (…) / @BatchSize), SELECT * → project columns, and repeated single-row insert/update → enable JDBC batching; web — a high error-rate endpoint → validate/handle first; cache — a low hit rate → check key cardinality / TTL; messaging — a synchronous per-message send → batch or send async; mongo — a repeated find → Filters.in, repeated insertOne → insertMany/bulkWrite; redis — many single-key reads → pipeline / MGET.

Every hot-leaf and allocation-site row already carries a file:line anchor (e.g. :129). Point --source at your source root(s) and jvmlens reads each anchored line and appends its text inline — floatString:129 ⟶ mantissa.substring(0, dot) + mantissa.substring(dot + 1) — so a reader (or coding agent) sees the offending line without opening the file. Pass several roots comma-separated. It is off by default, reads files locally, and degrades silently: a file it can’t find is simply left un-annotated (it never fabricates).

--top-k <n> keeps only the top n rows per section; --max-tokens <n> shrinks top-k until the summary fits roughly n tokens (chars/4). For an always-on agent that adjusts limits at runtime, the agent control plane’s topn does the same per-dimension (see Runtime control).

By default every interval is emitted. Pass any threshold and watch instead stays quiet and emits only when a window breaches it — the production "dump-on-trigger" mode:

Like a desktop profiler’s live controls, the agent can be steered at runtime — without a restart — through a control file it watches (control=<file>). No ports, no JMX: an operator appends commands over whatever access they already have, via the jvmlens control CLI (run it on the host: kubectl exec, ssh, …):

Commands: start / stop, clear (reset the window + stores), dump (emit now), enable <dim> / disable <dim> (db/web/messaging/cache/mongo/redis/micrometer/snapshot/deadlock), settings <profile|default> (sampling density), interval <seconds>, scope app|exclude <prefix> / scope reset (filtering), topn [<category>] <n> / topn reset (rows per section — category is cpu/perf, memory/mem, locks, io, pinning, or a plugin like db/web), and status. Each command makes the agent publish its state to <control-file>.status, which the CLI reads back and prints — so topn db 5 returns the resulting limits to you.

Launching paused and then start after warm-up is the clean fix for short cold runs profiling startup rather than the workload — no more guessing a --warmup duration.

out is overwritten each interval — only the latest window survives. For a multi-day watch, add history=<file.jsonl> and the agent instead appends one compact sample per interval (covering all three dimensions — CPU, memory, wait), so nothing is lost:

Let it run, then trend reduces the accumulated run to a change-over-time report — what moved across the days, not a single snapshot:

The digest reports each dimension’s first-third→last-third direction (rising / flat / falling), whether the hot path stayed stable or shifted, when lock contention appeared, and a hedged retention indicator — old-object growth alongside rising GC pressure is flagged as possible retention growth, never a confident "leak".

When the window spans several JVM lifetimes — a multi-day monitor across rolling redeploys — trend detects each restart from the large gap it leaves between sample timestamps (the agent appends on a fixed interval, so a replaced pod leaves a hole far larger than one interval) and adds a ## Lifecycle note. Each lifetime’s first window is a cold-start burst (classload, warmup, JIT — a 5–10× outlier that would otherwise skew the whole-window trend to "rising"), so it is excluded from the steady-state aggregates, and retention is assessed within the latest lifetime only (old-object counts reset on restart). A gapless single-lifetime run is unaffected.

Add db to the agent options and it instruments java.sql.Statement.execute* (ByteBuddy) to time JDBC calls, aggregating them by sanitized SQL shape (literals parameterized, so no values reach the summary) into a Top SQL (by total time) section — each shape with its call count, average latency, the app call-site it was issued from (· at <Repo>:line), and a shape-aware hint: a high-count low-latency select is a possible N+1 (unless it already uses WHERE id IN (…), the batch-fetch fix — then it isn’t flagged), while a repeated single-row insert/update reads as likely un-batched (enable JDBC batching):

The SQL comes from the statement argument (plain Statement) or the statement’s toString() (most PreparedStatement drivers — H2, PostgreSQL); unknown shapes degrade to ?. This is profiling signal, not a query log — it never records literal values.

Add web and the agent instruments HttpServlet.service (ByteBuddy; both jakarta.servlet and javax.servlet, read reflectively so jvmlens needs no servlet dependency — Spring MVC’s DispatcherServlet is covered by this one point). Requests aggregate by route shape (numeric / UUID / long-token path segments become {}, query strings dropped) into a Top HTTP endpoints (by total time) section, each with request count, average latency, and an error count (status ≥ 400):

Add messaging and the agent times each broker at its lowest client level — Kafka Producer/Consumer (send/poll), JMS MessageProducer/MessageConsumer (send/ receive, both jakarta.jms and javax.jms), and RabbitMQ com.rabbitmq.client.Channel (basicPublish/basicGet) — matched by interface name, so jvmlens needs no broker dependency. Hooking the client (not the Spring template above it — KafkaTemplate/ JmsTemplate/RabbitTemplate) means a Spring app is instrumented once, never double-counted. ActiveMQ (Classic on javax.jms, Artemis on jakarta.jms) is covered via the JMS path. A high-volume per-message publish with material latency is flagged a synchronous per-message send (batch or send async).

Add mongo and the agent instruments the sync driver’s com.mongodb.client.MongoCollection operations (find/aggregate/insert*/update*/delete*/…, matched by name, no Mongo dependency) into a MongoDB operations (by total time) section, each anchored to its app call-site. The document analogues of the SQL N+1 levers: a repeated find/aggregate reads as a possible N+1 document fetch (batch with Filters.in(…) / an aggregation), a repeated single-document insertOne/updateOne/deleteOne as un-batched (use insertMany/ bulkWrite). Note find/aggregate return lazy iterables, so their count is the N+1 signal, not their latency.

Add redis and the agent times direct Redis commands — Lettuce (RedisAsyncCommands, which the sync API delegates to) and Jedis (JedisCommands), matched by name — into a Redis commands (by total time) section, anchored to the app call-site. This is distinct from the cache dimension (which tracks Spring Cache hit rate); redis is for apps hitting Redis directly (session stores, rate limiters, queues). A high count of single-key reads (get/hget/…) is flagged possible N+1 round-trips — pipeline them or use a multi-key command (MGET/HMGET).

Beyond performance, the agent can answer correctness questions — what values flow through a method — without stopping the app. Add snapshot=Class#method (semicolon-separate several) and the agent instruments those methods and appends a Variable snapshots section to the summary: per call site, the call count and a per-argument digest (distinct values, null rate, numeric range):

Method arguments need no debug info; capturing locals (requires -g) and conditions/PII redaction are planned.