# compare: back-to-back time lies — so I git-isolated the benchmark

Source: https://www.michaelchurley.com/blog/compare-git-isolated-command-benchmarker  
Published: 2026-06-22  
Author: Michael C. Hurley  
Tags: compare, bash, cli, benchmark, posix-time, git-isolation, biome, oxlint, mdr, mermaid, devtools, michaelmonetized

![Git-isolated A/B workflow](/blog/compare-git-isolated-command-benchmarker/screenshots/workflow-isolation.png)

## Who

I kept running `time cmd-a; time cmd-b` and then arguing with myself about whether the second one won because it was faster, or because the first one warmed the cache and the tree had drifted.

For operators comparing formatters and linters who refuse that lie. Especially anyone already typing `vp exec biome format --write` next to `vp exec oxlint` and wanting the median, not the vibes.

## What

I built **compare**, public [michaelmonetized/compare](https://github.com/michaelmonetized/compare). Bash. `VERSION="0.1.0"`. `bin/compare` is **825** lines at HEAD (468 on init). `install.sh` drops it in `/usr/local/bin` (or `$PREFIX`). HEAD `c5dbcd6`. **3** commits. 0 stars. CLI only. No LICENSE. No release tag.

![CLI help surface](/blog/compare-git-isolated-command-benchmarker/screenshots/cli-help.png)

The product is git isolation for shell A/B:

1. If the working tree is dirty, commit `compare: snapshot before benchmark` (with `--no-verify`).
2. Force-create `compare/<slug-a>` and `compare/<slug-b>` from that shared SHA.
3. Checkout A, run command A `-n` times with `TIMEFORMAT`, append to the log; blank line; same for B.
4. Restore the original branch. Leave the snapshot and `compare/*` branches for inspection; delete them yourself.

Log lines look like POSIX `time`:

```text
vp exec biome lint --write  1.36s user 0.36s system 151% cpu 1.134 total
vp exec oxlint --write  0.21s user 0.18s system 73% cpu 0.528 total
```

Default path: `../tests/compare-<cwd>-<c1>-<c2>-<UTCstamp>.log`. Flags: `-n`, `-c`, `-o`, `-g/--graph`, `--md`. Subcommands: `compare graph <log>`, `compare report <log>`. Graphs: user / system / CPU% / total (blue A, yellow B). `--md` writes tables + mermaid and opens [mdr](https://github.com/CleverCloud/mdr) (else `open` / `xdg-open`). Commands go through `eval` (trusted only).

![Sample log + terminal charts](/blog/compare-git-isolated-command-benchmarker/screenshots/sample-log-graph.png)

Smoke on the pack box: `compare "sleep 0.05" "sleep 0.12" -n 3` totals ~0.051 vs ~0.121, branches `compare/sleep-0-05` and `compare/sleep-0-12` left behind. README caveats remain honest: no CPU pinning, mutating commands can diverge branches, one intentional variable is on you.

Residue: `plans/README.md` still says the repo had **no commits and no source code** (docs-only improve plans). Those four plans are marked DONE. `compare report` is in `--help` and code; it is not in the README Contents. Second nightly (`c5dbcd6`) shares the exact tree with the first nightly (empty HEAD commit).

![plans irony](/blog/compare-git-isolated-command-benchmarker/screenshots/plans-irony.png)

## Where

Code: [github.com/michaelmonetized/compare](https://github.com/michaelmonetized/compare), public, branch `main`. No homepage. No live web demo. Clone + `./install.sh` (or `PREFIX=$HOME/.local ./install.sh`).

Audience sits next to biome, oxlint, Vite Plus, and anyone who already treats `time` output as courtroom evidence.

## When

**2026-06-17, 6:57 AM Eastern.** `217d4ac` init: README, 468-line CLI, install script, four docs plans.

**2026-06-22, 5:40 PM Eastern.** `b4fea7b` nightly: CLI grows to 825 lines; README gains `--md` / mdr / CPU chart.

**2026-06-22, 6:17 PM Eastern.** `c5dbcd6` nightly: same tree as `b4fea7b`. HEAD. Empty.

![Commit arc](/blog/compare-git-isolated-command-benchmarker/screenshots/commit-arc.png)

## Why

Back-to-back `time` is a shared-state measurement pretending to be a tool measurement. A snapshot commit plus two named branches is the smallest honest isolator I would actually run. biome-vs-oxlint needs a log file outside the working tree, not another Slack debate.

**Engagement Q:** When you A/B two CLIs, what is the one variable you pretend you controlled, and which dirty-tree / warm-cache factor actually won?