Performance

RefKit's benchmark measures public bibliography operations with fixed inputs and checked outputs. Choose the capability that controls your workload: parsing, entry lookup, BibTeX edits, citation rendering, formatting, or Polars batch execution.

Measure a capability

The runner lives in packages/refkit-bench in the source repository. Follow the benchmark setup and methodology to install the participants and build release-mode adapters.

From that checkout:

uv run --no-sync refkit-bench check --lane parse.bibtex --dataset real
uv run --no-sync refkit-bench run --lane parse.bibtex --dataset real \
  --output packages/refkit-bench/results/baseline
uv run --no-sync refkit-bench report packages/refkit-bench/results/baseline

This case parses the same 12-record bibliography through RefKit, bibtexparser, and Pybtex. Validation checks complete metadata. Timings cover in-memory model creation, with validation outside the timer.

Read the evidence

pyperf, a Python benchmarking tool, calibrates batches and collects repeated values in separate workers. A result directory contains:

• manifest.json: input and artifact hashes, source provenance, setup boundaries, validation outcomes, runtime settings, and host identity.
• timings.json: measured values, calibration, warmups, and worker metadata in pyperf's format.

The summary reports the median of worker means for each case. Compare matched runs with:

uv run --no-sync refkit-bench compare \
  packages/refkit-bench/results/baseline \
  packages/refkit-bench/results/candidate

The ratio is candidate time divided by baseline time. Values below 1 mean lower elapsed time. With enough independent workers, the report includes exploratory per-case bootstrap intervals. Repeat small effects in alternating baseline/candidate sessions on the same idle machine before attributing a gain to an implementation change.

Match the workflow

Parsing, rendering, formatting, and dataframe execution have different units and setup costs. Compare within a lane and dataset. The shared rendering fixture checks exact basic author-date output. Polars batch citation cases use the plugin's bundled APA style. The formatter corpus checks exact expectations from a pinned bibtex-tidy revision and records differences from the published package.

The methodology guide lists every lane, its timed boundary, corpus coverage, known conformance differences, and commands for scaling experiments. A failed correctness check prevents that selected run from producing a timing comparison.