Benchmarks

Bounded qualification reports and the reproducibility contract.

gsuite versus DMU: 50,000-animal pedigree

Compare REML estimates, predictions and observed run times for gsuite 0.29.0 and DMU 5.6 on the same frozen pedigree and observations.

Grouped bars of wall time in seconds: single trait, gsuite 80.9 and DMU 178.4; multitrait, gsuite 105.6 and DMU 818.5; longitudinal, gsuite 386.9 and DMU 1182.7.

Observed runtime for the three pedigree models. gsuite fit versus the DMU preprocessing and fitting pipeline; lower is faster. Timings are single executions.

gsuite recorded lower wall times than DMU in all three comparisons, alongside close agreement in covariance estimates and predictions. These results show competitive runtime for the tested direct pedigree REML workflows.

Benchmark results Model
Single-trait results Pedigree animal model
Multitrait results Two traits with incomplete observations
Longitudinal results Rank-2 random regression

View all comparison results and saved tables.

The study checks numerical agreement on three frozen models. Timings are single executions, not a repeated performance benchmark or a general speed claim. The public result page includes saved tables and selected provenance; the full development report and execution scripts are not published here.

Other validation evidence

Additional development campaigns cover pedigree solvers, qgdata, genotype preparation and sparse LD. Their full reports are not included in the public snapshot. See the evidence and qualification limits for the scope of available evidence and unresolved failures.

Reporting standards

Each published benchmark should report:

  • library and version;
  • data dimensions;
  • hardware;
  • compiler and build type;
  • thread count;
  • elapsed time;
  • peak memory;
  • numerical agreement or tolerance, where relevant;
  • the exact command or script used.

Intended areas

  • packed genotype operations;
  • genomic matrix and sparse-LD operations;
  • numerical solvers;
  • ML/REML inference;
  • LD-score regression, where relevant;
  • genomic scoring.

Benchmark reports should distinguish method validation from performance qualification and should make comparisons reproducible from the information provided.