From a scientific question to a supported model
An assistant can help identify a missing interface, component combination or numerical capability, implement it and assemble validation evidence.
Code that runs becomes a supported model only after its statistical meaning and numerical behavior have been checked.
This tutorial illustrates the development workflow through example requests. It does not implement or qualify a new model. It complements analysis with an assistant, which describes working with existing gsuite interfaces.
1. Begin with an analysis that needs more
You might ask:
I have repeated body-weight measurements. I want to distinguish additive genetic variation from persistent environmental differences between animals. Can my current gsuite workflow express that model? Check the existing machinery before proposing an implementation.
A candidate repeatability model is
\[ y = X\beta + Z_a a + Z_p p + e, \]
with mutually independent random components
\[ \operatorname{Var}(a)=\sigma_a^2 A,\qquad \operatorname{Var}(p)=\sigma_p^2 I,\qquad \operatorname{Var}(e)=\sigma_e^2 I. \]
Here \(a\) is the additive genetic animal effect, \(p\) is a permanent-environment animal effect shared across that animal’s observations, and \(e\) is the record-specific residual. The fixed design \(X\) must also reflect the study, including time effects where appropriate. This model does not by itself describe changing genetic trajectories or residual serial correlation.
This model already has an explicit gvc() workflow: the repeatability example uses separate grouping columns for animal and permanent-environment effects. It records the optimizer tolerance needed for that dataset and the stricter stopping limitation. The model catalog also lists specialized and component routes. For this request, use the existing workflow; the development steps below apply when a further requirement reveals a gap.
The assistant first inspects the installed interface, implementation and evidence. If an existing supported route meets the request, the next step is an analysis script using that route. Development begins only for an identified gap, such as exposing an existing composition through a particular interface.
2. Classify the missing piece
You might ask:
Explain exactly what is missing. Is this an R-interface change, a combination of existing components, or new numerical work? Identify what can be reused.
| Gap | Typical work | Evidence needed |
|---|---|---|
| Missing R interface | Add validation and translate R inputs into an existing model contract | Equivalent inputs produce the same model and results as the established route |
| Missing composition | Connect existing relationships, covariance blocks and record mappings | The assembled covariance and fitted results agree with an independent small reference |
| Missing numerical capability | Implement an operator, derivative, inference step or solver capability in its owning library | Mathematical checks, numerical reference comparisons and tests of the supported parameter domain |
The qgtools-style specification separates formulas, record identities, relationship kernels, covariance descriptors and the fitting task. For the example, the two animal effects share grouping IDs but have different relationships: pedigree for genetic effects and identity for permanent environment. The internal representation must preserve that distinction.
The architecture guide identifies ownership: R validation and integration belong in gsuite, mixed-model inference in greml, and numerical solvers in gsolve. Genomic data operations remain with their existing owners. The primitive registry and model reductions help identify reusable machinery.
3. Agree on the statistical contract
Before writing code, the assistant records what the model means and which cases the proposed extension will support.
For the repeatability example, this includes the definition of an observation, the animal ID mapping, pedigree coverage, repeated-record handling, fixed effects, covariance assumptions and prediction targets. It also includes missing-data behavior, starting values and supported variance boundaries.
The study must contain information to distinguish the components. For example, with one observation per animal, a permanent-environment identity effect and an independent residual cannot be separated by their covariance contributions. A more general syntax does not resolve identifiability.
You might ask:
Write down the model assumptions, input contract and supported domain. Explain which components the data can distinguish, and identify unsupported cases before implementing the extension.
4. Implement a small, reusable extension
You might ask:
Implement the identified gap using existing public interfaces. Preserve current behavior, keep the change within the owning libraries, and make unsupported combinations fail clearly. Include focused validation.
Develop a reviewable patch. Where possible, translate new R syntax into an existing model description and reuse its fitting algorithm.
Keep the first supported scope explicit. Missingness, covariance boundaries, prediction and uncertainty each need their own contract; successful fitting does not automatically establish support for all of them.
5. Establish statistical and numerical evidence
For a small deterministic repeatability example, independently assemble
\[ V = \sigma_a^2 Z_a A Z_a^{\mathsf T} + \sigma_p^2 Z_p Z_p^{\mathsf T} + \sigma_e^2 I. \]
This dense calculation is a small reference, not the production execution strategy. It lets the assistant check the model independently of the new interface or assembly code.
| Check | What it establishes |
|---|---|
| Covariance and record mapping at fixed parameters | The implementation represents the stated statistical model |
| Likelihood and fixed-covariance predictions against the small reference | Numerical calculations agree under matched conventions |
| Scores and information, where applicable | Optimization quantities match independent calculations; derivative checks use appropriate interior points |
| REML estimates and convergence against a trusted route | The supported fitting path reaches a consistent solution within declared tolerances |
| Reordered IDs, repeated records and supported missingness | Input representation does not silently change the model |
| Invalid inputs, unidentified designs and unsupported boundaries | Failures are explicit and do not silently substitute another model |
| Existing affected workflows | The extension preserves supported behavior |
Tolerances and pass criteria should be chosen before interpreting results. Likelihood comparisons must reconcile constants and parameter conventions. Simulation is useful, but a fitted estimate need not equal its generating parameter in a finite sample. Agreement between two wrappers over the same engine checks integration; it is not an independent numerical reference.
You might ask:
Show me the validation evidence, including failures and untested cases. Separate evidence for model correctness, numerical accuracy and performance.
Larger workloads and repeated timings are a separate qualification step. A successful small example establishes only its tested scope; it does not establish production-scale performance or support on other platforms.
6. Make support explicit and reproducible
The extension becomes a supported workflow only when its declared statistical contract and relevant numerical checks are satisfied. Update the model catalog and qualification evidence for the exact model, backend and inference combination, recording remaining limitations. Follow the repository’s required build and package checks as well as focused model tests.
The completed work should contain a reviewable implementation, deterministic reference checks, updated interface documentation, and a small tutorial users can run. Record the versions used for qualification. An implemented but unqualified route must retain that distinction in its documentation.
You might ask:
Summarize what is now supported, which checks passed, what remains outside scope, and how another user can reproduce the example.
Grow capabilities through scientific use
Prefer extensions that other analyses can reuse: covariance structures, relationship mappings and inference capabilities. Scientific review and validation determine which resulting workflows gsuite can support.