Sample identity, table joins and missing assays

Practical 14: Genomics, systems biology and multi-omics

Learning objectives

Align assay columns to biological IDs; reject duplicate keys; distinguish absent assays from missing feature measurements; evaluate row-order errors.

Biological question and data

What happens when protein columns arrive in a different order from phenotype metadata?

Every fixture animal has an assay identifier. The shuffled column order is deliberate; the underlying biological labels remain correct. This checks metadata alignment, not physical sample identity.

No biological download is required. Numerical inputs are hypothetical, not observed animal data. The runnable examples use base R. Count and abundance units are specified where used.

1. Create a plausible order mismatch

Reverse the assay order while leaving labels attached. Dimensions still match.

helper_dir <- if (dir.exists("R")) "R" else "../R"
stopifnot(file.exists(file.path(helper_dir,"teaching_data.R")))
source(file.path(helper_dir,"teaching_data.R"))
d <- teaching_data()
meta <- d$metadata
shuffled <- d$protein[,rev(seq_len(ncol(d$protein))),drop=FALSE]
stopifnot(ncol(shuffled)==nrow(meta),!identical(colnames(shuffled),meta$animal))
head(data.frame(metadata_animal=meta$animal,assay_column=colnames(shuffled)))

  metadata_animal assay_column
1        animal01     animal48
2        animal02     animal47
3        animal03     animal46
4        animal04     animal45
5        animal05     animal44
6        animal06     animal43

A dimension check alone cannot identify this mistake.

2. Align explicitly and verify identity

Require unique IDs and complete matches before reordering.

stopifnot(!anyDuplicated(meta$animal),!anyDuplicated(colnames(shuffled)))
idx <- match(meta$animal,colnames(shuffled))
stopifnot(!anyNA(idx))
aligned <- shuffled[,idx,drop=FALSE]
stopifnot(identical(colnames(aligned),meta$animal),
 max(abs(aligned-d$protein))<1e-12)
c(correct_cor=cor(aligned[1,],meta$response),
 wrong_order_cor=cor(shuffled[1,],meta$response))

    correct_cor wrong_order_cor 
      0.1567479      -0.1811823 

The correctly aligned table reproduces the original measurements exactly. A wrong-order association can disappear or change direction.

3. Separate missing animals from missing features

Remove one entire assay column and compare this to NAs within otherwise present assays.

partial <- aligned[,-1,drop=FALSE]
index <- match(meta$animal,colnames(partial))
c(animals_without_assay=sum(is.na(index)),
 missing_feature_values=sum(is.na(d$protein_missing)))

 animals_without_assay missing_feature_values 
                     1                     11 

stopifnot(sum(is.na(index))==1,sum(is.na(d$protein_missing))==11)

An absent assay and an unobserved protein value need different handling and reporting. Neither is a measured zero.

4. Audit a many-to-many join

Assay metadata can have several specimens per animal. Joining two such tables on animal alone can create combinations that never represented matched specimens.

left <- data.frame(animal=c('a1','a1'),time=c(1,2),RNA=c(5,8))
right <- data.frame(animal=c('a1','a1'),time=c(1,2),protein=c(2,4))
wrong <- merge(left,right,by='animal')
right_join <- merge(left,right,by=c('animal','time'))
c(animal_only_rows=nrow(wrong),animal_time_rows=nrow(right_join))

animal_only_rows animal_time_rows 
               4                2 

stopifnot(nrow(wrong)==4,nrow(right_join)==2)

5. Define the biological matching key

Animal identity alone may be insufficient when tissue, collection time or specimen differs. Define the key for the intended comparison and preserve mappings to assay-specific identifiers. Count unmatched records separately from feature-level missing values.

If an assay was not collected, do not manufacture a zero-valued column. If the same label appears twice, investigate whether it is a technical replicate, repeated specimen or a data error. The solution depends on the scientific meaning rather than a blanket duplicate-removal command.

Interpretation challenge

Two layers have matching animal names but one was measured at baseline and the other after treatment. Are they concurrent matched measurements?

Suggested reasoning

No. They are linked longitudinal measurements. Keep timing explicit and define whether the analysis concerns baseline prediction, response or another temporal relationship.

Interpretation and limits

ID matching preserves supplied labels but cannot detect a mislabeled specimen collected in the field. Use collection records and appropriate sample-identity checks for real data. Avoid silent complete-case deletion when missing assays relate to treatment or health.

Exercises

1. Why does a sorted join still need an identity check?
2. What if one metadata animal has two assay columns?
3. Why is a missing protein value not log2 zero?

Suggested answers

1. Sorting conventions and keys can differ between tables.
2. Define technical replicates or repeated specimens explicitly; do not choose a duplicate silently.
3. Missing means unobserved; log2 zero corresponds to an original-scale value of one.

Continue learning

B03 Databases, B04 Programming and B15 Integration.

Continue with multi-omics QC. Return to the practical index or the unified map.