When I run pc_relate on my data set I get negative IBD1 and IBD2 values. To my understanding IBD1 and IBD2 denotes the probability for two individuals of sharing 1 or 2 alleles at a locus, respectively.
Since probabilities should be in [0, 1] I find it difficult to interpret the negative results.
I run on whole exome data with maf > 0.05 and output all pairs.
Can I simply threshold all IBD1<0 and IBD2<0 to 0?
Dan can provide more insight (he implemented the functionality in Hail), but my understanding is that while the relatedness estimates from pc_relate are comparable to IBD1 and IBD2 values, they have slightly different definitions. Importantly, they should be unbiased estimates of IBD1 and IBD2, and a noisy estimate of 0 can certainly be negative.
I think the 2016 PC-Relate paper probably has the best information about this.
Great, thank you.
I think the R implementation of PC-Relate clips values by setting negative values to zero and values greater than 1 to 1. Hail does not do this. The PC-Relate estimator is a noisy, linear estimator, so, as Tim states, it can give negative values.
Moreover, Appendix A of the PC-Relate paper discusses a negative bias of the estimators in the presence of admixture. The magnitude of the bias is a function of how many admixed SNPs are shared between the samples. The evaluation of the PC-Relate paper indicates that this bias is small. See Figures 5 and 6. In their studies the bias is on the order of 0.1. If you have significantly larger negative values then I would investigate whether your data matches the PC-Relate model.