The impact of Box-Cox transformation on phenotypic and genomic characteristics of litter size variability in Landrace pigs

The genetic background of variability remains of interest especially in traits of high economic importance, e.g. litter size in pigs. It has been indicated that the data transformation can affect the variability phenotype. This study aims to evaluate the phenotypic and genomic background of variability of litter size obtained from data before and after the Box-Cox transformation. In total, 67 500 records on the total number born (TNB) in Landrace pig population were used. Since the data presented skewness, the decision was made to perform Box-Cox transformation on TNB and obtain bcTNB. Next, the phenotypic variability was estimated as log-transformed variance of residuals (LnVar) for both TNB (LnVar_TNB) and bcTNB (LnVar_bcTNB). The variability traits were further used in the genome-wide association study (GWAS) performed on 10 688 sows genotyped with Axiom porcine 660 K or imputed to 660 K SNP-chip. The substantial difference in skewness was observed after data transformation, represented as a change from −0.46 to −0.02. Heritability for TNB was 0.118 vs 0.125 for bcTNB. The heritability for LnVar_TNB was 0.0025 vs 0.0037 for LnVar_bcTNB. The change in the genetic variance was confirmed when genetic coefficients on SD level were compared: 2% for LnVar_TNB vs 4% for LnVar_bcTNB. In bivariate analysis, the genetic correlation between the additive genetic effects of the mean TNB and its variability changed from 0.38 to 0.63. The observed positive genetic correlations indicated that selection focused on increasing the litter size will simultaneously cause an increase in litter size variability. Based on GWAS, 14 SNPs were detected for LnVar_TNB and eight for LnVar_bcTNB, with two of them indicating the most promising candidate genes. First candidate gene located on Sus scrofa chromosome (SSC) 3 is STAG3, which plays an essential role in gametogenesis. Second gene located on SSC 10 is ESRRG, which affects placenta development. The additional post-GWAS analysis indicated even more candidate genes for LnVar_TNB and LnVar_bcTNB. The most promising candidate gene was located on SSC 13 – MFN1, which is involved in embryonic development. The results of this study indicated a substantial change in variance components for variability when the Box-Cox transformation was applied to data presenting skewness. Moreover, the data transformation changed the phenotype substantially enough that only part of SNP overlapped between two variability traits. Our investigation shows that it is essential to perform Box-Cox transformation for skewed data in order to properly describe phenotypic and genomic properties of litter size variability in Landrace pigs.