Capacity Achieving Random Sparse Linear Codes

In this paper the existence of capacity achieving linear codes with arbitrarily sparse generator matrices is proved. In particular, we show the existence of capacity achieving codes for which the density of ones in the generator matrix is arbitrarily low. The existing results on the existence of capacity achieving linear codes in the literature are limited to the codes whose generator matrix elements are zero or one with necessarily equal probability, yielding a non-sparse generator matrix. This will imply a high encoding complexity. An interesting trade-off between the sparsity of the generator matrix and the value of the error exponent is also demonstrated. Compared to the existing results in the literature, which are limited to codes with non-sparse generator matrices, the proposed approach is novel and more concise. Although the focus in this paper is on the Binary Symmetric and Binary Erasure Channels, the results can be easily extended to other discrete memory-less symmetric channels.