Near-Optimal Learning of Tree-Structured Distributions by Chow and Liu.

We provide finite sample guarantees for the classical Chow--Liu algorithm [Chow and Liu, IEEE Trans. Inform. Theory, 14 (1968), pp. 462--467] to learn a tree-structured graphical model of a distribution. For a distribution P on \Sigman and a tree T on n nodes, we say T is an \varepsilon -approximate tree for P if there is a T -structured distribution Q such that D(P 11 Q) is at most \varepsilon more than the best possible tree-structured distribution for P. We show that if P itself is tree structured, then the Chow--Liu algorithm with the plug-in estimator for mutual information with Owidetilde\(1\Sigma 13n\varepsilon 1) independent and identically distributed samples outputs an \varepsilon -approximate tree for P with constant probability. In contrast, for a general P (which may not be tree-structured), \Omega (n2\varepsilon 2) samples are necessary to find an \varepsilon -approximate tree. Our upper bound is based on a new conditional independence tester that addresses an open problem posed by Canonne et al. [Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, ACM, 2018, pp. 735--748]: we prove that for three random variables X,Y, Z each over \Sigma , testing if I(X; Y 1 Z) is 0 or \geq \varepsilon is possible with Owidetilde\(1\Sigma 13/\varepsilon) samples. Finally, we show that for a specific tree T, with Owidetilde\(1\Sigma 12n\varepsilon 1) samples from a distribution P over \Sigman, one can efficiently learn the closest T -structured distribution in KL divergence by applying the add-1 estimator at each node.