Kullback-Leibler divergence Kullback-Leibler divergence is defined for two random variables X and Y by K-L divergence is non-negative, and it’s zero if and only if X and Y have the same distribution. But it is not a metric, for reasons explained here. For one thing, it’s not symmetric. Jeffreys divergence We can fix the symmetry problem by […] The post Turning K-L divergence into a metric first appeared on John D. Cook .