Computation of Kullback-Leibler (KL) distance between text-documents using numpy

Question

My goal is to compute the KL distance between the following text documents:

1)The boy is having a lad relationship
2)The boy is having a boy relationship
3)It is         


        

Answer 1

A potential issue might be in your NP definition of KL. Read the wikipedia page for formula: http://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence

Note that you multiply (p-q) by the log result. In accordance with the KL formula, this should only be p:

 return np.sum(np.where(p != 0,(p) * np.log10(p / q), 0))

That may help...