A Cross Entropy test for tSNE by Dr Oliver Burton

The low-dimensional representation of high-dimensional data makes t-SNE an attractive visualisation tool, yet it also has value as an analytical tool. We have developed the Cross Entropy test, a statistical test capable of distinguishing biological differences in single cell t-SNE representations, while being robust against false detection of differences in technical replicates or the seed-dependent variation in t-SNE generation. As the t-SNE algorithm is driven by the cross entropy of the individual cells in the dataset, and the t-SNE fixes the average point entropy, each t-SNE can be considered a distribution of cross entropy divergences. Deriving a distribution of cross entropy divergences per t-SNE plot therefore allows the use of the Kolmogorov-Smirnov test to evaluate the degree of difference between two, or more, t-SNE plots.

The Cross Entropy test is a useful tool for calculating p values on the difference between any two t-SNE or UMAP plots, whether the data comes from flow cytometry, mass cytometry or single cell sequencing. Further, the test generates a quantitative comparison of the extent of differences, allowing you to compare multiple t-SNE or UMAP plots and identify outgroups and clustered samples. See an overview of the Cross Entropy test given by Dr Oliver Burton:

Read the paper on arXiv here, get the GitHub package here or see the tutorial on how to use the tool here.

Comments

[Joseph Miller]

Wow, this looks like a really great algorithm, I watched the video carefully. moto road rash