Noise-contrastive estimation: A new estimation principle for unnormalized statistical models

• http://proceedings.mlr.press/v9/gutmann10a

Noise Contrasive Estimation is a nice way to train a Machine learning model. It asks the model to correctly classify actual co-occurrences from random pairs (noise).

Negative sampling can be considered as a simplified, biased version. This bias was studied in Murray2023unsupervised and Kojaku2021Residual2Vec.