Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words

• https://arxiv.org/abs/2205.05092
• Kaitlyn Zhou, Kawin Ethayarajh, Dallas Card, Dan Jurafsky

Cosine similarity, Word embedding, The impact of token frequency in embedding

Directly extends Zhou2021frequency‘s analysis of frequency-based distortions in contextualized embeddings, showing that cosine similarity is particularly unreliable for high-frequency words.