Barlow Twins: Self-Supervised Learning via Redundancy Reduction

• https://proceedings.mlr.press/v139/zbontar21a.html
• Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, Stephane Deny

Self supervised learning.

The method is inspired by the Efficient coding hypothesis proposed in Barlow1961possible.

It trains two identical networks, to which distorted images are fed into. The objective function is about these identical networks should produce as identical as possible representations for the distorted images.

See also Bielak2021graph for an application to Graph embedding.