Viewmaker Networks: Learning Views for Unsupervised Representation Learning

May 3, 2021

Speakers

About

Many recent methods for unsupervised representation learning involve training models to be invariant to different "views," or augmented versions of an input. However, designing these views requires considerable human expertise and experimentation, hindering widespread adoption of unsupervised representation learning methods across domains and modalities. To address this, we propose viewmaker networks: generative models which learn to produce input-dependent views for contrastive learning. We train these networks jointly with the main network to produce adversarial $\ell_p$ perturbations for an input, which yields challenging yet faithful views without extensive human tuning. Our learned views enable comparable transfer accuracy to the the well-studied SimCLR augmentations when applied on CIFAR-10, while significantly outperforming baseline augmentations in speech (+9% absolute) and IMU sensor (+17% absolute) domains. We also show how viewmaker views can be combined with SimCLR views to improve robustness to common image corruptions. Our method provides a roadmap for reducing the amount of expertise and effort needed for unsupervised learning, potentially extending its benefits to a much wider set of domains.

Organizer

Categories

About ICLR 2021

The International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning. ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.

Store presentation

Should this presentation be stored for 1000 years?

How do we store presentations

Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

Sharing

Recommended Videos

Presentations on similar topic, category or speaker

Interested in talks like this? Follow ICLR 2021