# learn2learn: A Library for Meta-Learning Research

@article{Arnold2020learn2learnAL, title={learn2learn: A Library for Meta-Learning Research}, author={S{\'e}bastien M. R. Arnold and Praateek Mahajan and Debajyoti Datta and Ian Bunner and Konstantinos Saitas Zarkias}, journal={ArXiv}, year={2020}, volume={abs/2008.12284} }

Meta-learning researchers face two fundamental issues in their empirical work: prototyping and reproducibility. Researchers are prone to make mistakes when prototyping new algorithms and tasks because modern meta-learning methods rely on unconventional functionalities of machine learning frameworks. In turn, reproducing existing results becomes a tedious endeavour -- a situation exacerbated by the lack of standardized implementations and benchmarks. As a result, researchers spend inordinate… Expand

#### 15 Citations

A Channel Coding Benchmark for Meta-Learning

- Computer Science, Mathematics
- ArXiv
- 2021

This work proposes the channel coding problem as a benchmark for meta- learning and uses this benchmark to study several aspects of meta-learning, including the impact of task distribution breadth and shift, which can be controlled in the coding problem. Expand

A Channel Coding Benchmark for Meta-Learning

- 2021

Meta-learning provides a popular and effective family of methods for data-efficient learning of new tasks. However, several important issues in meta-learning have proven hard to study thus far. For… Expand

Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation

- Computer Science, Mathematics
- ICML
- 2021

This work proves that for over-parameterized neural networks with sufficient depth, the learned predictive functions of MTL and GBML are close, and corroborates the theoretical findings by showing that, with proper implementation, MTL is competitive against state-of-the-art GBML algorithms on a set of few-shot image classification benchmarks. Expand

protANIL: a Fast and Simple Meta-Learning Algorithm

- 2020

A broad recognition of important practical benefits inherent to meta-learning paradigm has recently elevated the few-shot learning problem into the spotlight of machine learning research. While the… Expand

Embedding Adaptation is Still Needed for Few-Shot Learning

- Computer Science
- ArXiv
- 2021

This work proposes ATG, a principled clustering method to defining train and test tasksets without additional human knowledge, and empirically demonstrates the effectiveness of ATG in generating tasksets that are easier, in-between, or harder than existing benchmarks, including those that rely on semantic information. Expand

Minimax and Neyman–Pearson Meta-Learning for Outlier Languages

- Computer Science
- FINDINGS
- 2021

Two variants of MAML are created based on alternative criteria that reduce the maximum risk across languages, while Neyman–Pearson MAMl constrains the risk in each language to a maximum threshold, which constitute fully differentiable two-player games. Expand

On sensitivity of meta-learning to support data

- Computer Science
- ArXiv
- 2021

It is demonstrated the existence of (unaltered, in-distribution, natural) images that, when used for adaptation, yield accuracy as low as 4% or as high as 95% on standard few-shot image classification benchmarks, suggesting that robust and safe meta-learning requires larger margins than supervised learning. Expand

Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability

- Computer Science, Mathematics
- 2021

A probably approximately correct (PAC) bound is derived for gradient-based metalearning using two different generalization frameworks in order to deal with the qualitatively different challenges of generalization at the “base” and “meta” levels. Expand

Uniform Sampling over Episode Difficulty

- Computer Science
- ArXiv
- 2021

This paper proposes a method to approximate episode sampling distributions based on their difficulty and finds that sampling uniformly over episode difficulty outperforms other sampling schemes, including curriculum and easy-/hard-mining. Expand

Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD

- Computer Science
- ArXiv
- 2021

Sign-MAML is theoretically-grounded as it does not impose any assumption on the absence of second-order derivatives during meta training, and compared to MAML, it achieves a much more graceful tradeoff between classification accuracy and computation efficiency. Expand

#### References

SHOWING 1-10 OF 36 REFERENCES

Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML

- Computer Science, Mathematics
- ICLR
- 2020

The ANIL (Almost No Inner Loop) algorithm is proposed, a simplification of MAML where the inner loop is removed for all but the (task-specific) head of a MAMl-trained network, and performance on the test tasks is entirely determined by the quality of the learned features, and one can remove even the head of the network (the NIL algorithm). Expand

Meta-SGD: Learning to Learn Quickly for Few Shot Learning

- Computer Science
- ArXiv
- 2017

Meta-SGD, an SGD-like, easily trainable meta-learner that can initialize and adapt any differentiable learner in just one step, shows highly competitive performance for few-shot learning on regression, classification, and reinforcement learning. Expand

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

- Computer Science
- ICML
- 2017

We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning… Expand

Torchmeta: A Meta-Learning library for PyTorch

- Mathematics, Computer Science
- ArXiv
- 2019

The constant introduction of standardized benchmarks in the literature has helped accelerating the recent advances in meta-learning research. They offer a way to get a fair comparison between… Expand

Meta-learning with differentiable closed-form solvers

- Computer Science, Mathematics
- ICLR
- 2019

The main idea is to teach a deep network to use standard machine learning tools, such as ridge regression, as part of its own internal model, enabling it to quickly adapt to novel data. Expand

Meta-Learning With Differentiable Convex Optimization

- Computer Science
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2019

The objective is to learn feature embeddings that generalize well under a linear classification rule for novel categories and this work exploits two properties of linear classifiers: implicit differentiation of the optimality conditions of the convex problem and the dual formulation of the optimization problem. Expand

Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning

- Computer Science, Mathematics
- ICLR
- 2019

This work uses meta-learning to train a dynamics model prior such that, when combined with recent data, this prior can be rapidly adapted to the local context and demonstrates the importance of incorporating online adaptation into autonomous agents that operate in the real world. Expand

Benchmarking Deep Reinforcement Learning for Continuous Control

- Computer Science, Mathematics
- ICML
- 2016

This work presents a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with very high state and action dimensionality such as 3D humanoid locomotion, task with partial observations, and tasks with hierarchical structure. Expand

Matching Networks for One Shot Learning

- Computer Science, Mathematics
- NIPS
- 2016

This work employs ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories to learn a network that maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types. Expand

ProMP: Proximal Meta-Policy Search

- Computer Science, Mathematics
- ICLR
- 2019

A novel meta-learning algorithm is developed that overcomes both the issue of poor credit assignment and previous difficulties in estimating meta-policy gradients and leads to superior pre-adaptation policy behavior and consistently outperforms previous Meta-RL algorithms in sample-efficiency, wall-clock time, and asymptotic performance. Expand