Implementing super-convergence for deep neural network training in Tensorflow 2 with the 1Cycle learning rate policy.
September 02, 2019
Implementing the technique in Tensorflow 2 is straightforward. Start from a low learning rate, increase the learning rate and record the loss. Stop when a very high learning rate is reached. Plot the losses and learning rates choosing a learning rate where the loss is decreasing at a rapid rate.
An end-to-end example of how to create your own image dataset from scratch and train a ResNet50 convolutional neural network for image classification using the FastAI library.
Instructions for installing FastAI v1 within a freshly created Anaconda virtual environment.
This post will cover getting started with FastAI v1 at the hand of tabular data. It is aimed at people that are at least somewhat familiar with deep learning, but not necessarily with using the FastAI v1 library.