Super-convergence in Tensorflow 2 with the 1Cycle Policy
Implementing super-convergence for deep neural network training in Tensorflow 2 with the 1Cycle learning rate policy.
tensorflow
Finding a Learning Rate with Tensorflow 2
Implementing the technique in Tensorflow 2 is straightforward. Start from a low learning rate, increase the learning rate and record the loss. Stop when a very high learning rate is reached. Plot the losses and learning rates choosing a learning rate where the loss is decreasing at a rapid rate.