Deep Learning on avanwyk

Super-convergence in Tensorflow 2 with the 1Cycle Policy

andrich@avanwyk.com (Andrich van Wyk) — Mon, 02 Sep 2019 21:45:00 +0000

Implementing super-convergence for deep neural network training in Tensorflow 2 with the 1Cycle learning rate policy.

Finding a Learning Rate with Tensorflow 2

andrich@avanwyk.com (Andrich van Wyk) — Sun, 28 Jul 2019 11:05:37 +0000

Implementing the technique in Tensorflow 2 is straightforward. Start from a low learning rate, increase the learning rate and record the loss. Stop when a very high learning rate is reached. Plot the losses and learning rates choosing a learning rate where the loss is decreasing at a rapid rate.

African Antelope: A Case Study of Creating an Image Dataset with FastAI

andrich@avanwyk.com (Andrich van Wyk) — Sun, 14 Apr 2019 11:19:17 +0000

An end-to-end example of how to create your own image dataset from scratch and train a ResNet50 convolutional neural network for image classification using the FastAI library.

CDC Mortality Prediction with FastAI for Tabular Data

andrich@avanwyk.com (Andrich van Wyk) — Fri, 19 Oct 2018 20:57:06 +0000

This post will cover getting started with FastAI v1 at the hand of tabular data. It is aimed at people that are at least somewhat familiar with deep learning, but not necessarily with using the FastAI v1 library.

Encoding Cyclical Features for Deep Learning

andrich@avanwyk.com (Andrich van Wyk) — Fri, 13 Apr 2018 15:50:24 +0000

A key concern when dealing with cyclical features is how we can encode the values such that it is clear to the deep learning algorithm that the features occur in cycles.

This post looks at a strategy to encode cyclical features in order to clearly express their cyclical nature.