Deep Learning Jeremy Howard: For convolutional neural networks, the recent best architecture is the "100-layer tiramisu". Jeremy: This result can be now surpassed easily with the FastAI library. Source: Lesson 3: Deep Learning 2019 - Data blocks; Multi-label classification; Segmentation
Deep Learning Jeremy Howard: Gradually increasing learning rate at the start leads to better generalization Source: Lesson 3: Deep Learning 2019 - Data blocks; Multi-label classification; Segmentation
Deep Learning Jeremy Howard: If your training loss is higher than your validation loss, it definitely means that you are underfitting. "You want your training loss to be lower than your validation loss" - said Jeremy. When you are underfitting, you can: train for longer, train the last bit at a lower learning rate. If you are still underfitting, then you are going
Information Sciences Jeremy Howard: Using smaller images in training first works great, but is still largely unknown. Jeremy called that idea "progressive resizing". Knowing that starting from smaller images turns out to accelerate learning and results in better generalizations, an even more precise term for this technique would seem to be "progressive upsizing". One of his findings
Information Sciences Jeremy Howard: In transfer learning you start with a model trained to do something, then fine-tune it. When you have a model trained to recognize images of size 128x128, you don't need to train another model from scratch to be able to recognize images of size 256x256. What you can do instead, is: retrain the prevoius model for the new task.
Information Sciences Frank Nielsen: Information sciences seek methods to distill information from data to models. In a recent study published in a scientific journal named Entropy, Frank Nielsen proposed the term "Information Sciences" to describe the common goal of several modern efforts to turn data into models: In short, information sciences seek methods to distill information from
Deep Learning Jeremy Howard: Once the data is there, there is very little to do afterwards. What you can learn from Jeremy Howard from Fast.AI is that preparing the data is the crucial step in preparing a state-of-the-art Deep Learning model. For classifying cat and dog images, it is good to rotate some of the pictures; flip some of