Exploring Stochastic Gradient Descent with Restarts (SGDR)

Mark Hoffmann
5 min readNov 16, 2017
Example of Cyclic Learning Rates: From paper Snapshot Ensembles

This is my first deep learning blog post. I started my deep learning journey around January of 2017 after I heard about fast.ai from a presentation at a ChiPy meetup I attended (the Chicago Python user group). It was my first introduction to the topic and the only thing I knew about neural networks before were simple multi layer perception (MLP) models that acted as decent non-linear…

--

--

Mark Hoffmann

AI Engineer — Meta | Previously Chief Architect — Ubiety, AI/ML — NASA Jet Propulsion Lab / DARPA || AI / Software Engineering / Systems