Written by: Stephen Hsu
Primary Source: Information Processing
I think these are the slides.
One of the topics which I’ve remarked on before is the absence of local minima in the high dimensional optimization required to tune these DNNs. In the limit of high dimensionality a critical point is overwhelmingly likely to be a saddlepoint (have at least one negative eigenvalue). This means that even though the surface is not strictly convex the optimization is tractable.
Latest posts by Stephen Hsu (see all)
- Frank Herbert interview on the origins of Dune (1969) - February 15, 2018
- Genetic testing and embryo selection: current status and ethical issues - February 15, 2018
- The History of Synth Pop (video documentary) - February 11, 2018