Dynamics and Symmetry applied to theoretical problems in neural nets and statistical learning

speaker: Mike Field (UC Santa Barbara)

abstract: We describe some new results in dynamics, symmetry and analysis that have applications to the theoretical study of machine learning. Most of the talk will emphasise the mathematics; at appropriate points, brief comments will be made concerning the motivating questions from statistical learning.

Background: A notable characteristic of machine learning, especially deep learning, is that the methods generally work rather well in spite of theoretical predictions to the contrary. In particular, the optimization involved in the learning process is highly non-convex and so, from a general perspective, should surely not work well. Currently, much of the existing theory applies to limiting regimes. For example, in 2-layer networks, the thermodynamic limit from statistical physics (the number of inputs goes to infinity), and mean field theory, optimal control, Neural Tangent Kernel, compositional kernels, etc., where the number of neurons goes to infinity.

We address the highly non-convex optimization landscape in the natural (finite) regime and indicate how it is possible to give precise quantitative information about families of spurious minima and their Hessian spectrum, distinguish between different types of spurious minima, and understand the geometry involved in the creation and annihilation of spurious minima. The work presented in the talk is part of a collaboration with Yossi Arjevani (School of Engineering and Computer Science, Hebrew University, Israel).

timetable:
Wed 31 May, 15:00 - 15:50, Aula Dini
<< Go back