Material Detail

Probabilistic Interpretation of Quasi-Newton Methods

Probabilistic Interpretation of Quasi-Newton Methods

This video was recorded at NIPS Workshops, Lake Tahoe 2012. This talk is a case-study about the utility of probabilistic formulations for numerical mathematics. I present a recent result showing that quasi-Newton methods can be interpreted as performing Gaussian (least-squares) regression on the Hessian of the objective function, using a particular noise process to keep uncertainty constant, and a non-obvious structured prior which ignores the duality between vectors and co-vectors. This insight connects these numerical methods to important areas of machine learning (regression) and control (Kalman filters). It allows cross-fertilization: Better numerical algorithms can be built using existing knowledge from machine learning, and machine learning can benefit from a new structured prior... Show More

Quality

  • User Rating
  • Comments
  • Learning Exercises
  • Bookmark Collections
  • Course ePortfolios
  • Accessibility Info

More about this material

Comments

Log in to participate in the discussions or sign up if you are not already a MERLOT member.