Second Order Behaviour in Augmented Neural ODEs

Add to your list(s) Download to your calendar using vCal

• Alexander Norcliffe
• Tuesday 10 November 2020, 13:15-14:15
• Zoom.

If you have a question about this talk, please contact Mateja Jamnik.

Join us on Zoom

Neural Ordinary Differential Equations (NODEs) are a new class of models that transform data continuously through infinite-depth architectures. The continuous nature of NOD Es has made them particularly suitable for learning the dynamics of complex physical systems. While previous work has mostly been focused on first order ODEs, the dynamics of many systems, especially in classical physics, are governed by second order laws. In this work, we consider Second Order Neural ODEs (SONODEs). We show how the adjoint sensitivity method can be extended to SONOD Es and prove that the optimisation of a first order coupled ODE is equivalent and computationally more efficient. Furthermore, we extend the theoretical understanding of the broader class of Augmented NOD Es (ANODEs) by showing they can also learn higher order dynamics with a minimal number of augmented dimensions, but at the cost of interpretability. This indicates that the advantages of ANOD Es go beyond the extra space offered by the augmented dimensions, as originally thought. Finally, we compare SONOD Es and ANOD Es on synthetic and real dynamical systems and demonstrate that the inductive biases of the former generally result in faster training and better performance.

arXiv

NeurIPS pre-proceedings

This talk is part of the Artificial Intelligence Research Group Talks (Computer Laboratory) series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Artificial Intelligence Research Group Talks (Computer Laboratory)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• Guy Emerson's list
• Hanchen DaDaDash
• Interested Talks
• School of Technology
• Speech Seminars
• Trust & Technology Initiative - interesting events
• Zoom
• bld31
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list
• yk449

Note that ex-directory lists are not shown.