Numerical Analysis for Ordinary Differential Equations

This page serves as a permanent landing page for publications and code on probabilistic methods for the solution of ordinary differential equations. This particular html page is used by Philipp Hennig’s research group, but it is part of a wider research effort to study the possibility of assigning uncertainty to the result of deterministic computations. The corresponding community page can be found here.

  1. Hennig, P., & Hauberg, S. (2014). Probabilistic Solutions to Differential Equations and their Application to Riemannian Statistics. In Proc. of the 17th int. Conf. on Artificial Intelligence and Statistics (AISTATS) (Vol. 33). JMLR, W&CP.

    We study a probabilistic numerical method for the solution of both boundary and initial value problems that returns a joint Gaussian process posterior over the solution. Such methods have concrete value in the statistics on Riemannian manifolds, where non-analytic ordinary differential equations are involved in virtually all computations. The probabilistic formulation permits marginalising the uncertainty of the numerical solution such that statistics are less sensitive to inaccuracies. This leads to new Riemannian algorithms for mean value computations and principal geodesic analysis. Marginalisation also means results can be less precise than point estimates, enabling a noticeable speed-up over the state of the art. Our approach is an argument for a wider point that uncertainty caused by numerical calculations should be tracked throughout the pipeline of machine learning algorithms.

      author = {Hennig, Philipp and Hauberg, S{\o}ren},
      booktitle = {{Proc. of the 17th int. Conf. on Artificial Intelligence and
                        Statistics ({AISTATS})}},
      publisher = {JMLR, W\&CP},
      title = {{Probabilistic Solutions to Differential Equations and their
                        Application to Riemannian Statistics}},
      volume = {33},
      year = {2014},
      file = {},
      video = {},
      code = {},
      supplements = {}
  1. Schober, M., Kasenburg, N., Feragen, A., Hennig, P., & Hauberg, S. (2014). Probabilistic shortest path tractography in DTI using Gaussian Process ODE solvers. In P. Golland, N. Hata, C. Barillot, J. Hornegger, & R. Howe (Eds.), Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014 (Vol. 8675, pp. 265–272). Springer.

    Tractography in diffusion tensor imaging estimates connectivity in the brain through observations of local diffusivity. These observations are noisy and of low resolution and, as a consequence, connections cannot be found with high precision. We use probabilistic numerics to estimate connectivity between regions of interest and contribute a Gaussian Process tractography algorithm which allows for both quantification and visualization of its posterior uncertainty. We use the uncertainty both in visualization of individual tracts as well as in heat maps of tract locations. Finally, we provide a quantitative evaluation of different metrics and algorithms showing that the adjoint metric combined with our algorithm produces paths which agree most often with experts.

      author = {Schober, Michael and Kasenburg, Niklas and Feragen, Aasa and Hennig, Philipp and Hauberg, S{\o}ren},
      editor = {Golland, Polina and Hata, Nobuhiko and Barillot, Christian and Hornegger, Joachim and Howe, Robert},
      booktitle = {Medical Image Computing and Computer-Assisted Intervention --
                        MICCAI 2014},
      publisher = {Springer},
      location = {Heidelberg},
      series = {Lecture Notes in Computer Science},
      volume = {8675},
      year = {2014},
      pages = {265--272},
      title = {{Probabilistic shortest path tractography in {DTI} using
                        {G}aussian {P}rocess {ODE} solvers}},
      file = {../assets/pdf/Schober2014MICCAI.pdf},
      video = {},
      supplements = {}
  1. Schober, M., Duvenaud, D. K., & Hennig, P. (2014). Probabilistic ODE Solvers with Runge-Kutta Means. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, & K. Q. Weinberger (Eds.), Advances in Neural Information Processing Systems 27 (pp. 739–747). Curran Associates, Inc.

    Runge-Kutta methods are the classic family of solvers for ordinary differential equations (ODEs), and the basis for the state of the art. Like most numerical methods, they return point estimates. We construct a family of probabilistic numerical methods that instead return a Gauss-Markov process defining a probability distribution over the ODE solution. In contrast to prior work, we construct this family such that posterior means match the outputs of the Runge-Kutta family exactly, thus inheriting their proven good properties. Remaining degrees of freedom not identified by the match to Runge-Kutta are chosen such that the posterior probability measure fits the observed structure of the ODE. Our results shed light on the structure of Runge-Kutta solvers from a new direction, provide a richer, probabilistic output, have low computational cost, and raise new research questions.

      title = {Probabilistic {ODE} Solvers with {R}unge-{K}utta Means},
      author = {Schober, Michael and Duvenaud, David K and Hennig, Philipp},
      booktitle = {Advances in Neural Information Processing Systems 27},
      editor = {Ghahramani, Z. and Welling, M. and Cortes, C. and Lawrence, N.D. and Weinberger, K.Q.},
      pages = {739--747},
      year = {2014},
      publisher = {Curran Associates, Inc.},
      file = {},
      supplements = {}