Abstract:

The proposed PhD project aims to improve traditional approximate Bayesian inference techniques by leveraging concepts from differential geometry. The project seeks to address issues of existing approximate inference methods defined in Euclidean space that do not approximate well the local structure of the true posterior, and for example leads to underestimated predictive uncertainty in some scenarios. Hopefully, such problems can be addressed by establishing approximate posterior distributions that can flexibly adjust to the local geometric structure of the true posterior distribution. 
 
The specific research ideas consider enhancing existing geometric approximate inference techniques as well as proposing new geometric formulations of traditional approximate inference techniques defined in Euclidean space. A promising direction is to extend the computationally attractive Riemannian Laplace approximation of Bergamin et al. (2023) developed for Bayesian Neural Networks, by modifying its tangent space distribution, as the current method is provably biased due to the considered Riemannian metric. Other studies fix this issue by using the Fisher metric, yet at the cost of being computationally harder. A better solution is to develop a new Riemannian metric that has low computationally cost and limited bias. Such a flexible approximate posterior, can be easily utilized under the mean-field variational inference framework. In addition, the change of the parameter space geometry together with the associated approximate posterior distributions naturally leads to a geometric evidence lower bound that can potentially also be used to establish PAC-Bayes generalization bounds.  
 
The significance of the project lies in improving uncertainty quantification mainly in Bayesian deep learning, which is crucial for explainability and trustworthiness of safety-critical decision-making with AI systems. Traditional approximate Bayesian inference techniques mainly prioritize either approximation accuracy or computational complexity. Instead, the recently proposed Riemannian Laplace approximation demonstrates how simple geometric awareness can enhance approximation accuracy while retaining relatively low computational cost and ensuring explainability. Advancing geometric approximate Bayesian inference methods could ultimately improve model reliability and enable the scalability of Bayesian deep learning to even larger deep neural networks.