I’ve always been fascinated by the concept of Bayesian deep learning, but a recent Reddit post got me thinking – can we really trust the uncertainty quantification it provides? The author of the post questioned how a prior can be placed on all the parameters of a deep network, and I have to agree that it’s a fair concern.
Take, for instance, placing a 0,1 Gaussian prior over the parameters. Is this a good prior? Are there better options? And how can we define better priors given a specific domain? These are questions that I think are worth exploring.
What I find interesting is that in other areas of machine learning, such as Gaussian processes, we can relate the kernel structure to desired properties of the underlying function, like shocks or trends. This makes me wonder if we can do something similar in Bayesian deep learning.
So, I’d love to hear from experts in the field – can Bayesian deep learning provide truly grounded uncertainty quantification? Or are there limitations that we need to be aware of? Share your thoughts in the comments below!