A dive into understanding Mathematical Theory of Bayesian Statistics and Singular Learning Theory. Along the way we will explore optimization methods like Monte carlo Markov Chains, SGLD for sampling.