My thesis is focused on improvement the reliability and safety of modern reinforcement learning methods, by explicitly leveraging models of epistemic and aleatory uncertainty to help make RL more reliable. My work leverages theory and practical applications of (approximate) Bayesian inference, transfer and knowledge representation, ensemble methods and safety. I am currently also a post-graduate affiliate at the Vector Institute since April 2020.