In UO2 nuclear fuel, the retention and release of fission gas atoms such as xenon (Xe) are important for nuclear fuel performance by, for example, reducing the fuel thermal conductivity, causing fuel swelling that leads to mechanical interaction with the clad, increasing the plenum pressure and reducing the fuel-clad gap thermal conductivity. We use multi-scale simulations to determine fission gas diffusion mechanisms as well as the corresponding rates in UO2 under both intrinsic and irradiation conditions. In addition to Xe and Kr, the fission products Zr, Ru, Ce, Y, La, Sr and Ba have been investigated. Density functional theory (DFT) calculations are used to study formation, binding and migration energies of small clusters of Xe atoms and vacancies. Empirical potential calculations enable us to determine the corresponding entropies and attempt frequencies for migration as well as investigate the properties of large clusters or small fission gas bubbles. A continuum reaction-diffusion model is developed for Xe and point defects based on the mechanisms and rates obtained from atomistic simulations. Effective fission gas diffusivities are then obtained by solving this set of equations for different chemical and irradiation conditions using the MARMOT phase field code. The predictions are compared to available experimental data. The importance of the large XeU3O cluster (a Xe atom in a uranium + oxygen vacancy trap site with two bound uranium vacancies) is emphasized, which is a consequence of its high mobility and high binding energy. We find that the XeU3O cluster gives Xe diffusion coefficients that are higher for intrinsic conditions than under irradiation over a wide range of temperatures. Under irradiation the fast-moving XeU3O cluster recombines quickly with irradiation induced interstitial U ions, while this mechanism is less important for intrinsic conditions. The net result is higher concentration of the XeU3O cluster for intrinsic conditions than under irradiation. We speculate that differences in the irradiation conditions and their impact on the XeU3O cluster can explain the wide range of diffusivities reported in experimental studies. However, all vacancy-mediated mechanisms underestimate the Xe diffusivity compared to the empirical radiation-enhanced rate used in most fission gas release models. We investigate the possibility that diffusion of small fission gas bubbles or extended Xe-vacancy clusters may give rise to the observed radiation-enhanced diffusion coefficient. These studies highlight the importance of U divacancies and an octahedron coordination of uranium vacancies encompassing a Xe fission gas atom. The latter cluster can migrate via a multistep mechanism with a rather low effective barrier, which together with irradiation-induced clusters of uranium vacancies, gives rise to the irradiation-enhanced diffusion coefficient observed in experiments.