Quantum theory assigns definite probabilities even when measurement outcomes are indefinite. I will argue that the ideal reference frames underlying this operational picture cannot be physically realised in quantum gravity, owing to fundamental limitations arising from the interplay between gravity and quantum theory. With non-ideal quantum reference frames, relative frequencies can remain uncertain and mutually incompatible even in the limit of arbitrarily many system copies. Their correlations can violate Bell inequalities, excluding locally causal explanations based on hidden, definite probabilities. These results suggest revisiting the operational notion of a quantum state in quantum gravity.