When an impurity is immersed into an environment, it changes its properties due to its interactions with the surrounding medium. The impurity is dressed by excitations in the bath and, depending on the nature of the environment, new collective states of matter are formed. These states can, for instance, have the character of quasiparticles, called polarons, or can be states that are completely orthogonal to the original, non-interacting state of the system. In this talk, I will present recent experimental and theoretical progress on studying a variety of polaronic phenomena encountered in ultracold atomic systems, and discuss their relation to phenomena of relevance in novel two-dimensional semiconductor materials. I will then focus on employing polaronic effects in Rydberg systems as a probe of their many-body environment. In such systems the interaction between the Rydberg atom and their surrounding atomic gas gives rise to a new polaronic dressing mechanisms, where instead of collective excitations, molecules of gigantic size dress the Rydberg impurity, leading to the formation of Rydberg superpolarons. Using a functional determinant approach which incorporates atomic and many-body theory we show how bosonic and fermionic statistics can be probed by Rydberg excitations and we demonstrate that distinct Fermi and Bose polaron physics can be observed using Rydberg excitations in ultracold quantum gases.