Driven systems abound in the world of complex systems. We show that the underlying statistics of many of these can be undserstood by so-called sample space reducing (SSR) processes that offer an intuitive understanding of the origin and ubiquity of fat tailed distributions in countless systems, including power-laws. SSR processes are mathematically simple and offer an exact alternative to Boltzmann equation based approaches to non-equailibrium systems. We show that in many situations the statistics of driven systems is determined by the driving process and is universal with respect to the specific relaxation dynamics. With simple driving process we can naturally derive Zipf’s law, exact power-laws, exponential, Gamma, normal, Weibull, Gompertz, and Pareto distributions. We discuss a number of examples of SRR processes that range from fragmentation processes, language formation, cascading and search processes, as well as the derivation of the equivalent of the Maxwell Botzmann distribution for inelastically colliding particles in a box.