The diffeomorphism group of a compact manifold is canonically an infinite-dimensional manifold whose Lie algebra is the space of vector fields with its usual Lie bracket. The case of a non-compact manifold is more mysterious: As an infinite-dimensional manifold it can only be modelled on the space of compactly supported vector fields. Moreover, by a no-go theorem of Omori, there is no exponential map from the space of all vector fields to diffeomorphisms. In the first part of the talk, we review these issues. In the second part, we offer a solution by showing that: 1.) All diffeomorphism groups are elastic as diffeological spaces. That is, they reside in a subcategory of diffeological spaces which carries a tangent structure in the sense of Rosicky. 2.) The Lie algebra of any elastic group is, as usual, the space of invariant vector fields. 3.) The Lie algebra of a diffeomorphism group is the Lie algebra of all vector fields, even in the non-compact case.