Most field theories for active matter neglect effects of memory and inertia. However, recent experiments have found inertial delay to be important for the motion of self-propelled particles. A major challenge in the theoretical description of these effects, which makes the application of standard methods very difficult, is the fact that orientable particles have both translational and orientational degrees of freedom which do not necessarily relax on the same time scale. In this work, we combine modern mathematical methods from particle physics and nonlinear dynamics to derive the general mathematical form of a field theory for soft-matter systems with two different time scales. This allows to obtain a phase field crystal model for polar (i.e., nonspherical or active) particles with translational and orientational memory. Notably, this theory is of third order in temporal derivatives and can thus be seen as a spatiotemporal jerky dynamics. An analysis of the model reveals interesting effects of memory on the dynamics of active systems.