In galaxy clustering measurements, summary statistics, e.g., 2-pt correlation functions and power spectra, have been widely used to extract cosmological information. These statistics summarise the information of the observed galaxy number density field and theoretical models such as perturbation theory can accurately predict the statistics up to mildly non-linear regime. On the other hand, the statistics do not contain the full information of the observed field and, especially for non-linear fields, a large fraction of the information is not accessible solely from the statistics. In this work, we focus on the field-level approach that directly compares the observed field itself with the non-linear density field predicted by the forward modelling. To this end, we develop differentiable GridSPT, which is a forward modelling method based on standard perturbation theory and utilises accelerated Fourier transforms and automatic differentiation with GPUs. This functionality makes statistical inference more efficient and convergent. We discuss the validity of the field-level inference through reconstruction of the initial density field and cosmological parameters from non-linear halo density fields from N-body simulations.