Dynamical coupling between matter and gauge fields underlies the emergence of many exotic particle-like excitations in condensed matter and high energy physics. An important stepping stone to simulate this physics in atomic quantum gases relies on the synthesis of density-dependent gauge fields. Here we demonstrate deterministic formation of domain walls in a stable Bose-Einstein condensate with a synthetic gauge field that depends on the atomic density. The gauge field is created by simultaneous modulations of the optical lattice potential and interatomic interactions, and results in domains of atoms condensed into two different momenta. Modeling the domain walls as elementary excitations, we find that the domain walls respond to synthetic electric field with a charge-to-mass ratio larger than and opposite to that of the bare atoms. Our work offers promising prospects to simulate the dynamics and interactions of novel excitations in quantum systems with dynamical gauge fields.