This report explores connections between finite element methods (FEM) and neural networks in the context of function approximation and solving partial differential equations (PDEs). Neural networks are viewed as flexible function approximators, offering an alternative perspective to traditional polynomial-based methods.
Their approximation behavior is discussed in relation to function spaces such as Barron and Sobolev spaces, with some convergence properties outlined. A related approach, referred to as the finite neuron method (FNM), is considered as a linear formulation using neural network-based basis functions, allowing comparisons with classical FEM and suggesting potential benefits in higher-dimensional problems.