Artificial neural networks (ANNs) are powerful tools for learning complicated, non-linear connections between measurements and targeted observables from data. For instance, in physical experiments, they are widely used to reconstruct properties of a system from the measured data, which is inherently noisy due to technical limitations or fundamental quantum effects, such as fluctuations in the electromagnetic field. This noise ultimately limits the performance of an ANN as it learns properties of the system. An important question to ask is whether this limit can be identified and whether the network can perform at it. For a broad class of physical problems that entail the estimation of a continuous parameter of a distribution from a data set, the so-called Fisher information (FI) determines this fundamental bound. Although this was known for a long time, so far, the FI was only accessible for data following very simple distributions. To deal with the most general cases, where the distribution is complicated and unknown, we introduce a data-driven method for approximating the FI. We collect data from a scattering experiment that is dominated by uncontrolled noise and train ANNs to predict a physical parameter from the data. We then compare their performances with the bound we derived and demonstrate that the physical parameter estimation task can be performed optimally. In a next step, we extend our analysis to the hidden layers of the ANN. Not only does our method allow us to access the FI of the input data but it also applies to the complicated, high-dimensional representations of the data that are generated deep within the network. We interpret the FI as a quantity that flows through the ANN from layer to layer, while improper weights and biases manifest as sinks for the FI flow. Observing the FI during the training us to peek into the black box of the ANN.