Lattice Gauge Equivariant Convolutional Neural Networks (L-CNNs) leverage convolutions with proper parallel transport and bilinear layers to combine basic plaquettes into arbitrarily shaped Wilson loops of growing length and area [1]. These networks provide a powerful framework for addressing challenging problems in lattice field theory. In this talk, we explore the inner workings of L-CNNs, aiming to gain insight into the contributions of the different layers. Through visualization techniques, we analyze the patterns and structures of the Wilson loops that emerge, studying to what degree L-CNN architectures exhibit redundancy in the parameters. With our findings we aim to provide a deeper understanding of L-CNN behavior and improve its interpretability.