Reproducibility, Verification, and Validation of Experiments on the Marschner-Lobb Test Signal

Abstract

The Marschner-Lobb (ML) test signal has been used for two decades to evaluate the visual quality of different volumetric reconstruction schemes. Previously, the reproduction of these experiments was very simple, as the ML signal was used to evaluate only compact filters applied on the traditional Cartesian lattice. As the Cartesian lattice is separable, it is easy to implement these filters as separable tensor-product extensions of well-known 1D filter kernels. Recently, however, non-separable reconstruction filters have received increased attention that are much more difficult to implement than the traditional tensor-product filters. Even if these are piecewise polynomial filters, the space partitions of the polynomial pieces are geometrically rather complicated. Therefore, the reproduction of the ML experiments is getting more and more difficult. Recently, we reproduced a previously published ML experiment for comparing Cartesian Cubic (CC), Body-Centered Cubic (BCC), and Face-Centered Cubic (FCC) lattices in terms of prealiasing. We recognized that the previously applied settings were biased and gave an undue advantage to the FCC-sampled ML representation. This result clearly shows that reproducibility, verification, and validation of the ML experiments is of crucial importance as the ML signal is the most frequently used benchmark for demonstrating the superiority of a reconstruction scheme or volume representations on non-Cartesian lattices.