Query Answering without Join Computation: An Interactive Exploration of Practical Techniques

Abstract

The explosion of intermediate join results, frequently encountered in analytical and graph queries, remains a major performance bottleneck in modern database systems. Yannakakis' algorithm enables efficient processing of acyclic conjunctive queries by avoiding the materialisation of tuples not contributing to the final output. Despite its theoretical guarantees, Yannakakis' algorithm and its adaptations are still largely absent from modern query engines. In recent work, we have integrated Yannakakis-style query processing into Spark SQL via the extension of the query optimiser on the logical level and the introduction of a new physical operator: (Group)AggJoin. In this demonstration, we present Spark-Y, a system for exploring and analysing Yannakakis-style query processing in Spark SQL. The system visualises query hypergraphs, join trees and execution plans, providing side-by-side performance comparisons between the standard and optimised execution. Spark-Y serves as a practical demonstration of the performance gains achievable through this approach, as well as a tool for analysing the behaviour and effects of optimisation rules in Spark SQL. Beyond performance analysis, interactive visualisations provide an intuitive understanding of how Yannakakis-style query processing works in practice.