Turning Process Models into Videos

Abstract

Video generation models have opened new opportunities for simulating business processes through realistic visualizations. However, current video generation approaches often fall short of capturing the inherent dynamics and structure of business processes and tend to produce inconsistent simulations that lack the rigor provided by formal process models. To address these limitations, we introduce a novel method termed Petri Net structure-driven video generation, which integrates the inherent structural information from process models to tailor video simulations more closely to the dynamics of business processes. We explore multiple strategies for this tailoring, including i) the use of domain knowledge-rich prompting, ii) a storyboard employing image references extracted from process evidence data, and iii) generated image references informed by process models. We evaluate our method across diverse domains, and demonstrate that the Petri Net structure-driven approach improves the perceived usefulness and consistency of the simulated video, marking a step forward in the use of generative AI for more realistic business process simulation.