From Symbols to Signals: Representation Learning in Conceptual Modeling - A Generic Framework For Learning Semantics and Structure For Modeling Assistance

Abstract

Conceptual models (CMs) are foundational to Information Systems Engineering, providing formalized representations of domain knowledge that guide system design, analysis, and implementation. In contemporary enterprises, the size and complexity of conceptual models are growing rapidly as organizations digitize products and services, decompose systems into fine-grained architectures, and integrate heterogeneous data sources across business units. Currently, the construction and management of these models remain predominantly manual, cognitively demanding, and dependent on human expertise. Recent advances in Artificial Intelligence (AI) and Machine Learning (ML) open new possibilities for augmenting conceptual modeling—enabling systems to learn patterns, semantics, and structures directly from existing model corpora. However, two persistent challenges limit this vision: the semantic gap, where ML models rely primarily on lexical features while neglecting metamodel and ontological semantics; and the structural gap, where the inherent graph nature of conceptual models is underexploited. This cumulative dissertation investigates how AI and ML can learn, represent, and operationalize the semantics and structure of conceptual models to advance Model-Driven Engineering (MDE). Following the Design Science Research (DSR) paradigm, the work develops a sequence of artifacts that progressively bridge these gaps, culminating in a unified framework for representation learning in conceptual modeling. The research unfolds in three interconnected phases. Phase I focuses on algorithmic modularization, introducing a generic multi-objective Genetic-Algorithms (GA)–based framework for optimizing the modular structure of large conceptual models, followed by a Graph Neural Network (GNN)–based modularization approach that learns structural dependencies directly from model graphs. The comparison between GA and GNN-based modularization demonstrates that ML-driven reasoning can generalize structural patterns beyond manually crafted heuristics, establishing the feasibility of data-driven structural intelligence in conceptual modeling.Phase II advances toward the core of this thesis involving semantic representation learning. A systematic literature review on model encodings for ML identifies the need for unified representations that combine linguistic, metamodel, and structural semantics. Building upon this insight, the thesis introduces a GNN-based ontology driven classifier for predicting ontological stereotypes in OntoUML class diagrams. This approach is then generalized from OntoUML-specific node classification into a Graph–Language Modeling Framework for MDE (GLM4MDE) that supports several tasks across several modeling languages and integrates fine-tuned Pretrained Language Models (PLMs) with GNNs on Knowledge Graph representation of models. GLM4MDE learns embeddings that jointly capture natural-language, metamodel, and structural information, supporting tasks such as node and edge classification, link prediction, and partial model completion across modeling languages including Ecore, OntoUML,and ArchiMate. Extensive evaluations on multiple benchmark datasets demonstrate substantial gains in generalization and accuracy over existing ML4MDE baselines, establishing GLM4MDE as a foundational framework for semantic representation learning inconceptual modeling. Phase III explores generative AI in modeling using two contributions. The first contribution builds on the representation-learning foundations from Phase II, integrating semantic and structural signals to assist modelers. It operationalizes the representations of conceptual models, capturing both structural and linguistic signals using Retrieval-Augmented Generation (RAG) architecture that fuses vector, keyword,and graph-based retrieval with Large Language Model (LLM) reasoning to trace the modeling elements with natural language specifications. The second contribution qualitatively explores the role of LLMs as active modeling assistants. Through empirical observation of human–LLM interactions, it identifies recurring collaboration patterns that characterize how modelers co-construct and refine models with LLM support. Together, these works mark a shift from representation learning as an analytical tool to its usein generative and assistive modeling, highlighting how LLMs can transform conceptual modeling from a manual activity into an interactive, semi-autonomous process guided bylearned semantics. Together, these contributions define a coherent research trajectory that advances AIfor CM and MDE along three integrated axes: i) structural intelligence through algorithmic and graph-based learning, ii) structure-integrated semantic intelligence through graph–language representation learning, and iii) interactive intelligencethrough LLM-based reasoning and traceability. Beyond these conceptual advances, the dissertation contributes open-source libraries and the GLM4MDE Python package—providing foundational infrastructure for future ML4MDE research and ensuring reproducibility through transparent benchmarks, code, and evaluation pipelines. In essence, this dissertation charts a path from symbols to signals—transforming conceptual models from static symbolic artifacts into dynamic, machine-learnable representations that enable intelligent, context-aware, and semantically traceable modeling throughout the lifecycle of software systems.