The art of (large) system migration in early 21st century IT : an empirical engineering analysis of large scale migration case studies, their solution concepts and derived reasonable abstractions

Abstract

Governments, organisations and enterprises are facing an ever growing set of challenges stemming from their history of using IT systems to solve their business challenges. Legacy systems are the codified incarnations and the remnant of past technological state of the art processing the majority of all transactions worldwide. However, their main characteristics are inhibiting innovation and growth and excessively consuming resources. Migration is still considered the only sustainable approach to handle the inherent deficiencies without starting from scratch. Despite this, the topic is largely absent from research on software engineering methodology and techniques which instead continue to focus on increasingly improbable greenfield scenarios. This leaves practitioners with a mismatch between theory and reality, which can mean the difference between success and failure in a software project. This thesis represents a long term case study analysing a total of four cases over the course of several years each, spanning almost a decade in total. Each object of investigation was chosen by evaluating it against a set of criteria to ensure that it exhibits common characteristics found in legacy systems. It is then analysed in detail covering both technical and socio-economical aspects. Alternative outcomes are discussed to highlight key decisions which have significantly affected the direction of the respective project. Afterwards a series of cautious abstractions and generalisations are extracted to discuss commonalities and lessons learned. These abstractions are subsequently condensed into topic groups, each representing a result in the form of a central thesis and a set of challenges to be addressed. Together, topics groups and generalisations can be seen as the proposal for a new topology on Legacy System Migration (LSM). The resulting detailed description of each case gives a deep insight into the evolution of each unit of investigation. The challenges that had to be overcome range from organizations that are not prepared on either a technical or an organizational level or both to fighting the urge to chase ``silver bullet'' solutions or the need to clean up after past chases. The ensuing abstractions and generalisations link the experiences from each case and contrast or augment them with the academic state of the art if there is sufficient overlap. Depending on the perspective, they can also be read as lessons learned or cautious recommendations. At the same time these abstractions also serve to highlight the thematic diversity of the challenges facing a LSM effort. The thesis is completed by clustering them into a total of five topic groups ranging from low level technical patterns to high level strategic considerations, forming the proposed topology. Each group delivers a resulting thesis that propagates success factors and a discussion on challenges in the field which need to be addressed in future work. The topology therefore serves as a central medium for communication between researchers and practitioners and at the same time as a navigational aid for the topic of LSM. The conclusion culminates in a call to both industry and academia to collaborate more closely both in research and education. A stronger focus on legacy topics is urgently needed to avoid another legacy crisis. Keywords: Legacy System