A high-level programming paradigm for Java-based parallel and distributed applications

Abstract

This thesis proposes JavaSymphony, a novel programming paradigm for wide classes of heterogeneous systems ranging from small-scale cluster computing to large scale wide area meta computing. On the one hand, JavaSymphony supports automatic mapping, load balancing and migration of objects without involving the programmer. On the other hand, in order to enhance the performance of distributed applications, JavaSymphony provides a semi-automatic mode, which leaves the error-prone, tedious, and time consuming low-level details to the underlying system, whereas the programmer controls the most important strategic decisions at a very high level.<br />We have designed JavaSymphony Runtime System as a distributed Java-based middleware to support the execution of distributed JavaSymphony applications.<br />The components of the JavaSymphony middleware are running onto distributed resources and provide basic services needed by the applications such as communication, resource and application monitoring, or code execution.<br />This dissertation presents in detail relevant features of the JavaSymphony Runtime System design. Moreover, we introduce a framework for scheduling workflow applications in JavaSymphony. Our approach in this area differs from similar research in several ways: Whilst in most related work, the workflows are limited to DAGs of tasks, we present a workflow model that includes loops to model repetition in workflow applications, and conditional branches to address non-deterministic behaviour due to data that is available only at runtime. Furthermore, we build a dynamic scheduling strategy that addresses the new workflow elements and we apply this technique to enhance several static DAG-based scheduling heuristics.<br />In addition, the thesis introduces a theoretical framework to describe the functionality of the resource broker, which support advanced features like reservations and dynamic updates of the estimated task execution times.<br />

This thesis proposes JavaSymphony, a novel programming paradigm for wide classes of heterogeneous systems ranging from small-scale cluster computing to large scale wide area meta computing. On the one hand, JavaSymphony supports automatic mapping, load balancing and migration of objects without involving the programmer. On the other hand, in order to enhance the performance of distributed applications, JavaSymphony provides a semi-automatic mode, which leaves the error-prone, tedious, and time consuming low-level details to the underlying system, whereas the programmer controls the most important strategic decisions at a very high level.<br />We have designed JavaSymphony Runtime System as a distributed Java-based middleware to support the execution of distributed JavaSymphony applications.<br />The components of the JavaSymphony middleware are running onto distributed resources and provide basic services needed by the applications such as communication, resource and application monitoring, or code execution.<br />This dissertation presents in detail relevant features of the JavaSymphony Runtime System design. Moreover, we introduce a framework for scheduling workflow applications in JavaSymphony. Our approach in this area differs from similar research in several ways: Whilst in most related work, the workflows are limited to DAGs of tasks, we present a workflow model that includes loops to model repetition in workflow applications, and conditional branches to address non-deterministic behaviour due to data that is available only at runtime. Furthermore, we build a dynamic scheduling strategy that addresses the new workflow elements and we apply this technique to enhance several static DAG-based scheduling heuristics.<br />In addition, the thesis introduces a theoretical framework to describe the functionality of