| |
JOpera: Visual Composition of Grid Services
by Cesare Pautasso
The recent shift to service-based Grids enables the use of service composition tools for rapidly building and efficiently running distributed computations. At ETH Zurich, we have developed JOpera, a visual composition language and a set of integrated software development tools for composing Grid services. The system can be freely downloaded and has been successfully tested with data-intensive bioinformatics applications as well as large-scale, Monte-Carlo network simulations.
Early Grid middleware and tools emphasized low-level resource management aspects. Today, Grid computing has adopted the notion of ‘services’ as a basic building block for large-scale scientific computations. The focus has moved from CPU sharing to defining computational resources and functionality as composable services. Thanks to standards such as Web services, an increasingly large number of basic Grid services are being published. They encapsulate both algorithms and data sources, hiding the complexities and peculiarities of the underlying platforms. With services such as basic building blocks, it becomes possible to rapidly build Grid applications by composing these services at a high level of abstraction.
At the Information and Communication Systems Research Group at ETH Zurich we have developed JOpera, a system for rapid service composition based on a visual composition language. With JOpera, the data transfers between services (data flow), their order of invocation and the necessary failure handling behaviour (control flow) can be specified with a simple, graph-based, visual syntax. As opposed to an XML-based approach (eg BPML, BPEL4WS, XL), a visual language can greatly en-hance the understandability of Grid computations, as they can be specified by literally drawing them. Furthermore, at run-time, they can also be debugged and monitored using the same visual syntax. In addition to nesting, recursion and reflection, the JOpera Visual Composition Language offers several iteration constructs specifically targeting Grid computing. For example, large computational jobs are typically partitioned into smaller units that can then be executed in a pipelined, parallel or sequential fashion. JOpera includes simple primitives for these operations and allows the programmer to precisely control what happens in the case of failures.
JOpera can also suggest the most optimal partitioning strategy based on the available resources.
JOpera offers an open, flexible platform for Grid service composition. It maintains a library of re-usable components, which can be dragged, dropped and connected into a data and control-flow diagram. JOpera makes very few assumptions on the properties of these components, so that the user can freely choose to compose the most appropriate kind of services in terms of per-formance, reliability, security and convenience. Such components may represent the invocation of basic, remote Grid services, but also, for example, job submissions to external resource management and scheduling systems or the execution of local applications under a variety of operating systems. Additionally, composite Grid services can be re-used in two different ways. On the one hand, Grid computations can be decomposed hierarchically into different modules, which can be invoked and re-used independently. On the other hand, composite Grid services can define re-usable patterns of interaction between services, which can then be customized and tailored to specific applications. To do so, service interfaces are bound to actual Grid sites providing compatible services at deployment or invocation time.
 |
| Architecture of the JOpera system. |
For efficient execution, the JOpera Visual Composition Language is compiled into Java code. To ensure the required level of scalability, reliability and flexibility, the resulting code is dynamically plugged into a runtime container: the JOpera kernel. The flexible architecture of the JOpera kernel can be deployed in different configurations, eg stand-alone or embedded into other systems, such as development environments or application servers. An open API is a key feature for providing access to composite Grid services in a variety of ways. In the simplest case, once Grid services are composed, the resulting computation can also be published as a Grid service. To do so, the JOpera kernel manages the execution of the composite service behind a Grid portal. Similarly, JOpera can be embedded into end-user applications, thereby supporting the development of application-specific features built by composing existing Grid services. Additionally, we have developed generic, visual monitoring tools for keeping track of the progress of a Grid computation. In order to handle large workloads, the JOpera kernel can scale across a cluster of computers. We are currently investigating how to apply autonomic techniques in order to extend the system with a controller in charge of determining the optimal configuration of the system based on the current workload.
Link:
The latest version of JOpera can be freely downloaded from http://www.iks.ethz.ch/jopera
Please contact:
Cesare Pautasso, ETH Zurich/SARIT, Swizterland
Tel: +41 1 632 0879
E-mail: pautasso inf.ethz.ch
|
