|
 |
|
|||||
|
|||||||
|
The CORVAL2 Contribution to achieve Confidence in Middlewareby Ina Schieferdecker, Axel Rennoch and Dorota Witaszek The European Commission has initiated the development of a conformance test suite for the validation of Common Object Request Broker Architecture (CORBA) infrastructure products. Investigations of enhanced techniques for CORBA validation is the core of the CORVAL2 project. Any software user knows about in-stable software programs. Today we are software users in many life situations: eg, if we are customers of a (real or e-commerce) shop or travel agency, users of any communication system, players of electronic games, and even passengers of a (private or public) transport vehicle. The utilization of this software provides benefits and comfort to its users, as long as they trust the system and/or services. It is not necessary to go into psychological details to understand the increasing stress if software behaves different to its declared expectations, or more formally: if the software does not conform to its specification. What gives a user the trust in software? First of all, there will be her experience of availability and reliability. But, what if she is lacking these experiences? The situation may be even more difficult: what do users perceive and think, if they are exposed with a particular application, eg in the Internet? In almost all cases there is a complex system behind, which is layered and often distributed with a set of different services involved. And it seems obvious that end-user may avoid an application, if it is embedded in a faulty system. That leads to the view that middleware, which is the ability of computer programs to make use of any service implemented on heterogeneous hardware or operation system, is a technology which is crucial for software applications and needs to be verified carefully. In the telecommunication sector it is well accepted that the efforts for testing may cover up to 50% of the complete development process, since the service reliability is an extreme critical point. During the last decades an elaborated Conformance Testing Methodology Framework (CTMF) [ISO/ITU-T and ETSI 1997] has been developed and established in this context. But its applicability appears to be much more general. In January 2000 the EU commission started funding the R&D project CORVAL2 to enhance confidence in software quality in the realm of middleware. The target of this initiative is the conformance of the Common Object Request Broker Architecture (CORBA) based infrastructure for interworking in computer networks towards its international requirement specification [CORBA standard V.2.3, OMG 1999]. The idea is to enhance software reliability due to the application of high-qualified technical tests. Due to the nature of the CORBA middleware, any conformance testing of the CORBA infrastructure will focus on the interaction between a service user or provider and the CORBA infrastructure. Further, the communication within the CORBA entities across soft- and hardware boundaries is subject of testing. The major testing aspects in CORBA are:
Obviously, these aspects cover a large field of conformance aspects and testing technologies: operation-based interface testing, compiler testing, and protocol conformance testing. The major outcome of CORVAL2 is the availability of the conformance test suite for both C++ and Java API language binded CORBA ORB implementations. The total amount of test cases is very large, it varies on the conformance testing aspect to be verified: the numbers for the C++ test suite are for example: 400 syntax tests and 380 dynamic behaviour tests with reference to the IDL compiler output, 1240 API declaration tests and 470 API semantic behaviour tests, and about 150 tests on GIOP, the protocol definition for the inter-ORB communication. The different numbers are due to the different testing purposes. The high number of API declaration tests, for example, is reasonable due to the various features an API provides to an user (member functions, class inheritance etc.). The beta version or the test suite is available for inspection and download via the CORVAL2 project web site. Due to the IPR ownership, the test suite is publicly available, but the Open Source License does not apply. In CORVAL2, the test suite has been applied for the different CORBA ORB products, which are available among the project partners. Some failures have been identified during this test campaign, eg, incorrect extraction/insertion of CORBA: Any types or wrong GIOP codings.
The big volume of the test suite and the level of details of the tests give reasons to trust the quality of the System under Test (SUT) which have passed the conformance tests. But this technical argumentation might be sufficient for the technical engineers only and not to the end user of CORBA middleware. Therefore a Brand program has been started in October 2001 to give vendors the opportunity to prove the application of the tests to their CORBA ORB product. Branded ORB products will get a certification document and logo for promotional usage. If any deficit has been discovered within the SUT (due to the application of the conformance tests) the granularity of the test suite allows identifying and correcting the failure. The partners of the project are: The Open Group, Reading, UK; Fraunhofer FOKUS, Berlin; IONA - Object Oriented Concepts, Karlsruhe, Germany; Fujitsu - ICL, Dublin; Eric Leach Marketing Ltd., London; Object Management Group, Inc., Needham, MA, USA. Link: |
