Software Certification: Myths and Reality
by Fabrizio Fabbrini, Mario Fusani, Vinicio Lami
Certification is treacherous stuff. People tend to have contradictory attitudes. Some want to have a full guaranty that a given product or service is reliable, others consider the term and all that it concerns as deceptive and used only for marketing purposes. And when it comes to software, professionals may be found near to hysteria, insisting that the concept should be excluded from the technological, if not the commercial, realm. Despite all this, practitioners, scientists, and even users, can coexist happily with software certification, if only they take a positive view of the concepts, actions and entities involved. The problem is that many people talk and assert their opinions on certification without really knowing what it’s all about. Our aim here is to define the area of interest and show that nothing dramatic happens when software is involved.
Certification can be described as an act by which an accredited third party declares that a defined entity (product, service, process, characteristic) is conform to a standard.
For sure, we could explain ‘accredited third party’, ‘conform’, ‘standard’, and why ‘declares’ is used instead of ‘demonstrates’ or ‘guarantees’, but this is not necessary to get our point over. The point is that it is not a matter of demonstrating nor guaranteeing, but of giving confidence, and this implies the involvement of some ‘user’ entity in the certification process. The accredited third party is needed to transfer confidence to users, but this implies other concepts such as ‘third party liability’ and raises questions such as: ‘who accredits the accrediting organism?’. Such concepts are solved in the uncommented Figure.
And now for software. The idea that some concept of guaranty could be lurking in the word certification has actually been banished from the expression software certification as used in technical or commercial software literature. It is well known that all commercial software packages are sold (even though never advertised) together with an inseparable disclaimer by the supplier. But users do not like disclaimers; they actually want some kind of guaranty. We have spoken of confidence as a reasonable (if not full) level of certainty and as an aspect users can accept, but can we introduce guaranty in software certification? Of course we can – it just depends on what is to be guaranteed.
What can be guaranteed is hardly some functional property of the software product. We know that even functional standards for compilers, protocols and graphic kernels cannot describe exhaustive test suites. Even more disheartening is that the certification of specific functional/non-functional properties is unfeasible for general-use software products.
What can be guaranteed in software is that the supplier of a given product has performed a process that includes the proper use of defined standard practices, both general practices and practices for the specific application domain. This cannot do much more than establishing a certain level of confidence that the product suits specific user needs, but it may be all the user wants and needs. This can and should be accepted as a feasible software product certification.
It should, however, be noted that this use of certification must be distinguished from ISO 9000 or CMM certification, which refer to the quality of a system or the capability maturity levels of an organisation. Even if a positive correlation between organisation quality and product quality can be demonstrated, in principle this correlation is weaker than in the case of complete product certification.
A Centre for Software Certification has been established at IEI-CNR which operates within the framework defined above in which certification implies above all assessing whether a product satisfies the given set of criteria adopted when developing the product. The Centre is an evolution of a former service for the validation of software programs for electronic cash registers provided on behalf of the Italian Ministry of Finances. The activity of the centre regards product testing and quality evaluation, and software process assessment.Please contact:
Mario Fusani - IEI-CNR
Tel: +39 050 593 512
E-mail: fusani@iei.pi.cnr.it
