EUREKA-ITEA solution enhances European software testing capabilities

The EUREKA ITEA Cluster TT-Medal project has achieved a major breakthrough for the European electronics industry by developing a generic solution to enable automated testing of software systems. The methodologies and tools developed in the project were validated in industrial-scale demonstrators for automotive, railway, financial and telecommunications applications, proving the feasibility of a significant improvement in test efficiency, effectiveness and product quality, leading to significant cuts in testing costs. As a result, the TT-Medal project provides a unique opportunity for European suppliers and consultants to position themselves better in a world market previously dominated by the USA.

Software increasingly provides the core functionality of electronics products – from mobile phones to railway signalling. Testing is fundamental to developing software-intensive systems as the sooner errors are identified, the cheaper they are to fix. But testing is extremely difficult to achieve exhaustively. There is also a serious shortage of software developers in Europe, slowing time to market.

Generic tools

TT-Medal developed generic automated testing methodologies and tools based on the TTCN-3 testing language from the European Telecommunication Standards Institute (ETSI) that enable systems testing from beginning to end, using common tools. This makes possible reuse of test ware between different phases of a product’s lifecycle – from initial simulation at the design stage to regression testing during maintenance – and also saves on training.

An added advantage of the internationally recognised Testing and Test Control Notation (TTCN) language is that is driven by Europe. It can be used for many applications, including mobile communications, wireless local area networks (LANs), digital cordless phones, broadband technologies and Internet protocols. It is more productive, powerful, flexible and extendable than previous approaches, as well as being easier to learn.

The TT-Medal consortium consisted of 11 partners – including telecommunications manufacturers, software test tool suppliers, software test consultancy firms, academia and research centres working in the testing research domain – from three European countries, and was co-ordinated by Nokia in Finland. It set out to develop the tools and methodologies needed to turn the TTCN language into a robust product applicable to many different application domains – from transport to finance – and introduce it into a much wider range of industries.

Three application areas were selected to validate the approach:

1. Transportation – telematics for information and entertainment systems in cars and interlock subsystems for railway signalling and control;

2. Telecommunications – 3G radio access network operation and maintenance, GSM mobile terminal location, and 2.5G and 3G mobile module integration; and

3. Finance – integration of TTCN-3 on both user and application sides of financial distribution systems testing.

While there are differences in specific requirements between applications, many issues are common. All industrial sectors are searching for a universal testing language to combine the different technological areas that need to be tested, although each end-user domain has to be responsible for its own test scenario specifications.

Showing real savings

TT-Medal has done much to develop methodologies to enable European industry to test software effectively and efficiently. It has also spread awareness of the TTCN-3 language and its potential applications into new areas. In addition, the case studies played a vital role in disseminating the language far beyond ETSI and the conventional TTCN community.

Software development is an increasing important part of European industrial product development, points out TT-Medal project leader Dr Colin Willcock of Nokia. “Testing requires 25 to 50% of software development resources,” he emphasises. “TT-Medal was therefore not an academic exercise. We showed the real benefits of TTCN-3 in our demonstrations. External evaluations indicate that European industry could make up to 50% savings in testing costs.

“Contracted software development in western Europe cost €66,571 million in 2005. Assuming conservatively that 25% of development costs are in testing, this 50% reduction in testing cost would save European industry over €8,000 million a year!”

The project has also benefited both software tool and test device vendors in Europe. The former provide the general test infrastructure for TTCN-3, while the latter supply the specific hardware and interface adaptations necessary to access the devices being tested.

In addition to developing a generalised test platform, the specific research in transport, telecommunications and financial applications resulted in significant advances in test case reuse, validation and automated generation. Moreover, the combination of formal techniques such as data abstraction and constraint solving makes it possible to automate test generation for real industrial-size systems.

Wide dissemination

TT-Medal involved 67 person years and a budget of €9 million. The results were disseminated through an impressive number of papers and publications, as well at several conferences, and laid out in a test training package that includes a book on TTCN-3 testing.

The project also won the ITEA 2005 Achievement Award. “To achieve our objectives, we needed to develop proven solutions including both tools and training. Working with EUREKA and ITEA allowed us to build a consortium of very disparate companies,” explains Dr Willcock. “The results of TT-Medal should enable small and medium size enterprises (SMEs) to build new testing tools that will put Europe on the map for testing. And winning the ITEA Achievement 2005 award should now make exploitation easier.”

Project participants:
Finland, Germany, The Netherlands

Budget:
€10 million

Duration:
33 months