You are here

Designing Tests For Smart TVs

The presentation is sharing a case study on designing and execution of automatic tests of Smart TVs. 

Smart TVs have an important place in consumer electronics market with new technologies that they introduce to users. Quality of a product is an important criterion in consumer electronics market as end users become less tolerant to software based problems on their TV systems. As Vestel R&D Design Verification and Test Group, we are performing detailed testing with more test cycles. Test approaches that adopt the automated testing methods provide cost reduction, improving the product quality and effective time and resource usage. However, that is only the execution part and execution tools cannot find defects without effective test design. We are improving test process by using Model-based Testing-MBT (test design) and Test Automation (automatic test execution).

During test automation, so-called “torture tests” are also performed. Torture tests aim at discovering how the system behaves under sustained use. This test helps us to ensure that the product is fully functional after some long period of sustained activity and also helps us to predict how long this system may continue before it functionally breaks. We are trying to predict software lifespan of Smart TVs. Since it is not possible to run longevity test for years for a television as it is advertised with a lifespan of ranging 50,000 hours to 100,000 hours, we execute limited time test with heavy load to simulate long time usage. In this presentation, we provide accelerated longevity test based on daily usage simulation models by using MBT and test automation.

The method can be summarized as:

  • Detecting the mostly used modules of TVs by getting data from real users.
  • Designing a usage model
  • Generation of test cases by a MBT tool. Coverage of each generated test case is 100%, which means each generated test case visits all possible modules detected by usage data analysis.
  • Execution of each generated test case on test automation system. Executing test cases simulates 1-year usage of product in 23 days.

We demonstrate how to use MBT in Smart TV (and any consumer electronics) testing, as well as discuss an effective test design approach for it. We will also tell about automatic execution of these tests.

Key takeaways: 

  • Testing challenges in Consumer Electronics
  • Creating a usage profile as input for Model-based Testing
  • Designing test models for consumer electronics products
  • Automatic test case generation with Model-based Testing
  • Automatic test execution of automatically generated test cases