You are here

Väike saal

Partner track: Self-Contained Automated Tests and How to Remote Debug Them

How do you handle your work if you are employed at a fast-growing company where every sprint holds a ton of new features? The code base is growing every day and product coverage becomes a considerable challenge. Even if you solve it by writing autotests on a daily basis to handle the load, there may come a point where you have so many automated tests that maintaining them in their thousands becomes a whole extra headache. How can you keep track of what tests are actually important, and what can you do if they are gradually failing?

At Playtech, we have thought about those questions for many years and tried out different techniques. In this talk, I will give an overview of how we define self-contained automated tests – tests that are running continuously, proving their relevance – and how they can find the exact component version where they got broken. Additionally, I will touch upon how to use remote debug to find the issue so that others can use the same tests in same environment at the same time. In conclusion, we are aiming to design automated tests that can keep up with a fast-growing product.

Key takeaways:

* The idea of self-contained automated tests

* Why it is good to query the production database every night

* Why use a binary search algorithm to find bugs

* How to remotely debug problems so that others can use the same environment meanwhile


Gauge + Taiko: BDD for Web Revived

Does Behaviour Driven Development work? Unfortunately, it usually does not. While many people try to pitch as a way to bridge the gap between stakeholders on the project, many teams fail to communicate their test scenarios with everyone involved.

 Although this fundamental problem of lack of communication can be solved on the organization level, BDD is often used with Cucumber or Robot frameworks. Due to the complexities of these tools, developers and testers stop seeing the benefit in the entire approach of BDD and abundant the practice.

However, recently, Behavior Driven Development has seen a resurgence in adoption, thanks to Gauge framework. With the latest release of Taiko, they create a great combination of communication and testing tool with the use of Gauge+Taiko.

In this talk, we will discuss BDD principles and how Gauge can be used to take Behavior Driven Development to the next level. With Taiko, the audience will learn on how BDD can be taken to the web in few easy steps and to see what needs to be avoided when these tools are implemented in any organization.

Key takeaways:

  • How Gauge can be used to take Behavior Driven Development to the next level
  • How BDD can be taken to the web in few easy steps with Taiko
  • What needs to be avoided when these tools are implemented in any organization

WCAG 2.1 Standards and Accessibility Testing

The European Commission estimates that including all who have a “long-term physical, mental, intellectual or sensory impairment,” one in six people in the EU have a disability - or some 80 million. That is 80 million people who use our webpages, apps, programs and services in a way that is hard for a developer or a tester to imagine, making it maybe one of the most difficult concepts for a tester. 
Looking at accessibility as important quality standard in testing will be in many ways beneficial. Firstly, when designing our products we usually try to make its usability as comfortable as possible, so it is only right, we do the same curtesy to everybody. Population around us is ageing, so we can only expect the number of people using screen readers, and other assisting programs to use IT services to grow. 
Secondly, it is beneficial to the company, as there are 80 million new potential customers in Europe alone to be gained, if your product is accessible. Also guaranteeing it, is good for companies public image.

But if doing the right thing and potential benefits are not good enough motivator, then regulators will probably make it mandatory anyway. In USA for example there is already Section 508 amendment added to American Rehabilitation Act, giving standards to web pages. Also in 2017 there were 714 law suits agents different webpages / web service providers due to lack of accessibility, mainly claiming those pages/services to be in breach of Americans with Disabilities Act. While in Europe currently there is web standards only set to Europa web pages, then many governments, including Estonia are discussing setting accessibility standards as a law. 

So this matter is important today but will definitely become more important over time, making it important for testers to be ready to both understand how to test accessibility  and how to assure it is factor being considered already in development phase. 

That what my talk will do, I will present my research on the matter as well as practical example from my own work experience how to test accessibility and what are the standards that are sufficient to claim something to be fully accessible.

Key takeaways:

  • What is WCAG 2.1 standards and how to understand them.  
  • Why is assuring accessibility important. 
  • How to create a test strategy for accessibility. 
  • How to keep your product accessible over time.

Learning From Bugs

Bugs are great learning opportunities. So how do we make sure we learn as much as possible from the bugs we find and fix? One way is to reflect on what made the bug hard to solve, and how we could avoid this type of bug in the future. For the past 15 years I have written down short descriptions of the trickiest bugs I have solved. I have included the symptoms of the bug, the steps I took to find it, the fix, and the most important lessons I learned from it. This simple method has helped me distill patterns that have influenced how I write, test and debug code.

In this talk I will share my experience with this method. I will give examples of tricky bugs I have encountered, and show what the corresponding bug entries look like. I will also present the most important lessons I have learned from going through over 200 such bug entries. The lessons include rules for effective testing, and useful debugging techniques and heuristics.

Key takeaways:

  • Present a simple technique to maximize learning from bugs, one that everybody can start using right away
  • Describe several testing rules and heuristics that have resulted from the bug entries
  • Detail the most useful debugging techniques I have learned for hard-to-find bugs

Never Assume Non-Testers Cannot Test

Some test professionals are of the opinion that “only testers can test” - I am not; I say that testers are specialists who can share their skills with others and in doing so promote respect for the testing profession.

You’re in a test team, you work on test projects and for your work you receive a wage and maybe bonuses, promotions and other benefits. So why involve non-testers (or less experienced testers) in the process? Maybe it’s a necessity for a beta release programme, perhaps it’s about a crowd-sourced testing project, or a occasion such as an exercise for a team away day to bring some new energy into the team.

From my experience of crowd-sourced testing and utilising the assistance of non-testers I would like to share critical factors for success:

  • When to use non-testers to enhance your test activities
  • How to engage and properly incentivise your non-tester assistants, whilst minimising “gaming” of the system that you put in place
  • Routine aspects of testing that need to be communicated - what is a bug? how do you report a problem? how do you take a screenshot on an specific mobile device? where do you find log files?


Key takeaways:

Testers have specialist skills but can utilise non-testers, or remote testers, to help with testing. Guide them instead to help with test activities and increase appreciation of the testing craft.

  • Using non-/remote testers to help with testing requires careful planning, incentivisation & support
  • Properly incentivised helpers can enhance and improve the testing process
  • A framework that can be taken into the workplace and used to start gamifying straight away
  • Involving non-testers in testing activities garners intra-team respect for the work that testers do

Build Your Own Internet of Continuously Delivered Things

We all know that a modern tester should have both analytical skills and basic knowledge of programming as well as soft skills. If we add experience in agile software development, we get an almost perfect candidate in the eyes of employers. However, what if one day your employer says: - Our sensor has been recently working too short. - Please, do the battery life tests. - Current functionalities of our product will be enhanced soon. Please, write some tests for the embedded platform. - Clients are complaining about our system working too slowly. Please, prepare the testing environment for performance tests of microprocessor memory. 

Nowadays, everything tends to deliver high-quality products in a continuous way. How to test an application for your customer with so many tools, equipment, and ecosystems combining HW, FW, mobile devices and complex backend architecture? Over the past few years, I've already gone three times through the transformation from "QA Team" thinking to "Dev/TestOps" mindset in IoT companies.

During my presentation, I'll show you how to implement a multi-node system (HW/FW/SW) in the environment of Continuous Integration and (finally) Continuous Delivery. I will start with a big picture of a centralized test environment. After that, all components will be separately analyzed to present possible solutions and implementations. I will also show how to involve manual testers and let actively participate in the environment evolution. All of that supported by Jenkins tool, Python language, BDD approach, and some HW knowledge.

Key takeaways: 

  • General understanding of Continuous Delivery solution in IoT products context
  • The real implementation of centralized test automation infrastructure
  • Understanding that (because of real devices) not everything can be dockerized/moved to the cloud
  • Overview of how manual testers can participate in building CI environment

The Road To QA - From Developer To Test Automation Engineer

The Abstract: 
In my career as a software developer I have seen a lot of QA talents transition towards test automation - some of them even turning into developers. My journey was the other way around - from a developer into test automation. In this talk I want to show why this was a natural choice for me, what I learned along the way and why this changed my perception of developers and QA alike.

Key takeaways: 

  • How being a Test Automation Engineer combines the developer and QA roles
  • How to not lose sight of coding
  • How questioning one's career path can be a huge benefit 

9 Ways to Test Your Spaghetti Code

“Test the legacy code as well” has been a mantra for many years now. But how do you actually do that? When stuck with tangled legacy-spaghetti, it may be hard to see the way out. The path from struggling with your spaghetti into doing TDD is shorter than you think.

It's so easy to say that you should test code as you change it, now matter how legacy, but in a real-world project, you need to know some tools and techniques to be able to do that.

Many developers out there struggle with the impression that testing and TDD cannot work on their project. I’ll challenge that view and hopefully prove it wrong for most participants, and share the techniques and tricks I’ve used.

Key takeaways: 

  • Good design improves testability
  • You can get legacy code under test, although you sometimes must make some trade-offs
  • TDD is suitable for a legacy project

From Zero to Test! in 10 Minutes. Don’t Let Your Test Infrastructure Be Your Bottleneck. A Practical Report How We Avoided It.

It does not matter, if you work in a waterfall or agile working environment, at one point the focus of continuous improvement (you do improvements, right?) becomes your test infrastructure. It needs to grow, evolve and scale with you, otherwise it will slow you down and become a bottleneck.

I had the opportunity to observe and participate over three years how a company changed their test infrastructure in terms of overarching concept and underlying technology base.

In this talk I focus on “test data” and “test environments” as two enablers in an agile working culture. Let me tell you, how a company of over 25+ agile teams used a mix of central control & guidance and team autonomy & freedom to establish a fast and reliable infrastructure for every team member.

If you want to know, how you could get “From Zero to Test! in 10 Minutes”, visit this report from the trenches.

Key takeaways: 

  1. Why and how to structure your test data
  2. Why and how to structure your test environment
  3. Get insight in a practical approach which works for multiple teams
  4. Get an overview how a central change process can co-exist with team autonomy
  5. Learn from a real-world example with more than 30+ teams who uses this approach
  6. Learn which departments/skills should be involved to get an efficient setup

Behind the looking glass: A meta-talk on the reality of creating embedded test ecosystems for ASIC HW Functional Verification

Testing is often perceived by product developers as a non-creative activity.

​These developers are surprised when they discover the amount of work that goes into creating, growing, and maintaining the testing ecosystem. For the functional testing of complex ASIC devices this ecosystem involves almost all aspects of new product development: hardware, electronics, firmware, software, tools, architecture, reverse engineering, hacking, and innovation, as well as all the usual soft skills for teams and team members to communicate and work well together. The ecosystem's components follow typical product lifecycles just as the 'customer' products do.

​In fact, the development of embedded testing ecosystems mirrors most aspects of 'customer' product development.

​The surprise of the uninitiated developers, when they realise what is going on behind the scenes, would have been charming and amusing, had it been only just a rare occurrence. Unfortunately it is so regular that it is time to raise awareness: embedded testing is complex, it is creative, and it can even be liberating from the constrains of 'regular' product development.

​In this talk I will challenge the negative image perceptions of this interesting field and attempt to dispel them. I will use real life examples, drawing from my experience, and I will explain how my team has organised our embedded testing ecosystem. I hope this talk shows that embedded test development, when done correctly, can be just as complex and rewarding as any product development.

​Key takeaways: 

  • Developing embedded test systems is creative and rewarding.
  • The development of embedded test systems is equivalent to customer product development.
  • A presentation of different components involved in an ASIC embedded functional verification ecosystem

Building Your Own Api Testing Framework with Pytest and Requests

APIs lend themselves extremely well to test automation. There's a simple reason for this: APIs are designed to be consumed by computers, not by people. So where with UI tests, you need to rely on testing-specific libraries such as Selenium WebDriver, with API testing you can simply use the same libraries as the ones that are used to build applications. This also means that building an API testing framework is significantly easier than building one for UI testing. So why let's not just do that in a tutorial?

You'll be able to experience how easy it is - assuming you have some very basic general programming knowledge and know what an API is. We'll build our framework using Python, Pytest and the requests library. More importantly, we'll explore and discuss what your framework needs to be able to do to support the full life cycle of a test suite. Because I see all too often a focus on how easy it is to build tests and little mention of maintainability or readability of test results.

After this tutorial you'll have a basic API testing framework you can extend (I will be giving tips for next steps). More importantly you'll have a better understanding of testing frameworks in general. And that will come in handy when you have to evaluate a testing tool, or when you are discussing a framework with the developers that build it.

​Key takeaways:

  • How to make your framework support the full life cycle of a test suite (inc. menomonic)
  • Basic programming skills applied to Python, pytest and the requests library
  • Inspiration to build their own tests and frameworks through programming

Lightning Talks

Lightning talks have become a Nordic Testing Days tradition where every participant can grab the mic and give a 5 minute talk on a topic (preferably related to software development in some way). Sometimes speakers also join to get in another talk. Sometimes giving a lightning talk is the first step in someone’s journey of becoming an international speaker.So come along and speak or rant on a topic close to your heart. The rules are very simple:

  • Sign up on a poster with your name and predicted topic (you will find it near the common area).
  • Prep some bullet points to get through the talk.
  • 5 minutes is the maximum length.
  • Audience may ask some questions but it typically turns into a relaxed conversation.
Subscribe to Väike saal