Automated Functional Testing - Why you should seriously consider it

What is Functional Testing?

A properly written functional test ensures that the system is functioning exactly as the USER expects it to. Functional tests target business goals and typically defined and validated by an expert end user. Since it may not be possible for an expert to test the functionality of the system at each and every step of development, the test cases/scenarios are often documented with their help. This document defines the step by step instructions of system interactions along with the expected outcomes for every test case defined by the expert users. It can then help any user (not just the Expert) to be able to follow the steps to test the software and verify if the outcome is exactly as the expert has defined. To ensure completeness, functional tests should cover both the positive and negative scenarios.

Challenges:

Manual functional testing can be a tedious process as it can take quite a while for a tester to step through each of the steps and verify the outcome/results. When the user goes through these steps each time there is a software update, there is a high probability of erroneous information being entered or captured by the tester resulting in incorrect validations. If the number of test cases is extremely large and complex this problem gets magnified. The manual testing process can, therefore, be extremely laborious, monotonous and error-prone and requires highly dedicated and qualified resources to execute.

This can be overcome by automating the functional tests as much as possible. Granted that not every test can be automated, but even a 50% coverage via test automation can bring about huge relief in time, cost and resources. Once a test scenario is “automated”, it can be run on demand after each software release cycle. The feedback from an automated test is extremely fast compared to the same test run manually.

Consider an example where a test scenario requires a user to traverse through multiple screens, enter data into 20+ fields, select various check boxes, radio buttons, pick a value from a drop-down options, enter date in the right format, validate the response on 5 submit buttons, etc.. A real-life scenario like this can easily take the user over 5-10 minutes each time to run this test. For the same scenario, if the test requires a different data set or a negative test, it means another 5-10 minutes. When you look at multiple data sets and 100’s of test scenarios, it translates to weeks of testing that is required after each major software release. This also causes a bottleneck for the development team to move any further before getting a complete feedback from the test users. If all these test scenarios could be automated, it would take only a couple of hours to go through the tests and feedback cycle to the developers would be extremely fast compared to manually testing all the scenarios. That is a huge saving in terms of time. effort and resources and easily justify adopting functional test automation as much as possible.

A lot of companies, as a standard practice, do not thoroughly test their software application after each minor or major release because of the aforementioned manual testing challenges. The risk that they undertake, however, is extremely high if the software fails in a real-world environment – brand reputation, lost clients, fixes that take too long and that potentially break other parts of the software.

Automation Frameworks

One way to speed up functional testing is through web automation tools.  Selenium is one of the few software testing automation frameworks for web applications out there which is open source and has a very large following.  It has a set of tools to support test automation in multiple browsers.  Web test automation with Selenium, however, is a programming activity and people creating tests should be comfortable with one or more of the few programming languages that Selenium supports (Java, C#, Python, etc.).  Even though it might take some time (a few hours or days depending on the complexity of the test case), it can greatly reduce manual repetitive/regression testing efforts after each software release cycle.

There are a few companies that are working towards reducing the time to automate the tests with Selenium by providing a “wrapper” solution which essentially means “automating the Selenium test automation” efforts.  However, since there are many types of browser interactions in modern web technologies, not everything can be addressed easily.  The goal of a tester should always be to evaluate and use tools & methods that can address the maximum number of test cases with minimum effort.

Automation with CloudQA

There are quite a few testing tools that can assist in automating the functional tests but the challenge faced by most tools is the automation coverage (especially in large software applications). CloudQA has been designed to cover a lot of complex test scenarios. The goal of CloudQA is to cover automation of as many functional test cases as possible with it’s integrated selenium-based codeless regression test automation tool. It starts with a unique ability of CloudQA to discover as many test cases as possible of the software application via an auto-learning process. This, in itself, can save test managers weeks (if not months) of discovering scenarios that can be possibly tested. Besides that, some of the complex scenarios that CloudQA handles right out of the box includes

* Table data access
* Auto Suggest boxes
* Variable storage of a dynamic data set
* File Upload tests
* Data-driven testing (for multiple data sets)
* Various types of user assertions (check, check not, etc.)
* Various actions – click, alert, screen snapshot, wait

If you are thinking about automating your functional tests, talk to one of the CloudQA specialists (email: [email protected]) with it’s