Regression-Testing

Automated regression testing ascertains code changes and functionality issues

Automated Regression Testing ascertains code changes and functionality issues. In other words, it is a quality measurement check to discover if new code complies with the old code so that the remaining unmodified code stays unaffected. Automation regression testing also allows for finding any bugs that may have occurred due to changes in the code and if the testing is not done, the product could have a critical issue occur during a live event which can lead to negative marketing impact.

There are various types of automated regression tests and they include:

  1. Unit Regression – done during the unit testing phase when a code is tested in isolation.
  2. Partial Regression – done to verify the code works fine even when the code is changed performed while the unit is integrated with the unchanged or already existing code.
  3. Complete Regression – done when a change in code is in numerous modules and/or if the change impact in any module is uncertain.

It is understood that automated regression testing is hard because for every action performed there is a reaction. A few result in successful tests but there may be another two-hundred that will lead to failure. Unfortunately, there is no one size fits all test strategy for automated regression and shortcuts that are used, have not had consistent positive results. The good news is there are some comprehensive specs, rules and examples that countless software engineers have diligently put together for our knowledge base and application protocol. (Baril, Gounares, & Krajec, 2014)

Regression-Testing

The Reason We Have Automated Regression Testing

Automated regression testing’s intent is to speed things up, so we can increase quality and velocity simultaneously which results in obtaining the prize of all promotional tools – being the first to market. It doesn’t matter if you are releasing a new software suite, software feature or even if you wish to make sure a particular software feature is current and working properly, there are steps to take, rules to follow and regression automation tests to conduct. Common practice is to utilize a suite of four regression automation tests which perform in an exemplary manner. They are:

  1. Retest all and repeat frequently – the entire test case in the suite are re-executed to ensure there are no bugs from a change in the code. This regression test is expensive due to its expansive nature and requires more time and resources than any of the other types of automated regression testing methods.
  2. Selection testing is worth using for maintenance – test cases are selected from suites to be re-executed. The test cases are selected from code changes in the modules and have two categories – reusable and obsolete. The reusable test cases can be used in future regression cycles whereas the obsolete ones are not use in future regression cycles.
  3. Prioritization to create stability – priorities are created, and the test cases depend on the listed and needed priorities to be for product impact and functionality.
  4. Simple – a combination of regression test selection and test case prioritization. Rather than selecting the entire test suite, only test cases which are re-executed and are listed as a priority are designated.

It should be noted automated regression testing not only enables checks into various changes but can also release and prompt testers to conduct manual assessments in the more uncommon cases respective to their unique production environment.

Stakeholders are usually willing to accept automation regression testing as being a part of the final analysis of ‘completion’ for user stories being worked on and evaluated. User stories are only closed when the corresponding automated tests were run effectively and efficiently and had successful outcomes. When the feature is successfully released into production, the regression suite becomes part of the tests. In layman’s terms that means there is a stable version of tests which now exist as part of the regression suite-built layer by layer and are available whenever development of a new feature is added. (Briand, Labiche, & Soccar, 2002)

However, there is a more difficult automated regression test to perform which occurs when a feature was released into production without having any automated tests performed. The challenge then becomes finding a regression suite to put into place since you can only do that incrementally, layer by layer so prioritization is mandatory to ascertain what must be tested.

Automated Regression Testing Tools and Time Savers

There are various tools that can be used in automated regression testing which combine testing with functionality in a single platform and a couple of popular ones include Selenium and vTest. However, there is a sidebar that needs to be considered and understood when using automated regression testing tools. The implementation of the tests are faster than manual tests but be cognizant that everything else will take significant time, so preparation is the key. What does that mean? It means that writing the tests and setting up the suite needs must be prioritized, listed and understood. To help save some inefficient use of your time, we have listed some automated regression testing time savers. (Raulamo-Jurvanen, 2017)

  • Try to write individual and independent tests because you will ultimately regret it if you don’t. By not writing individual and independents tests, if an issue arises, you will find because you did not write an independent test, your solutions are problematic and must work around test orders and the storing of state and data between test runs.
  • Separate acceptance and end-to-end tests because they do entirely two different things and need to run separately to get proportions correct. Acceptance tests target and test one thing efficiently and effectively. An end to end test is implemented to cover the user’s journey through an app and then test the app the same way a user access it. The end to end tests do take more time and are considered fragile because they contain so many incremental steps.
  • If you want your test to perform brilliantly, decipher why you are doing automated testing and once you ascertain need, determine what measurements will be needed. Your end goal should be to have as few automated tests as possible and only use them for valid and objective business reasons because automated tests are time-intensive and costly.
  • Never forget that intention and implementation are two different things. When writing scenarios, it is logical to input how best to implement the set-up, but that thinking is faulty and will not help longevity within your specifications or enhance business readability. Intentional features and scenarios provide outcomes that are clear and easy to understand and if you really want to provide exemplary solutions you can even build in the ability to change your test, if needed without changing your specifications.
  • Automated regression testing is not a one shot and you’re done deal because if you don’t run them on a consistent basis, they will become almost useless when someone changes code. The test should be in the same source control repository as the app, so they will not be forgotten about or ignored.
  • Automated tests should never be run on several browsers because almost every browser performs differently with slight variations which invalidates true results. In essence you are probably wasting your time. Try to find the browser most of your customers will be using. Google Chrome is usually a good place to start.
  • There are nuanced differences in manual and automated testing. This sounds like a no-brainer but it’s not. Automated testing is the testing of choice for functionality, but it does not do well in testing stories or exploring your system. Automated, artificial regression testing no matter how brilliant, logical or error-free, rarely understands weird quirks or cultural definition variances. But humans can find those unique perspectives and manually test them which is more cost efficient and allows for fine-tuning for human users’ needs.
  • Try to run your automated tests as they incrementally grow and develop to speed up your run times. It takes almost no time to create an agent to run tests and collate the results on a consistent loop integrated with the testing server.
  • Use, use and use your application development team because each member should be accountable for quality. The most effective and efficient automated tests are developed with the application development team because they integrate what is needed with what can be tested with the results being successfully magnified.
  • Try to find any opportunity to extract the most value for the least amount of time and energy. If you have to keep running automated end to end testing after deployment of product, is it a good use of a company’s outlay? Always seek value in every level of testing. Always.

Automation regression testing is one of the most important aspects in helping deliver a quality product because it makes sure that any change in the code whether it’s small or large does not affect the existing or old functionality. You can even use automaton to create master data and setup configurations for manual testers which ultimately allows you to facilitate the needs of the various operations within your company or organization. There will be new tests in automation with techniques discovered and challenges to solve. The journey to achieving optimum levels of automation in regression testing must start by taking the first step.

Discover more on Regression Testing

Learn More

Bibliography

Baril, B. B., Gounares, A. G., & Krajec, R. S. (2014). Automated Regression Testing for Software Applications. Retrieved 10 12, 2018, from http://freepatentsonline.com/9355016.html

Briand, L. C., Labiche, Y., & Soccar, G. (2002). Automating impact analysis and regression test selection based on UML designs. Retrieved 10 14, 2018, from http://ieeexplore.ieee.org/document/1167775

Raulamo-Jurvanen, P. (2017). Decision Support for Selecting Tools for Software Test Automation. ACM Sigsoft Software Engineering Notes, 41(6), 1-5. Retrieved 10 12, 2018, from http://dblp.uni-trier.de/db/journals/sigsoft/sigsoft41.html

Tan, S. H., & Roychoudhury, A. (2015). relifix : automated repair of software regressions. Retrieved 10 16, 2018, from http://dblp.uni-trier.de/db/conf/icse/icse2015-1.html

LIKE THIS POST SHARE IT WITH YOUR FRIENDS

Talk to our Test Engineers

Fast track your ecommerce monitoring

Request a Demo

Recent Posts

Tagged under: , ,
Continuous Testing

Need To Know the Best Practices For Continuous Agile Testing

Continuous QA

Need To Know the Best Practices For #Continuous Agile #Testing

The Delivery Manager approached the QA Head – When do you stop testing? The Head replied – We never stop testing, it should be continuous we may put a ‘,’ to pause for a while but make sure to pick it again to near perfection. The day, we stop testing, means our product is going to be the history in the market. Even if your product is stable, we need to test it if it works with new browsers, new java version? We need to test it again to see if it could take up a load of new users. We want to deliver quality, and hence Continuity is the key, we do not deliver use-n- throw software.

That’s the power of Continuity!

So how do manage continuity? Is Continuous Agile a tough game? Let’s a take a sneak peek into the best practice’s that could help you gain control of Agile and keep it continuous –

Know Your Customer

Knowing your customer mindset is of prime importance, just for example if you are testing a bank app – How easy it would be for the user to log in who a layman is? In case he forgets his password, do they have enough information to retrieve his password? Or what kind of information could hacker use to hack the account? Is the password field allowing a ‘copy function’?

Get into the shoes of the customer and then design the test cases/test scripts.

Integration and Automation is the Key

For testing to be Continuous Agile, every piece of code checked in needs to be included in the build. The Build Automation needs to be integrated with Automated Deployments, Test Case Execution,  Regression Results and Bug Tracking.

Automate Test Cases TestAutomation

Manual Test Execution is a thing of past! Automate your test cycles to speed up the processes and widen your coverage.

Execute Smoke Test post Deployment

A test suite consisting of some bare minimum test cases in necessary for QA to start on Functional, Regression Testing to ensure the environment stability.

Convert Business UseCases to Test Scripts

Get in touch with the product owners/business users and try to get the use cases that could be elaborated to test scripts.

Access Your Software Risk Appetite RiskBasedTesting

What could be critical for a business user? What are the features that for software should work seamlessly and any issues in that could be disastrous? Weigh your features as per the risk associated with them and devise a Risk Based Test Suite. Either you could assign a number from 1-5 [ 1 being lowest and five being highest regarding risk] to your testcases and then could create a suite. Or you can get help from your business users/ top level professionals to assist you with that.

Adopt Lean Testing LeanTesting

Lean Testing is a concept where the waste is identified at an early stage to maximize quality. Just, for example, your banking app gets a new feature – as a new icon. As a tester now how would you design your test cycle? Does it need to have all the functionality tested or may be few would do? Lean Testing is a process through which you can not only speed up your testexecution but also identify the processes, delays that could hamper your continuous Agile process and provide guidelines for improvement.

Follow Standards, QA Processes, Testing Pyramid

Processes are meant to help you achieve faster and transparent results, so it should be a good practice to stick to them. To work on the test execution, you could also take help from the TestPyramid model and based on your risk appetite could work on UnitTesting, UITesting TestAutomation or IntegrationTesting.

Change is Constant

Charles Darwin said – “It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change.” Hence, there would be last minute changes, fixes, builds and processes, the important point is you being prepared for it?

Give some time to Exploratory Testing Exploratory Testing

You cannot automate exploratory testing but is an integral part of any QA cycle, allocate some time for its manual execution.

Continuous Agile Testing is a process that could provide you with maximum throughput when you are Continuous. As the quote says –

A phased approach to continuous delivery is not only preferable; it’s infinitely more manageable.”

– Maurice Kherlakian.

LIKE THIS POST SHARE IT WITH YOUR FRIENDS

Recent Posts

Tagged under: , , , , , , , , , , , , , , , , , , , ,