Test Automation 102: What It Doesn’t Do
Test automation isn’t the easiest process to deal with. It’s often technical nature, and oft-touted miraculous results lead to a lot of miscommunication and expectation management issues between supervisors and engineers who actually have to work the test. Expectation management is an especially crucial aspect, because both the engineer and his or her supervisor needs to be clear about test automation is, and what it can or cannot do. Now, one of the easiest ways to clear the debris of past expectations is to know what exactly test automation can’t do. This might seem like a redundant exercise, but you’ve only to look at the impact of negative spaces in marketing to realise the importance of this exercise! So, getting down to the meat of the matter, here are some of the things that test automation doesn’t do.
- It isn’t a magical cure-all. One of the main problems with test automation results is that supervisors tend to rely on it to fix all their problems. Reality? It won’t. It isn’t a substitute for bad coding, it won’t be a balm to slow progress and it certainly won’t make a bad piece of software a good one. It simply tests if the work you have done already is stable, and works as you have programmed it to. it won’t fix your UI for you, nor will it make your code stable. You can’t run your code through a program and have it magically fix everything. If that existed, most developers would be out of jobs! The closest you can get is in cases of stress testing, regression testing, and performance testing. Even there, you’ll still have to fix your foundations yourself!
- It isn’t a fix-all alternative to manual testing. Web test automation has a ton of uses and is useful and beneficial to use in a ton of situations. However, there are still some things you need to test manually. For example, test automation won’t tell you things about subjectively human experiences like the feel of a particular software, or its ease of use. The user interface is a particularly tricky area. If you automate too early, and your user interface undergoes massive changes, your costs for script maintenance will go up exorbitantly. Thus, even if you invest in a good automation setup, don’t expect to cut out manual testing altogether. You still need it for areas like UI, installation, compatibility, recovery, and documentation.
- Different automation tools don’t solve your automation problems. This is quite a common practice, especially if decision makers don’t fully understand the limitations of test automation. If you are repeatedly running into issues with your test automation software, the problem isn’t the software, and changing it will achieve nothing. You’d be dumping more money into what will ultimately turn out to be a failure. For example, a lot of people automate too early. As laid out in point 2, your script maintenance costs skyrocket when you test an unstable UI. This would lead to a lower ROI, making test automation too expensive to maintain. The problem lay with your usage of the software, though the blame was carried on to it. Thus, you’d be much better off researching into why your framework isn’t giving you the results you want, instead of blindly jumping boats.
- It isn’t quality analysis. You need your developers, your engineers and coders to build and maintain your test automation. Test automation deals heavily with the technical aspect of your software, and you need your technical team to be on top of it. It isn’t a part of the quality analysis, it is part of your software development. This fundamental misunderstanding is why most people run into point number 3, above. Save yourself and your project time, and don’t confuse the two. Test automation software will only check for problems that you know you will have. It is an automated software but lacks the creativity of a manual tester or a quality analyst. Even if you have a test automation suite at your fingertips, you can’t conflate it with quality analysis or a quality analyst. Humans may make mistakes, and software may be precise, but humans have access to the one thing software doesn’t come prebuilt with — imagination.
- Unit testing doesn’t replace end to end testing. If you’re looking at unit test automation to replace end to end testing, you’ve taken a wrong turn somewhere down the road. Unit testing has its place in the Test Driven Development process, but it will not help you with regressions or with finding bugs in your code. A whole team of well built and well-designed unit tests is an amazing resource to have, and you should invest both time and money into acquiring that. But without end to end test automation, you will run into the Voldemort of the testing world — regression problems. A unit test will check only one part of the code to see if it is functioning as it should. And out of context, this could seem fine. And then you run into the big R word (regression), and you’ll see why an end to end testing is essential to your overall test automation framework.
If you want to learn more about being more productive with Test Automation, contact us at CloudQA (firstname.lastname@example.org)
LIKE THIS POST SHARE IT WITH YOUR FRIENDS
Best Practices of Test Automation
To get maximum ROI out of automation, here are some best practices to be followed:
1. Know what to & what not to Automate
What to Automate?
· Test Cases which are important to be repeated in every release cycle.
· Test Cases which are time-consuming & tedious to execute manually because chances of human errors are high in this case.
· Test Cases which are High Risk or Business Critical.
· Test Cases which are easy to automate makes it cost effective
What Not to Automate?
· Test Cases which are not easy to automate. The effort will out weight the returns in this case.
· Infrequently executed test cases.
· Frequent requirement changing test cases
· Test cases where human intervention is a must.
· Low priority test cases
2. Keep your Tests Dry
Approach the design, implementation, and maintenance of your test suites with the same engineering and craftsman mindset you approach the design, implementation, and maintenance of your system’s data access layer, business logic layer, etc. If you’re not already familiar with the SOLID principles authored by “Uncle Bob” Martin, then take some time to read up on them. These principles are targeted to developers writing systems; however, the same principles apply directly to testers writing automation suites – if perhaps not quite to the same degree.
While I try to pay attention to all the SOLID principles in my test harnesses, I find that the DRY principle (Don’t Repeat Yourself) is especially critical for keeping my sanity as I build and maintain test suites. If you’re going to pick up one new thing about software engineering or craftsmanship, learn to love DRY.
DRY means avoid duplicating things in your test suite. One example of duplication in a test suite include having your element locators (IDs, CSS class names, XPath definitions, etc.) defined in more than one spot. Another example would be having every test perform the same steps to log on to a system.
This sort of duplication doesn’t seem like an issue when you have 20 or 50 tests, but wait until your test suite grows to hundreds or thousands of tests – and then go try to fix failing tests when (not if!) your UI or system flow changes. Duplication of this sort crushes the maintainability of your test suite over time – you spend more time fixing broken tests than you do deliver great value elsewhere.
3. Measure your ROI:
The success of automation cannot be determined by only comparing the manual v/s automation effort, but also by capturing the following metrics:
· Time required for executing automation testing for each and every release cycle
· Reduction in Time taken per release (Productivity improvement)
· Customer satisfaction Index & %age of defects found
It is good to follow a detailed process to achieve the best results from automation. At a high-level automation has following important stages:
A) Planning, Design & Development
B) Execution & Reporting
The above guidelines if observed can greatly help in making your automation successful.
Did you Know? If you have chosen CloudQA to handle all your web-testing, you don’t need to plan anything about scripting as it is codeless and keep your tests DRY, making it easier to maintain your tests. CloudQA magically runs all your test cases exactly the way you have recorded. Obviously, it saves you lots of money as well as resource time. You get more bandwidth to focus on important things to develop your product to beat the competition!!!
To know more about this amazing web-testing tool, visit www.CloudQA.io
LIKE THIS POST SHARE IT WITH YOUR FRIENDS
Benefits of Automation Testing with CloudQA