Test Automation 102: What It Doesn’t Do

Test automation isn’t the easiest process to deal with. It’s often technical nature, and oft-touted miraculous results lead to a lot of miscommunication and expectation management issues between supervisors and engineers who actually have to work the test. Expectation management is an especially crucial aspect, because both the engineer and his or her supervisor needs to be clear about test automation is, and what it can or cannot do. Now, one of the easiest ways to clear the debris of past expectations is to know what exactly test automation can’t do. This might seem like a redundant exercise, but you’ve only to look at the impact of negative spaces in marketing to realise the importance of this exercise! So, getting down to the meat of the matter, here are some of the things that test automation doesn’t do.

1. It isn’t a magical cure-all. One of the main problems with test automation results is that supervisors tend to rely on it to fix all their problems. Reality? It won’t. It isn’t a substitute for bad coding, it won’t be a balm to slow progress and it certainly won’t make a bad piece of software a good one. It simply tests if the work you have done already is stable, and works as you have programmed it to. it won’t fix your UI for you, nor will it make your code stable. You can’t run your code through a program and have it magically fix everything. If that existed, most developers would be out of jobs! The closest you can get is in cases of stress testing, regression testing, and performance testing. Even there, you’ll still have to fix your foundations yourself!
2. It isn’t a fix-all alternative to manual testing. Web test automation has a ton of uses and is useful and beneficial to use in a ton of situations. However, there are still some things you need to test manually. For example, test automation won’t tell you things about subjectively human experiences like the feel of a particular software, or its ease of use. The user interface is a particularly tricky area. If you automate too early, and your user interface undergoes massive changes, your costs for script maintenance will go up exorbitantly. Thus, even if you invest in a good automation setup, don’t expect to cut out manual testing altogether. You still need it for areas like UI, installation, compatibility, recovery, and documentation.
3. Different automation tools don’t solve your automation problems. This is quite a common practice, especially if decision makers don’t fully understand the limitations of test automation. If you are repeatedly running into issues with your test automation software, the problem isn’t the software, and changing it will achieve nothing. You’d be dumping more money into what will ultimately turn out to be a failure. For example, a lot of people automate too early. As laid out in point 2, your script maintenance costs skyrocket when you test an unstable UI. This would lead to a lower ROI, making test automation too expensive to maintain. The problem lay with your usage of the software, though the blame was carried on to it. Thus, you’d be much better off researching into why your framework isn’t giving you the results you want, instead of blindly jumping boats.
4. It isn’t quality analysis. You need your developers, your engineers and coders to build and maintain your test automation. Test automation deals heavily with the technical aspect of your software, and you need your technical team to be on top of it. It isn’t a part of the quality analysis, it is part of your software development. This fundamental misunderstanding is why most people run into point number 3, above. Save yourself and your project time, and don’t confuse the two. Test automation software will only check for problems that you know you will have. It is an automated software but lacks the creativity of a manual tester or a quality analyst. Even if you have a test automation suite at your fingertips, you can’t conflate it with quality analysis or a quality analyst. Humans may make mistakes, and software may be precise, but humans have access to the one thing software doesn’t come prebuilt with — imagination.
5. Unit testing doesn’t replace end to end testing. If you’re looking at unit test automation to replace end to end testing, you’ve taken a wrong turn somewhere down the road. Unit testing has its place in the Test Driven Development process, but it will not help you with regressions or with finding bugs in your code. A whole team of well built and well-designed unit tests is an amazing resource to have, and you should invest both time and money into acquiring that. But without end to end test automation, you will run into the Voldemort of the testing world — regression problems. A unit test will check only one part of the code to see if it is functioning as it should. And out of context, this could seem fine. And then you run into the big R word (regression), and you’ll see why an end to end testing is essential to your overall test automation framework.

If you want to learn more about being more productive with Test Automation, contact us at CloudQA ([email protected])