Test Automation

Best Practices of Test Automation

To get maximum ROI out of automation, here are some best practices to be followed:

1. Know what to & what not to Automate

What to Automate?
· Test Cases which are important to be repeated in every release cycle.
· Test Cases which are time-consuming & tedious to execute manually because chances of human errors are high in this case.
· Test Cases which are High Risk or Business Critical.
· Test Cases which are easy to automate makes it cost effective

What Not to Automate?
· Test Cases which are not easy to automate. The effort will out weight the returns in this case.
· Infrequently executed test cases.
· Frequent requirement changing test cases
· Test cases where human intervention is a must.
· Low priority test cases

2. Keep your Tests Dry

Approach the design, implementation, and maintenance of your test suites with the same engineering and craftsman mindset you approach the design, implementation, and maintenance of your system’s data access layer, business logic layer, etc. If you’re not already familiar with the SOLID principles authored by “Uncle Bob” Martin, then take some time to read up on them. These principles are targeted to developers writing systems; however, the same principles apply directly to testers writing automation suites – if perhaps not quite to the same degree.

While I try to pay attention to all the SOLID principles in my test harnesses, I find that the DRY principle (Don’t Repeat Yourself) is especially critical for keeping my sanity as I build and maintain test suites. If you’re going to pick up one new thing about software engineering or craftsmanship, learn to love DRY.

DRY means avoid duplicating things in your test suite. One example of duplication in a test suite include having your element locators (IDs, CSS class names, XPath definitions, etc.) defined in more than one spot. Another example would be having every test perform the same steps to log on to a system.

This sort of duplication doesn’t seem like an issue when you have 20 or 50 tests, but wait until your test suite grows to hundreds or thousands of tests – and then go try to fix failing tests when (not if!) your UI or system flow changes. Duplication of this sort crushes the maintainability of your test suite over time – you spend more time fixing broken tests than you do deliver great value elsewhere.

3. Measure your ROI:

The success of automation cannot be determined by only comparing the manual v/s automation effort, but also by capturing the following metrics:

· Time required for executing automation testing for each and every release cycle
· Reduction in Time taken per release (Productivity improvement)
· Customer satisfaction Index & %age of defects found

It is good to follow a detailed process to achieve the best results from automation. At a high-level automation has following important stages:
A) Planning, Design & Development
B) Execution & Reporting
C) Maintenance

The above guidelines if observed can greatly help in making your automation successful.

Did you Know? If you have chosen CloudQA to handle all your web-testing, you don’t need to plan anything about scripting as it is codeless and keep your tests DRY, making it easier to maintain your tests. CloudQA magically runs all your test cases exactly the way you have recorded. Obviously, it saves you lots of money as well as resource time. You get more bandwidth to focus on important things to develop your product to beat the competition!!!

To know more about this amazing web-testing tool, visit www.CloudQA.io

LIKE THIS POST SHARE IT WITH YOUR FRIENDS

Benefits of Automation Testing with CloudQA

Fast track your testing process

Recent Posts

Selenium Test Automation

Selenium is by far the most preferred Test Automation tool for web applications and more people are adopting it for their test automation strategy. As someone new to test automation, do you know what it takes to set up a successful test automation environment and be productive with it? Here are some thoughts.

Selenium Test Automation

Starting out with Selenium

Evaluate the skill set of your existing resources: Identify if you have experienced resources in-house or how you will acquire resources with experience in one of the languages that Selenium supports (Java, C#, Python, Ruby, Php, Perl & Javascript).

Selenium training: Once you have identified the people who will work on your test automation efforts, you have to account for the training period which could be anywhere from a few days to a few weeks depending upon the existing expertise of the individuals.

Test automation framework: Deciding on a framework can take a lot of work. There are a few approaches that are available that you have to evaluate and decide the one that fits best in your environment. The frameworks that exist include behavior driven, data-driven, keyword driven & now agile.

Define Object Repositories: These repositories are a collection of UI and Application object data recorded by the testing tool for the application under test.

Define a test reporting framework: Selenium does not offer any sophisticated reporting mechanism so you will have to decide what type of reports you would want for your test environment – this may include test execution results reports and test management related reports.

Once the items above are addressed, you can start building your web automation tests.

As you can see that building out an effective test environment with Selenium will require commitment, up-front time and qualified resources. For an organization with a large application that has frequent updates, you will also have to prepare for managing the test cases already written & evaluate their usability in an ever-changing environment. Just to get the initial framework ready, you can spend anywhere from a few weeks to a few months before writing a single automation test.

Selenium Productivity Add-ons

While Selenium is probably one of the best test automation tools, you should look at tools that can help you speed up the entire automation process. There are a few Selenium Automation “wrapper” tools that run on top of Selenium and make you a lot more productive and you should definitely not shy away from evaluating them for:
1. Complexity – how complex are they to set up, learn & run
2. Test coverage (what % of selenium features can they support, can they automate, say, 80% of your automation needs with little effort)
3. Test management: Chances are that you will end up with 100’s of test scenarios over a period of time. You want to ensure that can easily manage the test cases
4. Reporting: You should be able to get a wide variety of reports including test execution & analytics with past results
5. Test Execution: You want to ensure that your texts can be validated against multiple browsers
6. Other productivity benefits: check if the tool helps you in your agile development process, CI/CD integration, assist in data-driven testing, etc.

If you want to learn more about being more productive with Selenium, contact us at CloudQA (info@cloudqa.io)

LIKE THIS POST SHARE IT WITH YOUR FRIENDS

Benefits of Automation Testing with CloudQA
Fast track your testing process

Recent Posts

web automation Functional Testing

What is Functional Testing?

A properly written functional test ensures that the system is functioning exactly as the USER expects it to. Functional tests target business goals and typically defined and validated by an expert end user. Since it may not be possible for an expert to test the functionality of the system at each and every step of development, the test cases/scenarios are often documented with their help. This document defines the step by step instructions of system interactions along with the expected outcomes for every test case defined by the expert users. It can then help any user (not just the Expert) to be able to follow the steps to test the software and verify if the outcome is exactly as the expert has defined. To ensure completeness, functional tests should cover both the positive and negative scenarios.

web automation Functional Testing

Challenges:

Manual functional testing can be a tedious process as it can take quite a while for a tester to step through each of the steps and verify the outcome/results. When the user goes through these steps each time there is a software update, there is a high probability of erroneous information being entered or captured by the tester resulting in incorrect validations. If the number of test cases is extremely large and complex this problem gets magnified. The manual testing process can, therefore, be extremely laborious, monotonous and error-prone and requires highly dedicated and qualified resources to execute.

This can be overcome by automating the functional tests as much as possible. Granted that not every test can be automated, but even a 50% coverage via test automation can bring about huge relief in time, cost and resources. Once a test scenario is “automated”, it can be run on demand after each software release cycle. The feedback from an automated test is extremely fast compared to the same test run manually.

Consider an example where a test scenario requires a user to traverse through multiple screens, enter data into 20+ fields, select various check boxes, radio buttons, pick a value from a drop-down options, enter date in the right format, validate the response on 5 submit buttons, etc.. A real-life scenario like this can easily take the user over 5-10 minutes each time to run this test. For the same scenario, if the test requires a different data set or a negative test, it means another 5-10 minutes. When you look at multiple data sets and 100’s of test scenarios, it translates to weeks of testing that is required after each major software release. This also causes a bottleneck for the development team to move any further before getting a complete feedback from the test users. If all these test scenarios could be automated, it would take only a couple of hours to go through the tests and feedback cycle to the developers would be extremely fast compared to manually testing all the scenarios. That is a huge saving in terms of time. effort and resources and easily justify adopting functional test automation as much as possible.

A lot of companies, as a standard practice, do not thoroughly test their software application after each minor or major release because of the aforementioned manual testing challenges. The risk that they undertake, however, is extremely high if the software fails in a real-world environment – brand reputation, lost clients, fixes that take too long and that potentially break other parts of the software.

Automation Frameworks

One way to speed up functional testing is through web automation tools.  Selenium is one of the few software testing automation frameworks for web applications out there which is open source and has a very large following.  It has a set of tools to support test automation in multiple browsers.  Web test automation with Selenium, however, is a programming activity and people creating tests should be comfortable with one or more of the few programming languages that Selenium supports (Java, C#, Python, etc.).  Even though it might take some time (a few hours or days depending on the complexity of the test case), it can greatly reduce manual repetitive/regression testing efforts after each software release cycle.

There are a few companies that are working towards reducing the time to automate the tests with Selenium by providing a “wrapper” solution which essentially means “automating the Selenium test automation” efforts.  However, since there are many types of browser interactions in modern web technologies, not everything can be addressed easily.  The goal of a tester should always be to evaluate and use tools & methods that can address the maximum number of test cases with minimum effort.

Automation with CloudQA

There are quite a few testing tools that can assist in automating the functional tests but the challenge faced by most tools is the automation coverage (especially in large software applications). CloudQA has been designed to cover a lot of complex test scenarios. The goal of CloudQA is to cover automation of as many functional test cases as possible. It starts with a unique ability of CloudQA to discover as many test cases as possible of the software application via an auto-learning process. This, in itself, can save test managers weeks (if not months) of discovering scenarios that can be possibly tested. Besides that, some of the complex scenarios that CloudQA handles right out of the box includes

* Table data access
* Auto Suggest boxes
* Variable storage of a dynamic data set
* File Upload tests
* Data-driven testing (for multiple data sets)
* Various types of user assertions (check, check not, etc.)
* Various actions – click, alert, screen snapshot, wait

If you are thinking about automating your functional tests, talk to one of the CloudQA specialists (email: info@cloudqa.io)

LIKE THIS POST SHARE IT WITH YOUR FRIENDS

Benefits of Automated Functional Testing with CloudQA

Fast track your testing process

Recent Posts

software test management

The effectiveness of Automated Testing wherever applicable in software test processes has been rarely disputed. Automated Testing reduces testing time, identifies bugs faster and facilitates continuous development (CD). However, Automated Testing environments tend to get extremely large (one with over 100 test scenarios would qualify as large and test scenarios can easily run into 1000’s) and tedious to maintain. It produces results that cannot be fully relied on. Therefore, it is critical to managing a large test environment effectively using well-defined processes and a decent set of tools.

If your test library is large, it is important that you find out if the individual test scenarios are properly organized, critical to the core application and relevant to the latest application version or to the usage pattern.

software test management

1. Test Organization: Your applications tests may involve testing various functional capabilities of an application. How do you determine which tests to run after you have made updates to just one section of your application (for example, your “billing” functionality has been updated)? You may not want to run all your test scenarios (unless doing regression tests) in order to test a small update. Your test framework should allow you to categorize your tests by functional areas so that you can quickly filter out everything else and run only those tests that belong to a specific function.

2. Test Criticality: Ideally you would create tests that cover both the critical and non-critical areas of your application. Your testing framework should provide an efficient way to manage bugs and failures. You should be able to categorize your test upfront by criticality (Critical, High, Medium or Low) so as to prioritize the testing as well as the resolution based on the criticality of the test.

3. Test Relevance: In most environments, software development and testing is a continuous process and over time you can have a large number of tests. Unfortunately, as software evolves (with new or updated features, changed processes, etc.) some of the tests become redundant or duplicate. It can be challenging managing test library to ensure that only quality test cases exist, especially if your testing framework does not allow for easy review of the individual test cases/scenarios. You should be able to log and keep a track of the entire test creation life cycle (including updates). For instance, you should be able to track the test creation date, the module/release version for which the test was created, the test case revision history along with the name of individuals who authored the tests. This would make a periodic review of test cases to identify redundant scenarios much easier and keep your test library lean and clean. It will ensure a test environment that provides high-quality test results that you can rely on to make decisions. You do not want to rely on the results from redundant tests by ignoring “passed” results or waste your time by acting on “failed” results that they produce.

4. Test Usage: If there are test scenarios exceeding 2 months sitting in your library, there is a possibility that they are just taking up space and may not be required anymore. It is vital that your testing framework should be able to identify such test scenarios so that you can make apt decisions, either to re-prioritize or re-categorize them for efficient usage or get rid of them.

By maintaining your test libraries through regular checks, you will certainly have a manageable test environment that produces reliable results on which you can make assertions and also ensure a faster and better software development process.

CloudQA and Effective Test Management

CloudQA is much more than just a Web Test Automation tool. It has some helpful features that make test management much simpler.

Test Organization: You have the ability to tag your test cases and test scenarios by functional groups or any other preferred method. (show image)

Test Criticality: Tag your test cases/test scenarios as Critical, High, Medium or Low.

Test Usage & their Relevance: CloudQA keeps a record of the run history of all your test cases. You can view the history in many ways – for example – show the behavior of “Critical” Test Scenarios in “Billing” function over the last 4 weeks OR show the “Critical” test scenarios that have not been run in the last 2 months OR look at the revision history or a particular test.

LIKE THIS POST SHARE IT WITH YOUR FRIENDS

Benefits of Automation Testing with CloudQA
Fast track your testing process

Recent Posts

Testing As A Service

As companies rush to meet the ever-evolving, customer focused, multi-interface, and technologically complex solutions, software quality and time to market becomes a critical need. One quickly realizes that software testing plays an important role in the growth of any company offering such solutions. However surging costs, high-security environments, and software sourcing variability has made the software testing function challenging.

Testing As A Service

In such an environment, it is important to assess and implement the right methodologies to achieve scale, minimize costs and offer improved services. Testing-as-a-Service (TaaS) is considered a viable testing model by many organizations to achieve reduced costs and improved service for their IT test requirements.

TaaS does this by outsourcing testing activities associated with some of a company’s business activities to a service provider rather than using their own employees/resources. By using a consumption-based pay model, there is less risk and investment on the part of the organization.

TaaS can be used for various QA activities, such as:

1. Functional testing
2. Load / Stress testing
3. Performance and Benchmark testing
4. Regression testing

Advantages of TaaS over traditional testing

1. Scalable model via public cloud – utilize computing power, disk space, memory as per current requirements but have the ability to ramp up on demand very quickly
2. Costs – Pay for what you use. No need to own your own infrastructure. No capital costs incurred.
3. Licensing benefits – Operating system (OS), test tools, hardware, application licenses are managed by the cloud.
4. Standardization – Centralized test environment with a standard set of tools and processes will ensure repeatability, efficiency and quality results

Besides self-service SaaS-based subscription model for automated testing, CloudQA also offers Testing as a Service for companies that do not have the bandwidth to carry on this essential step in their agile development process. Customers do not have to worry about any software licensing, infrastructure, expensive testing resource or steep learning curve for the testing tool & processes. Instead they can focus on developing their own software which is their area of competency.

Why CloudQA?

CloudQA has an established framework for delivering enterprise-ready testing and providing clients with the ability to take advantage of paying for what they need, when they need it, rather than investing in expensive tool sets and experts.

1. Center of Excellence – You will interact with a group of people that promote collaboration, employ software testing best practices with both tools and techniques
2. Process – We employ industry best practices and methodologies that will result in higher productivity and you will get more bang for your buck
3. Governance – We offer relationship management, QA Project management, and a centralized test environment, with standardized software library and test suites. We also provide a self-service portal which cuts downtime required to provision test environments.
4. Infrastructure – We own the automation software and even have an offshore facility for providing a cost-effective, highly scalable, and fast automated test solution

Jump start automation of your web application

Try our Testing As A Service starting at $499/month

LIKE THIS POST SHARE IT WITH YOUR FRIENDS

Benefits of Managed Testing with CloudQA
Fast track your testing process

Recent Posts

software test management

The effectiveness of Automated Testing wherever applicable in software test processes has been rarely disputed. Automated Testing reduces testing time, identifies bugs faster and facilitates continuous development (CD). However, Automated Testing environments tend to get extremely large (one with over 100 test scenarios would qualify as large and test scenarios can easily run into the 1000’s) and tedious to maintain. It produces results that cannot be fully relied on. Therefore, it is critical for managing a large test environment effectively using well-defined processes and a decent set of tools.

software test management

If your test library is large, it is important that you find out if the individual test scenarios are properly organized, critical to the core application and relevant to the latest application version or to the usage pattern.

1. Test Organization: Your applications tests may involve testing various functional capabilities of an application. How do you determine which tests to run after you have made updates to just one section of your application (for example, your “billing” functionality has been updated)? You may not want to run all your test scenarios (unless doing regression tests) in order to test a small update. Your test framework should allow you to categorize your tests by functional areas so that you can quickly filter out everything else and run only those tests that belong to a specific function.

2. Test Criticality: Ideally you would create tests that cover both the critical and non-critical areas of your application. Your testing framework should provide an efficient way to manage bugs and failures. You should be able to categorize your test upfront by criticality (Critical, High, Medium or Low) so as to prioritize the testing as well as the resolution based on the criticality of the test.

3. Test Relevance: In most environments, software development and testing is a continuous process and over time you can have a large number of tests. Unfortunately, as software evolves (with new or updated features, changed processes, etc.) some of the tests become redundant or duplicate. It can be challenging managing test library to ensure that only quality test cases exist, especially if your testing framework does not allow for easy review of the individual test cases/scenarios. You should be able to log and keep a track of the entire test creation life cycle (including updates). For instance, you should be able to track the test creation date, the module/release version for which the test was created, the test case revision history along with the name of individuals who authored the tests. This would make a periodic review of test cases to identify redundant scenarios much easier and keep your test library lean and clean. It will ensure a test environment that provides high-quality test results that you can rely on to make decisions. You do not want to rely on the results from redundant tests by ignoring “passed” results or waste your time by acting on “failed” results that they produce.

4. Test Usage: If there are test scenarios exceeding 2 months sitting in your library, there is a possibility that they are just taking up space and may not be required anymore. It is vital that your testing framework should be able to identify such test scenarios so that you can make apt decisions, either to re-prioritize or re-categorize them for efficient usage or get rid of them.

By maintaining your test libraries through regular checks, you will certainly have a manageable test environment that produces reliable results on which you can make assertions and also ensure a faster and better software development process.

CloudQA and Effective Test Management

CloudQA is much more than just a Test Automation tool. It has some helpful features that make test management much simpler.

Test Organization: You have the ability to tag your test cases and test scenarios by functional groups or any other preferred method. (show image)

Test Criticality: Tag your test cases/test scenarios as Critical, High, Medium or Low.

Test Usage & their Relevance: CloudQA keeps a record of the run history of all your test cases. You can view the history in many ways – for example – show the behavior of “Critical” Test Scenarios in “Billing” function over the last 4 weeks OR show the “Critical” test scenarios that have not been run in the last 2 months OR look at the revision history or a particular test.

LIKE THIS POST SHARE IT WITH YOUR FRIENDS