Case Study
Case Study of CITI Program
At a glance
- Customer: CITI Program (Collaborative Institutional Training Initiative), a provider of peer-reviewed web-based courses on research ethics, compliance and related topics
- Challenge: Load test two course enrolment journeys and an administrator transcript check with unique users: 930 + 930 concurrent learners and 150 administrators (2,000 distributed users), with screenshots and proper reporting
- What CloudQA did: Built the three scenarios in CloudQA, converted them to JMeter scripts, used random variables for unique users and admin accounts, and split the load between CloudQA and JMeter
- Outcome: Combined reporting of the CloudQA and JMeter runs, with screenshots, in CloudQA
CITI (The Collaborative Institutional Training Initiative) Program is dedicated to promote the public’s trust in the research enterprise by providing high quality, peer-reviewed, web-based educational courses in research, ethics, regulatory oversight, responsible conduct of research, research administration, and other topics pertinent to the interests of member organizations and individual learners.
CITI Program Requirement
CITI Program wanted to load test two of their courses to which students will enrol and check the respective enrollement in administrative account.
- Create two cases for testing a member registering a new account, and then enrolling in courses and passing a course.
- Third case is to log in as the Institutional Administrator account and check the transcripts of the students who passed the course.
- Load Test : 2000 distributed user load.
- 930 concurrent users – For 1st course registration and enrolment
- 930 concurrent users – For 2nd course registration and enrolment
- 150 concurrent users – For Institutional Administrator
Problem CITI Program Faced
- CITI Program wanted to generate unique users for all the registrations, check the administrative login using two hundred different admin accounts and verify the reports of registered users.
- Also, they needed the Load Test results with conventional screenshots to analyze the results from their end.
- And proper reporting tool for the load tests.
Tools out there in the market can perform load tests with any number of concurrent users, but it was hard to customize their tests.
Solution CloudQA provided:
CloudQA has customized CITI Program’s requirements by integrating with Jmeter.
- The three use cases were created in CloudQA, and the scripts were converted to Jmeter, so that functionally same tests can be invoked at the same time.
- Using CloudQA’s random variables, it could generate a unique user on every registration, random application variables were used to pick different administration credentials during load tests and store variables to verify the registered users in admin portal.
- The load was distributed on CloudQA and Jmeter so that conventional screenshots and reporting are shown in CloudQA. as Jmeter doesn’t show up the screenshots for a test case, unless it is a failure.
- CloudQA had shown the complete reporting of both Jmeter and CloudQA, which helped the CITI Program to analyze better.
CloudQA runs load tests like this one as a service on request: we reuse your recorded user journeys, run a realistic mix of them at scale and show where your servers start to struggle.
Lessons for your team
- Use realistic, unique test data. Thousands of identical users do not behave like real ones.
- Load test the whole journey (register, enrol, complete, report), not only the home page.
- Screenshots from real browsers make load-test failures much easier to analyse.
- Reuse your functional test scenarios for load tests instead of writing new scripts.
Frequently Asked Questions
How many users did the CITI Program load test simulate?
2,000 distributed users: 930 concurrent users for each of two course registration and enrolment journeys, and 150 concurrent institutional administrators.
How were unique users generated?
CloudQA random variables created a unique user for every registration and picked different administrator credentials during the run; stored variables were used to verify the registered users in the admin portal.
Can CloudQA run a load test like this for us?
Yes. Load testing is available as a service on request. Contact us with the journeys and the traffic you need to prepare for.