Responsive UI Layout Testing Case Study

A perfect example of this need for Responsive UI Layout Testing comes from a recent project with one of our large enterprise customers in the Media sector. The customer was facing problems where its end users were unable to easily navigate to get access to content in the e-newspaper application due to its responsive nature. Users were experiencing issues with the content displaying properly on various viewports.

Challenges of Responsive UI Design and Testing

There were multiple challenges that needed to be addressed in order to provide a flawless user experience, including :

• The daily e-newspaper was to be published with the UI showing news updates from various parts of North America, including content such as images and videos.
• Advertisements needed to be properly visible in every dimension so that they could be clearly visible to the user, and the user should be able to click on the ads, enabling more ad revenue generation.
• Menu Navigation for every page needed to be displayed properly on each device and orientation to enable accessibility.
• Breaking news for the hour needed to be displayed as a footer.
• Top news from every section needed to be visible with proper clicks to read the whole story.
• Text needed to be fixed so as not to overlap at the edges with the decreasing size of the device.
• Ability to flash correct error messages in case of any issues with good colors used to display the messages.
• Images displayed needed to be of good quality and clearly visible.
• Adjust alignment of text and images with change in orientation and screen size.
• Change colors on hovering the mouse over necessary buttons and links.
• Needed a good content consumption environment which followed the rule “follow users not devices.”
• All the above mentioned criteria needed to be tested for every page along with the regression testing in case of some changes as there needed to be innovation and creativity in the design.

Ultimately, the customer needed a solution that could help make its product more optimized to enrich the end user experience, as well as make its content easily visible with varying screen sizes over different devices with optimum accessibility. This also meant that advertisements placed on the content needed to be properly validated so that there is no loss of revenue leading to stakeholder dissatisfaction.

Automated Responsive UI Layout Testing Solution

The Bitwise QA Center of Excellence took up the challenge by first chalking out the Layout Testing Automation solution. The team’s approach included the following steps, and the solution delivered was aligned with best practices :

• Analyzing the pages with the most critical content.
• Defining the viewports (Browser / Device) on which the UI needs to be validated.
• Using open source API (Galen) and creation of various gspec files, which essentially include layout tests, ensuring best practices are followed.
• Writing user interactions using Selenium WebDriver in the tests.
• Use of single tests for both Browser and Mobile devices (Android and iOS).
• Executing these tests on Cloud Devices (Android and iOS) and Browsers.
• Reporting automated test results through emails to stakeholders.

The approach ensured that CB (Cross-Browser) and CD (Cross-Device) Testing was aided using a cloud solution coupled with an open source API to ensure optimum test coverage. These tests were then executed periodically during regression.

This resulted in ensuring that the content and elements on the pages are displayed flawlessly over various viewports. As a result, the test suite created added value to regression testing by optimally validating the UI on various layouts after changes were done to the applications.

Increased Customer Satisfaction

Bitwise provided a complete automated responsive UI test solution that received enormously positive feedback from its users, leading to a 40% increase in ad revenues. In addition, the subscription rate increased due to the efficacious automated testing provided by our test suite design.

With our expertise in delivering solutions around Responsive UI Layout Test Automation, we set up a framework using open source tools and APIs. As a result of using our test suite to identify issues on various viewports and OS, the customer achieved ROI in terms of better quality and content leading to increased customer satisfaction. The test suites created were generic and easily maintainable, which also reduced maintenance costs and served as a cost saver during the long run.

What challenges are you facing in designing and testing your Responsive UI Layout? Leave a comment below or contact Bitwise…we’d love to help.

The use of shift-left testing yields the following benefits

• Bugs are found and fixed early
• Saving time and resources
• Allowing good test coverage
• Better teamwork with developers and testers – this model minimizes frictional differences between them
• Delivery of software is expedited
• Cost-effectiveness

If you need a solution to your quality, cost, and time woes, contact Bitwise and see how we can help.

The Mobile App

The App was built on Jelly Bean Android Platform Version 4.3 specifically designed for Car Service Center chains across North America and Canada.

The App would help the Supervisors and the Technicians working at the Service Center make informed decisions while routing the car owners to a particular Service Station within 4 miles in case that the car bay at any given Station was at full capacity. This situation would mostly happen over weekends and the customers would experience long wait times to get their cars fixed or serviced. The App would also correctly route customers to stores based on the availability of parts.

The App would provide a benefit to the client – effectively manage the inflow over weekends and would also improve Customer Satisfaction, which in turn would positively impact the bottom line of the business. It was tested thoroughly and a limited-feature version was put on the Android Play Store available as a FREE download.

The Presentation

Jeff flew to Orlando, FL to present the App at the Car Service Company. This deal would be a milestone for his company if it was cracked.

Jeff was to present the salient features of the App to the top brass of the company which included 12 people across the different verticals of the organization. He started his presentation with his Android device connected to a device projector.

He ruled the roost when it came to presenting the App and the value it would add to the company. Soon everyone was impressed and the work seemed done before he even closed on the presentation to solicit any queries. He was standing on the podium with confidence until…..

The Bug

A rather amplified voice from the Manager of Services asked Jeff whether he could download the same App to his Android device. Jeff smiled and gave him the instructions to download. The Manager became busy with his mobile device for some time and soon he found out that he was unable to download the App on his device and every time he would do so, he would get an error:
“<insert app name >” could not be downloaded due to an error. (403)

After a couple of attempts in vain, he raised his voice yet again “it just won’t get on my phone”. Everyone in the board room seemed curious about what was going on. Jeff was left with no answer and he was just beating around the bush handling queries. Then someone from the team suggested to try downloading the App on WIFI without using the Carrier provided Network.

That worked and the App downloaded in a flash. But it would not work on the Carrier provided network. There it was, “A BUG”, everything seemed so well if this nasty bug had not surfaced. Skepticism overruled optimism and it all came to a grinding halt!

The Retrospection: Autopsy of Disaster

Jeff‘s disappointment slowly transformed into a thorough retrospection during the long trip back to the pavilion. “Why was this not tested?” was the question that was hitting the neurons of his brain again and again. “What’s going on? We are a service-based company with a special focus on Products, how could such a thing happen?”

Jeff’s retrospection uncovered that the QAs were accustomed to testing their Apps only on a selection of devices, but carriers required more money for procurement. It was not long before Jeff realized that Mobile Testing was completely different compared to Desktop Testing.

At BitWise, we understand this. Our Mobile App Testing approach focuses on covering the following salient features:

• Crowd-Sourced Testing: Accommodate the need of a wide array of devices and different carriers to cover testing the app on all the possible hardware combinations
• Performance Testing: Mobile Apps NEED to be tested on various Networks such as 3G, 3G+, H+, and 4G LTE
• Mobile App Battery Consumption Testing: Usage of Freeware tools to analyze the battery consumption by an App on a device.
• Automated Testing: Stress the use of Automation testing but the main challenge in Automation is technicality and Cost of the Tools. Cloud-based tools which provide a gamut of devices on various carriers also require technical knowledge but as far as the Cost is concerned; such tools are at times out of budget for many organizations.

The Out of the Box Solution: Automation

To beat this budget problem, BitWise has come up with a clever combination of testing strategies based on various cloud-based mobile testing providers with complementary strengths. Apart from reducing the cost, the user gets a multitude of choices for mobile platforms and carriers. Best of both worlds!