BackstopJS is a framework for visual regression testing, written in Javascript. Visual Regression Testing is a form of snapshot testing that tests a web front-end. It goes beyond testing the markup or layout by testing the rendered page captured in an emulated browser. As such, they can “catch CSS Curve Balls” as BackstopJS says.
Why use Visual Regression Testing?
I’ll assume that you appreciate why testing as a general practice is necessary, so here are several scenarios that visual regression tests will catch and other testing techniques won’t.
CSS Regressions: Often we’ll make a change to a CSS style to move that one button into the space we’d like it to be, but how do we know that it hasn’t had an undesirable knock-on effect on some other part of the website that uses the same CSS class? Instead of checking every element that may match your CSS selector manually (which we will most often forget to do), run your visual regression tests.
Responsiveness: Most often we’re working on a 1080p screen, but many (perhaps most) of our users will be using their smartphones. Visual regression tests can test an assortment of different screen sizes, giving you confidence that your changes haven’t broken the responsive page at other sizes and saving you time manually testing the page at different resolutions.
Dependabot: It’s a wonderful tool that saves you from manually keeping all of your dependencies up-to-date. In my current team, we use dependabot aggressively; we have extensive testing and auto-merge any Dependabot PRs that pass all our tests. In fact, in terms of PRs or commits it’s the most active member of our team by far. However, you can’t place your trust in Dependabot’s auto-merge in the front end if you aren’t testing for visual regressions. Before we introduced visual regression testing we had instances where Dependabot would automatically update to a new version of a dependency that introduced a visual change (at one point even removing a prominent navigation link), and it would be automatically merged and deployed to production. We trust Dependabot to change our software’s behavior because we have extensive tests of that behavior, and we trust it to change our software’s visual appearance because we have visual regression tests.
The Benefits of Visual Regression :
1. A single test covers multiple checks on the screen including the presence of labels, font types, alignment, layout, color, and links
2. Exposes both material and small-scale UI differences
3. Ability to add more value to functional automated checks
4. Improved product quality and user experience
5. Comparison of visual elements across multiple browsers
6. Improved speed
7. Improved accuracy of checks
8. Allows test analysts to spend time on interpretation and more complex problems
9. Visual regression tools also provide simple image diff functionality
10. Great visual designs help to build trust and credibility.
11. Reinforces brand image.
12. Forms a visual balance on the screen.
13. Guides the users towards actions.
14. Persuades the eye to shift its attention towards certain page elements.
Some areas where visual validation testing is highly recommended:
1. Mobile Apps
2. Mobile Web/Responsive Web
3. Marketing websites
4. Content management systems
5. Consumer systems like airlines, travel, banking, etc.
Visual Regression Testing Key Features
Test Recorder: Records the steps of manual testers who are testing features within the user interface.
Reporting: Robust reporting system so you can be proactive about your fixes.
Cross-platform Testing: Ability to test across a variety of platforms and devices.
Single Dashboard: Everything you need is easily managed from a single dashboard.
Where you can avoid Visual Regression Testing (VRT)
In your web/mobile page having GIFs, Dynamic content, and animations, the dynamic layout will lead to heterogeneous behavior in the testing results and major frustrations.
Spot the Difference
Visual Regression Testing
The BackstopJS Workflow
Just run backstop test and you’ll start seeing text stream across your screen like you’re a hacker on TV. When it’s done, your default browser should pop up with a report generated by BackstopJS
The BackstopJS Workflow
Percy vs BackStopJS :
Percy
Excellent in-browser reporting (and management)
Storybook compatible
Owned by Browserstack (potential pairing/bundling),
Can only test against Chrome and Firefox
SaaS / Tiered Pricing
BackStopJS
Good command line and in-browser reporting
Actively maintained
Can only test against Chrome
Requires Storybook configuration
Open source and Free
The BackstopJS Reporting
1. BackStopJS Reference
2. BackStopJS Test :
BackStopJS Test
In Conclusion, Visual Regression Testing is a powerful layer to add to your quality gates, which can provide extensive test coverage with minimal effort. It actually also reduce the number of UI test cases you currently have, which are just checking the element or component displayed.
We at CaratLane are solving some of the most intriguing challenges to make our mark in the relatively uncharted omnichannel jewellery industry. If you are interested in tackling such obstacles, feel free to drop your updated resume/CV to careers@caratlane.com!
Leave a Reply