We don’t have many sites on our network that run ads. On the whole, this fact contributes to the success of both our platform and our customers who use it. But in some cases, ads are an inescapable fact of internet life, and we have to deal with them properly.

One problem we have with ads is that they change a lot. Therefore they trigger false positives in our visual regression testing (VRT), and they do so in two ways:

  1. Their content changes very frequently, as different ads come and go within an ad slot, sometimes on every page load.
  2. The ad slots themselves come and go, because ad stakeholders tend to want to manage them closely and frequently.

This kind of false positive looks something like this:

The testing suite says, “Hey! Scott! Look! Something has regressed! But actually it’s just that a site admin has placed a new ad on the page and no one manually excluded it from VRT.

Point #1 is easy to resolve. We have developed a custom UI in WordPress admin for excluding a widget from QA testing:

The admin UI for manually excluding a widget from VRT.

Point #2 is harder because most customers, or even employees, are not in the habit of routinely thinking about excluding widgets from QA testing. So I’ve taken a different tact. I’ve started using some heuristics to determine if a widget seems like it contains an ad. If so, automatically exclude it from QA:

Improving our VRT’s is probably my favorite way of improving platform stability. Ads may have nearly destroyed a democratic society, but they won’t bother our VRT’s!

Photo of Scott Fennell Scott Fennell

Scott is LexBlog’s Director of TechnologyChief Technology Officer and Lead Developer.  He has authored hundreds of custom WordPress themes and plugins, and has deep expertise in full-stack web development. Scott has been published in the prestigious A List Apart, and many times

Scott is LexBlog’s Director of TechnologyChief Technology Officer and Lead Developer.  He has authored hundreds of custom WordPress themes and plugins, and has deep expertise in full-stack web development. Scott has been published in the prestigious A List Apart, and many times in CSS-Tricks, which is the web’s leading authority on cutting-edge front-end development. His work has been presented at the Google campus in Fremont, Washington as well as WordCamp in Portland, Maine. When not building the web, Scott is focused on his life-long goal of skiing from the summit of tall-ish mountains on all seven continents, for which he is about 85.7% complete.