Measuring the Impact of 3rd-Party Tags With WebPageTest

There's been an explosion in 3rd-Party Tags…

And as Tim Kadlec once said "Everything should have a value, because everything has a cost".

So how do we measure the performance cost of all the 3rd-party tags we keep adding to our sites?

Although Chrome supports blocking individual requests, my favourite approach still uses WebPageTest.

If you're interested in the Chrome approach Umar covered how to use DevTools to block a network request just after it was released in Chrome Canary – it still works the same way today and now you can even block whole domains too.

One of the reasons I prefer WebPageTest is its side-by-side comparison of the filmstrip is a really powerful way to communicate the impact of tags to co-workers, bosses and clients.

'Comparison of Marks and Spencer with and without 3rd-Party Tags'

Creating the Comparison

The first step is to test a page with nothing blocked i.e. in it's default state.

'Configuring a test in WebPageTest'

Remember to enable video capture, and labelling the tests will make it easier to differentiate them when viewing the filmstrip. In the spirit of being obvious I often label this step 'Original' or 'Default'!

Blocking Requests

Next we need to create a test with some requests blocked – I tend to label this one '3rd-Parties Blocked' or similar depending on what's actually blocked

There are a few ways to block requests:

Using the Block Field

The easiest way to block requests is to add entries to the block field:

The block field takes a space separated list of values and like its name suggests blocks any requests containing one of the values.

As it's a substring match it's pretty generous with what it accepts – anything from a single letter to a full URL – so if you add .com WebPageTest will block all requests with .com anywhere in the URL.

I use the block field approach more often than the script based approaches below.
Using a Script

Alternatively there are three script commands that can be used to block requests - they're straightforward and self-explanatory.

block – followed by a space separated list of substrings

This works in the same way as the block field in the previous example.

  block    .com
  navigate    https://andydavies.me

Example of test using block command

Note: the script also needs the command to navigate to the page you want to test too.

blockDomains - followed by a space separated list of domains to block

Blocks all the domains specified.

blockDomains   fonts.googleapis.com
navigate  https://andydavies.me

Example of test using blockDomains command

blockDomainsExcept - followed by a space separated list of domains to block

Blocks all the domains except those specified.

blockDomainsExcept andydavies.me
navigate  https://andydavies.me

Example of test using blockDomainsExcept command

Steve Souders covers using blockDomainsExcept for testing Service Workers, and it's a great example of where it can be really useful.

Note: One thing to watch for when viewing waterfalls from scripts that used blockDomains and blockDomainsExcept is the requests are blocked at the network level so you'll see the request in the waterfall with a -1 result code.

Comparing Results

Once both tests have completed we can hop over to the Test History tab, select the tests we want to compare and view the filmstrip.

'Choosing results to compare'

Choosing which Requests to Block

I generally use one of two strategies for determining which requests to block depending on whether I want to raise awareness that 3rd-parties tag are a problem in general, or whether I want to determine the cost of a single tag.

Block Everything

The option that requires the least thought is 'block all the things'.

One way to do this is to this is by copying the list domains from the domains view of a completed test and using this as a block list.

It's a bit of manual labour so I tend to use a shell command to create the list.

The domains view can also produce a JSON formatted response and one way I build the list of domains to block is to pipe the JSON output through jq and then copy it to the clipboard using pbcopy.

curl 'http://www.webpagetest.org/domains.php?test=180130_NH_4c83de2dfe315f19e5367b91b6ac4a37&run=1&cached=0&f=json' | jq -rj '.domains.firstView[].domain + " "' | pbcopy

Then I paste the list of domains into the block field and remove any that shouldn't be blocked – domain being tested, and any domains it relies on, for example, static or media assets domains.

Note: Of course pbcopy is macOS only but there are alternatives such as xclip on Ubuntu etc.
Block a Subset of Requests

Blocking all 3rd-parties is a dramatic way to show their combined cost, but it's slightly less useful when it comes to discussions about the impact or value of an single 3rd-party tag.

Blocking a single tag, perhaps one that delays the page from rendering, or slows other critical content from displaying, is a powerful way of understanding just its impact.

And if you have a Real User Monitoring product (such as NCC Group's RUM, or Soasta's mPulse) that models the impact of speed on conversions and other business metrics, you can use the time differences to get a real indication of what the 3rd-party is actually costing in lost conversions, for example.

Gotchas

Blocking 3rd-parties isn't a foolproof process and sometimes blocking a 3rd-party doesn't have the expected impact or introduces side effects that make the 'slimmer' page slower.

In this comparison of debenhams.com with and without 3rd-party tags the test with 3rd-party tags blocked is actually slower than the one with tags present (you might notice the base page gets re-requested as request #12 in the tests with the tags blocked)

So don't be surprised if occasionally blocking 3rd-parties doesn't have quite the effect you expect.

In these cases, I tend to reduce the list of 3rd-parties being blocked until I isolate the one that's causing the unexpected behaviour.

Closing Thoughts

Filmstrip comparisons are a great way to illustrate how 3rd-parties affect the visual experience and they're easy for everyone to understand too.

Although filmstrips only help a little in understanding whether a page is in a state where the visitor can interact with it (or not), WebPageTest has an estimation of when a page is interactive at the bottom of the waterfall.

Comparing the charts from two runs give us some indication of whether the page with 3rd-parties removed becomes usable sooner.

'Time-to-Iteractive Comparison'

One of the performance challenges we face is helping people to understand the cost of 3rd-party tags in terms of user experience and WebPageTest's ability to block requests offers a great way to help with that.

Andy Davies

Web Performance Consultant