Integration QA Steps: Headline and Image Testing

Integration QA guide for Headline and Image Testing.

Testing your integration in staging or production

Your first headline and/or image tests can be initiated either on your staging site or production homepage(s). To test your integration in a QA/staging environment, the first step is to configure a staging Site on your Chartbeat account and set the domain variable in your stage-site Chartbeat code to the test Site ID. For example:

_sf_async_config.domain = 'test.mysite.com';

Follow these instructions from our website tracking integration guide to configure a QA Site in Chartbeat if you prefer to test your integration in staging and there is not already a test Site enabled for your account.

Tips for running your first tests

How to generate HUD pins on your staging site homepage:

We require click-through data on homepage links in order to generate Heads Up Display pins needed to initiate a headline test, so this click-through needs to be simulated on staging sites that have zero organic traffic and clicks. If you click a headline link a few times to open a few new tabs to that article in your browser, this should generate enough click data to create a pin on your staging site. The target article page must also have our code installed with the domain variable assigned to the same test Site ID as the homepage. The Heads Up Display will not assign pins for links to other landing pages or to external domains, so begin your test on a link to an article page under your staging domain.

How to verify image/headline replacement:

Whether testing on a staging site or your production homepage, you will need to use an incognito window to simulate visiting your site as a user in order to be served a variant. You will need to continue to close and re-open your incognito window to your homepage until a variant is shown. If a variant is not shown after five or so new homepage loads in fresh incognito windows, there may be an error in your integration.

Advice for testing on your production website:

If your team chooses to initiate your first tests on your production site homepage, you should always use legitimate headlines/image variants that your team is comfortable having displayed for a subset of your homepage audience while testing the integration.

If there are errors in your integration, or you initiate a test in an incompatible position of your homepage, the effect on your visitor experience should be minimal if at all noticeable. In most cases, a non-working headline/image test results in your visitors seeing the original headline/image from your CMS as usual, so the failed test will be unnoticeable.

Headline Testing only QA checklist: for sites that do not use our Image Testing add-on

The following variables are initialized prior to chartbeat_mab.js loading in the <head>: domain, uid, flickerControl, useCanonical and useCanonicalDomain OR path.
_sf_async_config.flickerControl is set to false if your site does not make use of our Headline Testing flicker control code (option 1), or it is set to true if your site does make use of our Headline Testing flicker control code (option 2).

Initiate a few headline tests in different positions to verify that headlines are being replaced successfully by the Chartbeat code on your staging or production website. If this is your first time initiating a Chartbeat headline/image test, learn how in this Help Center article. Before getting started with your first test, review our tips for QA'ing your integration at the top of this page.

Image Testing QA checklist: for sites that use Headline and Image Testing

Reviewing your code

The Chartbeat code on your site should include the following variables:

_sf_async_config.articleBlockSelector: Must be set in the Chartbeat snippet only if your site is not using <article> tags to encapsulate all headline and image <a> tag(s).
_sf_async_config.lazyLoadImageAttributes: Must be set in the Chartbeat snippet only if your site makes use of custom lazy load attributes for your images outside of data-src and data-srcset which our code will detect automatically.
_sf_async_config.domain: For testing on a staging site, this value should be set to the Site ID the staging site dashboard that has been created on your account. This is typically the root domain of the staging site.
_sf_async_config.flickerControl: If you are implementing flicker control for headline and image testing, this variable must set to false and must be loaded in the code snippet prior to the chartbeat_mab.js script.

If you are planning to test above the fold, flicker control code must be inserted in the <head>:

For optimal performance, the flicker control code should be loaded early in the page's head code.
There are two variables that must be set in the flicker control code: headlineSelector and articleBlockSelector.
A third variable, timeout, can be modified to allow more time for the Chartbeat Javascript to finish loading.

Running headline and image tests

Once your test environment is setup with Chartbeat Image Testing code and the corresponding configuration variables, we highly recommend starting image tests on multiple areas of your homepage to determine compatibility with your page markup. Tests should be started in multiple homepage areas above and below the fold, and should include both a variant image and headline. With tests running, you will need to load the test homepage in a Chrome Incognito window until the variant image and headline is served to verify successful replacement. Repeat for all areas where you anticipate your homepage team running headline and image tests.

If this is your first time initiating a Chartbeat headline/image test, learn how in this Help Center article. Before getting started with your first test, review our tips for QA'ing your integration at the top of this page.

Tests should be initiated in different positions on your homepage to determine image test compatibility with each position's unique HTML markup format.
Tests should be started with a single test variant (variant B): a different headline and a different image than the original (variant A). The headline and image should be easily distinguishable.
Once served a variant, the browser should be updated to simulate a mobile device. After simulating a mobile device and refreshing the page, you should confirm that the variant image and headline are still being shown.
In all tests, the variant headline should always be shown with its corresponding variant image. You should never see a scenario where the variant headline is shown with the original image.

A headline and image test should be started above the fold to evaluate the flicker control implementation:

Images and headlines should be hidden until either chartbeat_mab.js loads or the timeout, 1-second by default, elapses.
If the timeout elapses, the original headline and image should appear to the user and a trial should not be recorded.
If the timeout does not elapse and chartbeat_mab.js loads, the user will be served either the original headline and image or a variant. In either case, a trial should be recorded.
When images and headlines are eventually shown, users should not see any flicker effect.

A headline and image test should be started below the fold to evaluate the lazy load implementation:

The variant headline should always be shown with its corresponding variant image. You should never see a scenario where a variant headline is shown with the original image.

Use DevTools to review Chartbeat requests from your site

You can use your browser's site inspector tools to monitor network activity from your site, filtering on "chartbeat" to view all Chartbeat related requests made from your page.

Filtered on "chartbeat" in the Network tab of DevTools, here's the expected order of requests from a standard implementation of our snippet with mab.js included:

chartbeat_mab.js
chartbeat.js (or chartbeat_video.js)
mab.chartbeat.com/mab_strategy/headline_testing/get_strategy/
mabping.chartbeat.net/ping/mab
ping.chartbeat.net/ping (repeated every 15 seconds)

Up next

Choose your next integration.

Previousmab.js Specifications NextVideo Engagement

Last updated 3 years ago