Integration QA Steps: Headline and Image Testing
Integration QA guide for Headline and Image Testing.
Your first headline and/or image tests can be initiated either on your staging site or production homepage(s). To test your integration in a QA/staging environment, the first step is to configure a staging Site on your Chartbeat account and set the
domain variable in your stage-site Chartbeat code to the test Site ID. For example:_sf_async_config.domain = 'test.mysite.com';
Follow these instructions from our website tracking integration guide to configure a QA Site in Chartbeat if you prefer to test your integration in staging and there is not already a test Site enabled for your account.
We require click-through data on homepage links in order to generate Heads Up Display pins needed to initiate a headline test, so this click-through needs to be simulated on staging sites that have zero organic traffic and clicks. If you click a headline link a few times to open a few new tabs to that article in your browser, this should generate enough click data to create a pin on your staging site. The target article page must also have our code installed with the domain variable assigned to the same test Site ID as the homepage. The Heads Up Display will not assign pins for links to other landing pages or to external domains, so begin your test on a link to an article page under your staging domain.
Whether testing on a staging site or your production homepage, you will need to use an incognito window to simulate visiting your site as a user in order to be served a variant. You will need to continue to close and re-open your incognito window to your homepage until a variant is shown. If a variant is not shown after five or so new homepage loads in fresh incognito windows, there may be an error in your integration.
If your team chooses to initiate your first tests on your production site homepage, you should always use legitimate headlines/image variants that your team is comfortable having displayed for a subset of your homepage audience while testing the integration.
If there are errors in your integration, or you initiate a test in an incompatible position of your homepage, the effect on your visitor experience should be minimal if at all noticeable. In most cases, a non-working headline/image test results in your visitors seeing the original headline/image from your CMS as usual, so the failed test will be unnoticeable.
- The following variables are initialized prior to chartbeat_mab.js loading in the
<head>:domain,uid,flickerControl,useCanonicalanduseCanonicalDomainORpath.
Initiate a few headline tests in different positions to verify that headlines are being replaced successfully by the Chartbeat code on your staging or production website. If this is your first time initiating a Chartbeat headline/image test, learn how in this Help Center article. Before getting started with your first test, review our tips for QA'ing your integration at the top of this page.
The Chartbeat code on your site should include the following variables:
_sf_async_config.articleBlockSelector: Must be set in the Chartbeat snippet only if your site is not using <article> tags to encapsulate all headline and image <a> tag(s)._sf_async_config.lazyLoadImageAttributes: Must be set in the Chartbeat snippet only if your site makes use of custom lazy load attributes for your images outside ofdata-srcanddata-srcsetwhich our code will detect automatically._sf_async_config.domain: For testing on a staging site, this value should be set to the Site ID the staging site dashboard that has been created on your account. This is typically the root domain of the staging site._sf_async_config.flickerControl: If you are implementing flicker control for headline and image testing, this variable must set to false and must be loaded in the code snippet prior to the chartbeat_mab.js script.
If you are planning to test above the fold, flicker control code must be inserted in the
<head>:- For optimal performance, the flicker control code should be loaded early in the page's head code.
- There are two variables that must be set in the flicker control code:
headlineSelectorandarticleBlockSelector. - A third variable,
timeout, can be modified to allow more time for the Chartbeat Javascript to finish loading.
Once your test environment is setup with Chartbeat Image Testing code and the corresponding configuration variables, we highly recommend starting image tests on multiple areas of your homepage to determine compatibility with your page markup. Tests should be started in multiple homepage areas above and below the fold, and should include both a variant image and headline. With tests running, you will need to load the test homepage in a Chrome Incognito window until the variant image and headline is served to verify successful replacement. Repeat for all areas where you anticipate your homepage team running headline and image tests.
If this is your first time initiating a Chartbeat headline/image test, learn how in this Help Center article. Before getting started with your first test, review our tips for QA'ing your integration at the top of this page.
- Tests should be initiated in different positions on your homepage to determine image test compatibility with each position's unique HTML markup format.
- Tests should be started with a single test variant (variant B): a different headline and a different image than the original (variant A). The headline and image should be easily distinguishable.
- Once served a variant, the browser should be updated to simulate a mobile device. After simulating a mobile device and refreshing the page, you should confirm that the variant image and headline are still being shown.
- In all tests, the variant headline should always be shown with its corresponding variant image. You should never see a scenario where the variant headline is shown with the original image.
A headline and image test should be started above the fold to evaluate the flicker control implementation:
- Images and headlines should be hidden until either chartbeat_mab.js loads or the timeout, 1-second by default, elapses.
- If the timeout elapses, the original headline and image should appear to the user and a trial should not be recorded.
- If the timeout does not elapse and chartbeat_mab.js loads, the user will be served either the original headline and image or a variant. In either case, a trial should be recorded.
- When images and headlines are eventually shown, users should not see any flicker effect.
A headline and image test should be started below the fold to evaluate the lazy load implementation:
- The variant headline should always be shown with its corresponding variant image. You should never see a scenario where a variant headline is shown with the original image.
You can use your browser's site inspector tools to monitor network activity from your site, filtering on "chartbeat" to view all Chartbeat related requests made from your page.
Filtered on "chartbeat" in the Network tab of DevTools, here's the expected order of requests from a standard implementation of our snippet with mab.js included:
- 1.chartbeat_mab.js
- 2.chartbeat.js (or chartbeat_video.js)
- 3.mab.chartbeat.com/mab_strategy/headline_testing/get_strategy/
- 4.mabping.chartbeat.net/ping/mab
- 5.ping.chartbeat.net/ping (repeated every 15 seconds)
- Headline and Image Testing ✅
Last modified 1yr ago