Skip to content
alikhan.tech

Writing

XM Cloud A/B/n Testing at the Edge — A Developer’s Setup Notes

Muhammad Ali Khan · 01 Nov 2026 · 06 Mins read

XM Cloud A/B/n Testing at the Edge — A Developer’s Setup Notes

The test looked conclusive. Variant B won by a margin the marketing lead could quote, and the recording they showed me was real. What the recording did not show was the CDN. We had cached the HTML of the page at the edge under the public URL, with no variant in the cache key, and the personalize middleware never got a vote for most visitors. Variant B won among the people who missed the cache: editors, the QA VPN, and anyone who sent a header our rule did not expect. I have set this up wrong in both directions. Caching nothing makes the edge middleware tax every request. Caching the anonymous page makes the test a fiction.

Sitecore's docs still describe this under the XM Cloud add-on even though the product name on the tenant is SitecoreAI. I use their names for the APIs and our name for the tenant. These are the setup notes I want in the repo before anyone creates the experiment in the page builder.

What the middleware actually does

Page personalization and component A/B/n testing ship in the Next.js XM Cloud add-on. Component tests need JSS 22.1 or later. From JSS 21.6 the older nextjs-personalize add-on is obsolete and the XM Cloud add-on is the one that carries this. Content SDK apps run the same personalization through the middleware plugin the current guides call personalize. Marketers create the variants in the page builder. When the page is published, the variants are part of the layout data in Experience Edge. No extra component code is required for a test between two datasources of a rendering the app already knows.

On a request, the middleware does three things. It asks Experience Edge which variant ids exist for the route. It asks Personalize, with the request and user context, which variant to serve. It rewrites the request onto a personalized path that the app knows how to read. The Cloud SDK stores a cookie named sc_{sitecoreEdgeContextId}_personalize. Sitecore's Cloud SDK docs say that cookie is required for A/B/n tests to run. A consent script that deletes every cookie it does not recognize will silently end your test, and the symptom is that everyone sees the default. I put the cookie name in the consent ticket so it is classified on purpose.

The Edge query the middleware is built around is small:

query ($siteName: String!, $language: String!, $itemPath: String!) {
  layout(site: $siteName, routePath: $itemPath, language: $language) {
    item {
      id
      version
      personalization {
        variantIds
      }
    }
  }
}

A response with a variant id means this route has something to decide. An empty list means the middleware should leave the path alone. I have debugged "personalization is down" by reading this payload and finding that the variant was never published to Edge. The middleware was healthy. The publish was not.

The rewrite path is the cache key, if you let it be

Sitecore's JSS 22 documentation gives the path structure as /_variantId_<variantId-1>/[more variant segments]/<path>. A page variant 1234 of /foo/bar is /_variantId_1234/foo/bar. Component variants are separate segments, and the example they publish is /_variantId_component1_1234/_variantId_component2_5678/foo/bar. The app strips that prefix before it fetches layout data, then applies the variant to the layout. You are not supposed to build these paths by hand. You are supposed to recognize them in a log when a rewrite did not happen.

Two documented limits decide the hosting design. Static generation can be used for personalized page variants, because the variant id is in the path and the page can be generated for that path. Static generation does not currently work with component-level A/B/n testing. I do not "try SSG anyway" for a component test. The page has to be rendered dynamically so the middleware's choice is the choice the visitor gets.

The second limit is ours. A CDN rule that keys only on the public URL, and that runs in front of the middleware, will store whichever HTML happened to be generated first. The rewrite path never becomes a cache key if the HTML is cached before the rewrite. I cache static assets aggressively. I do not cache HTML for a route that has a running component test. For a page-level test, I cache only if the cache key includes the variant segment, and only after the middleware has rewritten. If the host cannot express that, the route stays dynamic for the life of the experiment.

Timeouts, exclusions, and the order of plugins

The starter's personalize plugin sets an order that leaves room for redirects to run first, and it configures two timeouts that default to 400 milliseconds in the samples I see in current starters: one for the Edge call, one for the Personalize call. Four hundred milliseconds is a product choice. Raising it hides a slow origin. Lowering it turns a slow Personalize response into the default variant and biases the test. I log a timeout as a distinct event. A test that "has low traffic on variant B" is sometimes a test whose B decision is slower than the timeout.

disabled belongs to the consent decision, not to my impatience. The sample turns the middleware off when NODE_ENV is development, which is right for local work and wrong as the only production switch. I drive disabled from the consent cookie the site already uses. A visitor who has rejected analytics cookies gets the default experience, and they are excluded from the test on purpose.

excludeRoute is how I keep the middleware off paths that must not pay for it and must not vary. The default already ignores files and Next.js API routes. I add the obvious others.

const PERSONALIZED_PREFIX = ["/account", "/api", "/sitecore"];

export function excludeRoute(pathname: string): boolean {
  if (pathname.includes(".")) return true;
  return PERSONALIZED_PREFIX.some((prefix) => pathname.startsWith(prefix));
}

Authenticated account pages that vary by user are a separate personalization problem, and stacking them under the same edge rewrite has gone badly for me. Search pages with a query string are the other exclusion I consider per site. A test on a keyword results page fragments the cache and usually measures the query, not the component.

Plugin order is the Content SDK footgun I hit during the JSS migration. On the Pages Router the documented order is multisite, then redirects, then personalize. On the App Router it is locale, then multisite, then redirects, then personalize. Personalize runs after redirects so a redirected visitor is tested on the destination, not on the URL they typed. I have seen the opposite, and the test then collected data for a page that immediately 301'd. The Cloud SDK page-view event has to fire for the page the visitor actually saw, using the variant the rewrite selected. An event for the pre-rewrite path attributes the conversion to the wrong arm.

What I configure in the page builder, and what I refuse

A component test compares variants of a rendering the app already ships. The variant is a datasource or a presentation difference the layout service can return. It is not a new React component that exists only in the experiment. If variant B needs a rendering the component map does not contain, the test is a deploy, and I say so before anyone starts it.

Page-level variants are the ones static generation can key. I still publish them, and I still confirm personalization.variantIds on Edge for the route, before I trust a report. Sitecore documents that certain personalizations run at the edge. I read that as the middleware and the Personalize call described above, not as permission to skip the publish.

I do not run a component A/B/n test and a full-page CDN cache on the same route in the same week. I do not turn the middleware off in production to "make TTFB look like the slide." The cost of the Edge and Personalize calls is real, which is why exclusions and a sane timeout exist. The cost of a cached single variant is a decision the business thinks it made.

The checklist I run when a test is about to start

  1. Confirm the JSS or Content SDK version understands component tests, and that the personalize plugin is actually registered. A copied starter that lost the plugin looks identical in the page builder.
  2. Publish the page. Query Edge and read personalization.variantIds for the route and language under test.
  3. Load the public URL with a fresh profile, confirm the rewrite in the middleware log, and confirm the sc_…_personalize cookie survives consent.
  4. Confirm HTML cache does not key solely on the unwritten public path while the test is live. Confirm component tests are dynamic.
  5. Click through to the conversion on both arms and find both variant ids in the page-view events. A test with no events is a UI toggle.

When I follow that list, the edge is doing the job Sitecore documented: pick a published variant per request, rewrite, render, record. When I skip the cache step, the edge serves yesterday's winner to everyone, and the experiment report is a diary of cache misses. I would rather have a boring dynamic route for two weeks than a fast page that pretended to run a test.

Related notes