Making Sitecore Search Ready for AI Discovery — Attributes, Extractors, and Content That LLMs Can Cite

Sitecore spent the back half of 2025 telling the market that discovery is moving from a list of blue links to answers. The 4 November 2025 SitecoreAI announcement said it directly: people meet a brand inside a feed or an AI-written summary, and the useful response is a governed message grounded in real content. Sitecore Search already had a feature for the on-site version of that problem. Questions and Answers, documented as a Sitecore Stream capability on top of your Search domain, generates answers from the content you indexed.

I spent this year watching Q&A groups, and a couple of external answer engines, fail in the same way. The model was fine. The index document was a title, a thumbnail, and an empty body. This post is the attribute and extractor work I do before I turn generation on. It is also the part I am willing to call engineering guidance rather than a product feature: nothing in Sitecore's docs says "make your pages citable by any LLM". The docs do say that an answer is only as good as the indexed items, and they return document_id and context with a Q&A hit. Citation, in the product, means pointing at a document you extracted on purpose.

Answers are an index feature

Sitecore's Q&A documentation is specific. You create a question-and-answer group under Administration, Domain Settings, Feature Configuration. The group has a name, one or more sources, an entity, and up to five attributes. Those attributes are what generation is allowed to read. Textual relevance has to be enabled on each of them before you generate. You publish the feature configuration, open the Q&A Browser, and run batch generation. The group gets an rfk_id. That id is what the widget request uses later.

If the source does not contain the fact, the API returns nothing. The same docs state that the feature supports English only. I re-read that page for multilingual sites, because a "currently" in the docs can move. As of the documentation I used for this post, I do not promise French or German answers from the product.

There is a second, quieter constraint. The visitor's keyphrase has to qualify as a question or a statement. A bag of keywords returns an empty Q&A payload even when the index has the document. I have watched teams file this as an API bug after wiring Q&A to the same box that serves keyword search. It is a documented qualification rule. Keyword search and answer generation are different requests. I issue them separately and I render an answer only when the response actually contains one.

Related questions use questions.related_questions.limit. An exact answer uses questions.exact_answer, and the example Sitecore publishes sets query_types to ["statement"]. You can send both in one widget item. The response includes the question, the answer, a context passage, and a document_id. That pair is the citation. I store it. I do not display an answer I cannot trace to a document in the group.

{
  "widget": {
    "items": [
      {
        "entity": "content",
        "rfk_id": "rfkid_product_docs",
        "questions": {
          "keyphrase": "How long does a publish take to reach the CDN?",
          "exact_answer": { "query_types": ["statement"] },
          "related_questions": { "limit": 3 }
        }
      }
    ]
  }
}

The Cloud SDK's server package describes the same request as QuestionsAnswersOptions: a required keyphrase, optional exactAnswer, optional relatedQuestions, optional rule. I keep the Next.js call on the server. An API key that can read the domain does not belong in a browser bundle, and the answer is not a client-side concern until it is HTML I chose to render.

Attributes are the product

Sitecore Search does not search your CMS. It searches index documents, and an index document is a set of attributes. The domain starts with a Content entity and a handful of default attributes. Everything else is one you create under Domain Settings, Attributes, and then extract.

The properties I set deliberately:

Attribute name. Alphanumeric, underscores, no spaces. This is the key in the extractor and the key in the API response when "return in API response" is on. I treat a rename as a migration. Widgets, sorting options, and Q&A groups all bind to it.
Data type. The type decides which features light up. A string can be a term facet. A number can be a histogram. A geo value is how I indexed coordinates back in 2024, and it is still a different type from a string that happens to look like "lat,lon".
Required. If the extractor cannot find a required attribute, Search does not index the item. I mark id, title, url, and type required. I do not mark a nice-to-have badge required, or a single missing badge drops the whole article.
Default value. Useful for a constant like type=article when the template implies it. Dangerous for description, because a default sentence will be the sentence the model cites.
Placement. Standard, or Locale when the domain is multilingual. Localized titles overwritten by a single shared value are a quiet relevance bug.

Use-for-features comes after the attribute exists. Textual relevance is configured at the domain and then weighted on the global widget. Sitecore's own guidance is to keep a single attribute's weight at or below 5 so one field does not dominate. I weight title above description, and description above a long content_text, then I look at real queries before I touch the numbers again. Facets have their own settings: aggregation, and/or behavior, order, and a max. The docs warn that alphabetical sorting of a large value set and a max above 50 hurt performance, and that 100 is the ceiling. Q&A does not require every attribute to be a facet. It requires the five you selected to carry text and to have textual relevance on.

A practical set for an article-like page:

Attribute	Type	Required	Why it exists
`id`	string	yes	Stable identity. Q&A returns this as `document_id`.
`type`	string	yes	Facet and a filter so Q&A is not reading PDFs and landing pages as if they were the same.
`title`	string	yes	Result link and the highest textual weight.
`url`	string	yes	The human-visible citation target. Canonical, not the crawler's redirect hop.
`description`	string	no	Short summary. Often the best exact-answer source.
`content_text`	string	no	Body text with nav, cookie banners, and related-links stripped.
`published_at`	timestamp	no	Sort, and a date I am willing to show next to a citation.
`language`	string	yes on multi-locale domains	Keeps an English Q&A group from reading the wrong locale.

That table is my default, not a Sitecore schema. Your entity might be a product, a location, or a document. The rule that travels is: if a human would need the fact in order to trust a quote, extract it as its own attribute. Do not hope the model will recover it from a blob of chrome.

Extractors decide whether the attribute is real

A source is either pull or push. Pull sources are the web crawler, the advanced web crawler, and the API crawler. Push sources take index documents through the Ingestion API. I use push when the content already lives in Sitecore and I can map fields myself. I use a crawler when the HTML is the source of truth, including pages the head application renders from Experience Edge.

The document extractor writes key: value pairs onto the index document. Sitecore's rule for non-JavaScript extractors is that they stop at the first match. A JavaScript extractor runs to the end. If the first h1 on the page is the cookie-banner title, an XPath rule for title will lock onto it and never see the article heading. I either tighten the XPath to the main region or I switch that attribute to JavaScript and ignore the chrome on purpose.

For an advanced web crawler the documented extractor types are XPath, CSS, and JavaScript. An API crawler adds JSONPath. URLs-to-match on a tagger keep a blog extractor off the search page and the cart. Multiple taggers mean multiple sets of documents. I want one tagger per entity, with one rule per attribute, until I have a reason to split.

Validation shows the values a URL would produce and creates nothing. It also does not execute client-side JavaScript. A Next.js page that paints the article in the browser validates as an empty shell. Sitecore's documented workaround is the one I use: turn rendering on, set MAX_URLS low, run the crawler, and read content_text on the indexed document.

Sitemap triggers use lastmod. A sitemap that stamps every URL with the request time forces a full recrawl. Emit the real modification time.

The citation payload I want in every document

This is the engineering guidance, labeled as such. Sitecore Q&A will cite document_id and context because that is the response shape. A third-party answer engine that fetches the public URL will only cite what the HTML contains. I make those two views agree.

For each URL I am willing to have quoted, the HTML and the index document both expose:

a canonical URL
one title
a description that a person could read aloud
a visible published or updated date
a stable id I also store in the index
body text that is present without executing a client-side bundle, where I can manage it

The last bullet is the hard one on a pure client render. If the article body arrives only after hydration, a crawler that is not configured to render JavaScript indexes the shell, and some external fetchers never render at all. I do not have a Sitecore switch that fixes someone else's crawler. The lever I do have is server-rendered HTML for the pages we want discovered, plus an index document we control through a push source if the crawler is the wrong tool.

A push document for the same contract looks like this:

{
  "document": {
    "id": "article-2026-10-27-cache-purge",
    "fields": {
      "type": "article",
      "title": "How long a publish takes to reach the CDN",
      "url": "https://example.com/docs/publish-and-cdn",
      "description": "A publish updates Experience Edge first. The CDN keeps the previous HTML until revalidation or a cache-tag purge runs.",
      "content_text": "Publishing a Sitecore item updates the content delivery API. HTML cached at the CDN stays stale until the site purges the tag for that path.",
      "published_at": "2026-10-27T03:00:00Z",
      "language": "en"
    }
  }
}

example.com is a stand-in, not a client. The body is short on purpose. Q&A was documented to read up to five attributes, not the entire CMS. I would rather extract two tight paragraphs than a 4,000-word page with the header menu concatenated in front of them. Concatenated chrome is how an answer quotes the login link.

A checklist I run before the first batch generation

I keep this next to the source configuration. It is all documented product behavior, in the order that usually hurts.

Create the attributes, including url, title, description, and content_text. Mark only the identity fields required.
Enable textual relevance on the attributes the Q&A group will use. Weight them on the global widget. Stay at or under a weight of 5 on any single attribute until you have query evidence.
Extract them. For a web source, confirm whether the rule is XPath, CSS, or JavaScript, and remember that only the JavaScript rule runs past the first match.
Validate a handful of URLs. Then do a rendered crawl with a low MAX_URLS if the site is a JavaScript app. Open an indexed document and read content_text with your eyes.
Create the Q&A group, pick the source and the entity, add no more than five attributes, publish feature configuration, and run batch generation. Note the rfk_id.
Call the widget with a full question and with a two-word keyword. The question should be eligible for an answer. The keyword should come back empty. If the keyword returns an answer, I do not trust the integration yet and I re-read the qualification note.
Render answer together with document_id and the url of that document. An answer with no link is not a citation. It is a paragraph I cannot defend.

The product documentation I am using limits Q&A to English. I still extract language for keyword search, and I do not point the English group at every locale's source.

A clean index does not guarantee that some other assistant will quote the site. That depends on crawlers Sitecore does not operate. Sitecore Search can ground an answer only in attributes you extracted, and the Q&A API returns the document identity and a context passage when it has one. A Next.js page that hides its text in the client feeds both an empty document.

Name the attributes. Extract the words a person would quote. Prove it with a small crawl. Then turn generation on.