AVJ Citation Index — Methodology – AI Visibility Journal

Methodology v0, published October 23, 2026. This version describes the design for the first, beta edition of the Index (v0), planned for December 8, 2026. The design will be locked after internal review and before data collection begins; any change made then, or later, will be listed in the changelog at the end of this page.

The AVJ Citation Index measures which websites AI answer engines cite as sources when they answer a fixed, published set of realistic questions in the United States, the United Kingdom, Germany, France and Australia, and how stable those citations are from one run to the next and from month to month.

What the Index measures, and what it does not

The Index answers three questions:

Which domains receive the largest share of citations, overall, by AI system and by market?
How often does each domain appear at least once in an answer?
How much do citations change between repeated runs of the same question, and from month to month?

Topic-level breakdowns are exploratory and will not be reported in v0, because each topic has too few questions to meet our minimum sample (see Statistics).

It does not measure traffic, clicks, revenue, how a specific person sees a consumer app, or brand sentiment.

The headline Index is about citations. Brand mentions are recorded separately and, when reported, appear in a clearly labeled module.

Terms used on this page

Term	What we mean
Answer	The full response returned for one run of one question on one AI system
Run	One request of one question to one AI system at one time
Citation	A linked source URL that an AI system attaches to its answer
Citation slot	One position in an answer’s list of citations; the same domain can fill several slots in one answer
Mention	A brand or organization named in the text of an answer, with or without a link
Domain	The registrable domain of a cited URL, for example `wikipedia.org` for `en.wikipedia.org`
Cell	One combination of AI system, market, topic and edition (v0 reports cells by AI system and market only)

These definitions match our general Methodology page and the AVJ Glossary.

The question panel

Questions reflect real information needs in each market. They do not name brands or years and avoid leading wording. Questions for Germany and France are written by native speakers, not translated. Questions that ask about a single brand are excluded from the Index. Each question is reviewed by two people for naturalness, neutrality and market fit.

The panel is frozen for each edition and published with it, so results can be checked. The panel size and the number of questions per market and topic for v0 are still being decided; they will be locked before data collection and published with the edition.

How answers are collected

We collect answers only through compliant routes: official APIs whose terms permit this use, data licensed from a provider whose collection and publication rights we have checked in writing, or small samples recorded by a person. We do not scrape consumer interfaces, use proxies to get around restrictions or bypass access controls.

Each edition names the AI systems covered, how each was accessed, the model where known, and any third-party data provider. The access route, the AI systems covered and the exact model requested from each system for v0 are still being decided; they will be locked before data collection begins and recorded in the changelog. AI systems for which we have no compliant route are not included in the Index.

Collection settings:

Repeated runs. Each question is run several times on each AI system within a fixed collection window, so that we can measure run-to-run variation. The number of runs and the collection times are fixed in a run configuration before collection. The v0 collection is planned for November 9–11, 2026.
No conversation history. Every run starts a new conversation.
Default behavior. We add no instructions about sources or answer length and use each system’s default settings, which each edition documents.
Approximate location. Where a system lets us set a user location, we set it to the market. This approximates a user in that market; it does not reproduce a real user’s account, history or settings. Differences between markets are reported descriptively only: each market has its own questions, and German and French questions are in other languages, so a difference between markets is not an effect of location.
Failures are data. Transport errors are retried at most twice. Refusals and answers without sources are not retried; they are counted.
Storage. We keep the full raw response for every run, unchanged, plus a parsed table.

How citations and mentions are processed

Citations are taken from the structured citation fields each system returns, not from the answer text, and their order is preserved (position 1, 2, 3 and so on).
URLs are normalized: scheme and host lowercased, fragments and tracking parameters (such as utm_*, gclid and fbclid) removed.
Domains are reduced to the registrable domain using the Public Suffix List. Domain families, such as youtube.com and youtu.be, are merged using a published mapping file.
Self-exclusion. aivisibilityjournal.com and any AVJ-owned domains are excluded from ranked tables and reported in a footnote if cited.
Mentions are identified by matching a dictionary of brand names and aliases, plus extraction assisted by a large language model (LLM). People check a sample of at least 5% of answers (minimum 50 answers), and we report precision and recall from that check.
Answer status. Each answer is classified as ok, refusal (declined to answer), no-citation (answered without sources) or error.

Metrics and formulas

Notation: A is the set of valid answers (status ok) in a cell. For an answer a, C(a) is the set of distinct domains it cites, c(a) is its list of citation slots, and M(a) is the set of brands mentioned in its text. d is a domain and b is a brand.

Metric	Definition	Formula
Citation share (headline ranking)	Share of all citation slots that go to domain d	(citation slots held by d, summed over all a in A) ÷ (all citation slots, summed over all a in A)
Presence rate	Share of answers that cite d at least once	(number of answers in A where d is in C(a)) ÷ (number of answers in A)
Mention rate (mentions module)	Share of answers whose text names brand b at least once	(number of answers in A where b is in M(a)) ÷ (number of answers in A)
Prompt reach	Share of questions for which d is cited in at least 50% of runs	questions meeting the threshold ÷ all questions
Average position	Mean position of d in the citation list when it is cited (1 = first)	reported only when d is cited in at least 30 answers
Citations per answer	Mean number of citation slots per answer	mean number of slots in c(a), by AI system
No-citation rate	Share of answers with no citations	answers with status `no-citation` ÷ all answers
Refusal rate	Share of answers that decline to answer	answers with status `refusal` ÷ all answers
Source-type mix	Share of citation slots by domain category (publisher, retailer or brand, forum or user-generated, video, reference, government or education, review site, other)	categories from a published, human-coded taxonomy; two coders must agree at κ ≥ 0.70 on a 200-domain sample
Run-to-run volatility	How much the cited domains change when the same question is run again	for each question, 1 − the mean pairwise Jaccard similarity of C(a) across runs in the window; reported as the median and interquartile range across questions
Month-over-month churn	How much the top domains change between editions	1 − Jaccard similarity of the top-50 domain sets of consecutive editions, plus rank-biased overlap (p = 0.9); starts with the second edition
Concentration	How much of all citations go to the largest domains	share of citation slots held by the top 10 domains, and the Herfindahl–Hirschman index over domains

Citation metrics and mention metrics are never combined into a single score. A domain can be cited without its brand being named, and a brand can be named without any of its pages being cited.

Statistics

Questions are the unit of uncertainty. Answers to the same question are related, so confidence intervals come from a cluster bootstrap that resamples questions (at least 2,000 resamples, fixed random seed), stratified by market.
Intervals. Estimates are shown with 95% confidence intervals.
Noise band. A month-to-month change for a domain is reported as detectable only if the 95% interval of the difference excludes zero and the change is larger than the noise band: the 95th percentile of changes between two random halves of the same month’s data, recalculated every edition. Everything else is reported as “no detectable change.”
Many comparisons. When we screen many domains at once, we control the false discovery rate with the Benjamini–Hochberg method (q = 0.05).
Minimum sample. Estimates based on fewer than 30 questions or fewer than 100 answers are not shown. In v0, each topic has fewer than 30 questions, so no topic-level results are reported.
Weighting. All questions count equally within each market and topic, and all markets count equally. We do not weight by search or prompt volume, because no reliable, comparable volume data exists across AI systems and markets.
Exploratory analyses are labeled as such.

Publication cycle

From 2027, each monthly edition follows the same steps: freeze the panel and settings, collect on days 1–3 of the month, check data quality (at least 95% of planned runs completed for each AI system and market), analyze, have a second person reproduce the analysis, fact-check, then publish the edition, its dataset and a changelog.

The v0 (beta) edition is an exception. It uses the first three days of our launch study’s collection (November 9–11, 2026) instead of its own window and is planned for publication on December 8, 2026. Month-over-month churn starts with the January 2027 edition.

Limitations

These limitations apply to every edition and are repeated in each one:

API results are not consumer apps. Consumer apps add personalization, memory, system instructions and model routing. API results are a proxy, and we label them that way.
Coverage gaps. AI systems without a compliant collection route are not in the Index.
The panel is not the population. Questions are designed, not sampled from real traffic. Results describe this panel, and equal weighting is not real-world importance.
Location is approximate. Language and account settings are not simulated.
Each edition is a snapshot. AI systems change models and retrieval often.
Model names can hide changes. A returned model name may not reveal changes in retrieval.
A citation is not proof of influence. A cited source may not be what shaped the answer, and uncited sources may have.
Parsing. Citation fields differ between systems; references that appear only inside the answer text may be missed.
Domain level. Domain-level results hide differences between pages.
Markets are not like-for-like. Markets use different questions and languages, so market comparisons describe differences; they do not isolate the effect of location.

Data release

We plan to release, under a CC BY 4.0 license, the question panel, run settings, per-answer citation lists (URLs, domains and positions), metadata, metrics and analysis code for each edition, with a DOI for each dataset. Full answer text is released only where each provider’s terms allow it; otherwise we release what is needed to verify the data without redistributing the text.

Versioning

Methodology versions follow MAJOR.MINOR.PATCH. A MAJOR change breaks comparability (for example, a change of AI systems, metric definitions or more than 20% of the panel); trend lines then restart or are bridged by running the old and new designs in the same month. A MINOR change keeps comparability. A PATCH fixes errors or documentation.
This page is v0: the design for the beta edition. The version number assigned when the design is locked, after internal review and before data collection, will be recorded in the changelog.
Every edition states the methodology version and panel version it used.
Errors are corrected under our corrections policy, and earlier dataset versions remain available.

Conflicts of interest and independence

AVJ’s own domain is excluded from ranked tables. No company can pay to be included in, excluded from or ranked in the Index, and no sponsor or vendor sees results before publication. If we use data from a commercial AI-visibility vendor, we name the vendor and describe its collection method as the vendor states it. See our Ethics & Independence Statement.

Review and registration

The v0 design will be checked internally before it is locked: AI models from two different companies write critiques, and the editor-in-chief answers each point in a written memo. We describe this as “design checked internally.” The Index has not been independently or peer reviewed, and we will not describe it that way unless it is.

We plan to pre-register the launch study, including the Index v0 analysis in an annex, on the Open Science Framework (OSF) before data collection begins. We will link to the registration here once it exists.

Changelog

October 23, 2026: Methodology v0 published, replacing the interim page. Added term definitions, formulas for every metric (including the mention rate), collection settings, statistical rules, publication cycle, limitations, data release and versioning. Same day: clarified that topic-level breakdowns are exploratory and not reported in v0, that market differences are descriptive only, and that the panel size, access route and models are still to be locked before data collection. The design will be locked after internal review and before data collection.
October 22, 2026: Interim methodology published ahead of the first edition.

Questions or criticism: [email protected]