Effective: October 22, 2026 · Last updated: October 22, 2026
This page explains the general principles behind AI Visibility Journal’s research. Each study and each edition of the AVJ Citation Index has its own detailed methodology. The Index methodology is at /index/methodology/.
Why measuring AI visibility is hard
AI systems do not return a fixed list of results. The same question can produce different answers depending on the system, model version, time, location, language, account state, conversation history and random variation. Any honest measurement has to account for this. We therefore treat AI visibility as something to be estimated from samples, with uncertainty, rather than “checked” with a single query.
Our principles
- Define before measuring. We state the research question, the systems studied and the metrics before collecting data. For confirmatory studies, we pre-register the hypotheses and analysis plan on the Open Science Framework (OSF) before collection, and link to the registration. Changes after registration are reported as deviations.
- Sample deliberately. Prompt sets are designed to represent real information needs. We document how prompts were written or selected, and their mix by topic, intent and market. Prompts do not name brands unless the study is about a specific brand.
- Repeat runs. We run each prompt multiple times and report how stable the results are, not just a single outcome.
- Record the context. Each observation records the AI system and, where available, the model or product version; the date and time in UTC; the market or approximate location; the language; and the collection route (for example an official API).
- Define every metric. We publish the formula for every metric we report, such as citation share or presence rate, so it can be recalculated.
- Report uncertainty. We give sample sizes and, where appropriate, confidence intervals. We do not report differences we cannot distinguish from noise.
- Separate observation from interpretation. Findings describe what we observed. Explanations of why are labeled as interpretation unless tested experimentally.
- State limitations. Every study has a Limitations section. Common limitations include the gap between API results and consumer apps, personalization we cannot observe, model updates during collection, and coverage limited to specific markets and languages.
- Publish data and code. Where licensing, terms of service and privacy allow, we release datasets under CC BY 4.0 with a persistent identifier (DOI), along with the code needed to reproduce our figures.
- Check reproducibility. Before publication, the analysis is re-run from the raw data and must reproduce every published figure.
Key terms
| Term | What we mean |
|---|---|
| AI visibility | Whether, how often and how accurately an AI system mentions, describes or cites a brand, organization or website |
| Citation | A link to a source that an AI system attaches to its answer |
| Mention | A brand or organization named in the text of an answer, with or without a link |
| Prompt panel | The fixed, documented set of questions we put to AI systems in a study |
| Run | One request of one prompt to one AI system at one time |
| Citation share | The share of all citations in a sample that point to a given domain |
| Presence rate | The share of answers that cite a given domain at least once |
| Volatility | How much the cited sources change when the same prompt is run again |
What every study publishes
Each research report includes, at minimum:
- the research question and, for confirmatory studies, a link to the pre-registration and any deviations from it;
- the AI systems, routes (for example, an official API), models where known, markets, languages and dates of collection;
- the data provider, if any, and what it did;
- the prompt panel or a description of it, and the number of runs;
- the definition of every metric and the statistical methods used;
- results with sample sizes and uncertainty;
- limitations;
- the data and code, or the reason they cannot be released;
- how the design was reviewed.
How we collect data
We collect AI-system outputs only through compliant routes: official APIs whose terms permit the use, data licensed from providers whose collection and publication rights we have checked, or small manual samples recorded by a person. We do not try to manipulate the systems we measure, except in clearly described, pre-registered experiments designed to cause no harm.
Data providers are always named. If a study uses data collected or licensed by a third party, its methodology names the provider, describes what it collected and how, and states any terms that affect what we can publish.
Design review
Before data collection, each study design is reviewed internally: AI models from two different companies write independent critiques, and the editor-in-chief answers every point in a written memo. We describe this as “design checked internally”. We do not describe a study as independently or peer reviewed unless it has been, and we will name the reviewers when it is.
Human judgment and agreement between raters
When outputs must be judged, for example whether an AI description of a brand is accurate, we use written scoring guidelines, have a sample scored twice, and report the agreement between scorers and who or what did the scoring.
Changes to methods
Methods improve. When we change a method, we version it, explain the change and its likely effect on comparisons, and do not present results from different methods as a continuous trend without saying so.
Independence
No company can pay to be included in, excluded from or ranked in our research. Sponsors, if we ever have any, will have no access to data before publication and no input into methods or findings. See Ethics & Independence.
Questions and challenges
We welcome scrutiny. If you believe a method is flawed, write to [email protected]. With your permission, we may publish substantive critiques alongside our response.