When a prospective customer asks an AI assistant for product recommendations in your category, does your brand appear in the answer?
Unlike traditional search engines where SERP rank tracking tools (like Ahrefs or Semrush) monitor keyword positions automatically, measuring visibility in generative AI models requires a prompt-based benchmarking methodology.
AI responses are non-deterministic—they vary based on user context, prompt phrasing, and model updates. Building a disciplined, repeatable benchmarking protocol allows you to track inclusion trends over time.
The AI Visibility Measurement Framework
Benchmarking Workflow
├── Step 1: Build a 30-Prompt Customer Intent Library
├── Step 2: Establish Test Execution Protocols (Clean Context)
├── Step 3: Track 4 Core Metrics (SoR, Inclusion, Sentiment, Spec Match)
└── Step 4: Map Missing Attributes to Catalog Remediation
1. Building a Prompt Library (30 Buyer Intent Scenarios)
Create a standardized library of 30 buyer prompts across four distinct query types:
Prompt Library Categories
├── Category A: Budget-Constrained Queries ("under ₹5,000")
├── Category B: Use-Case Specific Queries ("for high-altitude trail running")
├── Category C: Attribute / Ingredient Queries ("fragrance-free, sensitive skin")
└── Category D: Direct Category Comparisons ("Brand X vs Brand Y")
Sample Prompt Matrix:
| Query Type | Example Benchmark Prompt | Target Product / SKU |
|---|---|---|
| Budget + Intent | "What are the best minimalist running shoes under ₹7,000 for daily training?" | Men's Trail Runner v2 |
| Attribute Filter | "Recommend a non-comedogenic serum with Niacinamide for oily skin." | Clarifying Serum 30ml |
| Comparison | "Compare top Indian cookware brands for induction stove compatibility." | Tri-Ply Stainless Steel Pan |
| Gifting / Persona | "What is a good eco-friendly corporate gift under ₹2,000?" | Bamboo Office Desk Set |
2. Standardized Testing Protocol
To minimize response noise and output randomness, follow this standardized testing protocol:
- Use Incognito / Clean Context: Conduct prompt tests in fresh, logged-out browser sessions or via standard API calls to prevent personal conversation history bias.
- Standardize Temperature: When using API benchmarks, set temperature to
0.0or0.2for maximum factual consistency. - Test Across Major Engines: Run the prompt library across ChatGPT, Perplexity, Claude, and Google Gemini simultaneously.
- Repeat Monthly: Execute the benchmark suite on the 1st of every month to track visibility trends over time.
3. Core AI Visibility Metrics
Record outputs in a structured benchmark spreadsheet and compute four key metrics:
1. Share of Recommendations (SoR):
(Total times your brand is recommended ÷ Total prompts in suite) × 100
2. Product Inclusion Rate (PIR):
(Total times specific SKU is recommended ÷ Prompts matching exact product category) × 100
3. Top-Position Share:
Percentage of recommendation lists where your brand appears in Position #1.
4. Attribute Accuracy Score:
Percentage of extracted product features that match your actual product specifications.
Monthly Benchmark Scorecard Example:
| AI Discovery Platform | Prompts Tested | Brand Mentions | Position #1 Share | SoR (%) |
|---|---|---|---|---|
| ChatGPT (GPT-4o) | 30 | 18 | 6 | 60.0% |
| Perplexity AI | 30 | 22 | 11 | 73.3% |
| Claude 3.5 Sonnet | 30 | 14 | 4 | 46.6% |
| Google Gemini | 30 | 16 | 5 | 53.3% |
4. Connecting Benchmark Gaps to Catalog Fixes
When a competitor is consistently recommended over your product, analyze the recommendation rationale output by the AI engine.
Real-World Case Study:
USER PROMPT:
"Find me a lightweight daily sunscreen for sensitive skin that leaves zero white cast."
AI OUTPUT:
"I recommend Competitor Brand A. It features SPF 50, zinc oxide formulation,
is 100% fragrance-free, and specifically lists a non-greasy clear finish."
GAP DIAGNOSIS:
Your product page has identical specs, but your page text only read:
"Gentle daily protection for glowing skin."
REMEDIATION:
Updated product page HTML table & schema to explicitly list:
- Zinc Oxide 15% (Physical Sunscreen)
- Fragrance-Free, Paraben-Free
- Clear Finish / Zero White Cast Tested
Once the explicit attributes were added to the product page and schema, the product was included in 4 out of 5 subsequent benchmark runs.
Future Automated Tool: Referbro AI Brand Monitor
Referbro is building an automated brand monitoring module:
Referbro AI Brand Monitor
- Automates monthly execution of 50+ category prompts across ChatGPT, Perplexity, and Claude.
- Tracks Share of Recommendations (SoR), competitor mentions, and recommendation sentiment over time.
- Sends automatic alerts when catalog data gaps cause recommendations to drop.
Related Articles
- How to Check Whether AI Can Understand Your Shopify Products
- AI Commerce vs SEO: How Shopify Product Discovery Is Changing
- How to Track AI-Referred Traffic to Shopify