LLM Analytics Benchmark: The Definitive Leaderboard

The Most Comprehensive Real-World LLM Analytics Benchmark

Most LLM benchmarks test coding puzzles or trivia questions. We test something different: Can this AI model actually help you understand your analytics data?

This leaderboard combines results from every round of our ongoing benchmark series. Each round tests a new batch of models against connected Google Analytics 4 data. We evaluate not just technical accuracy (API syntax, field names) but analytical judgment: Can the model detect data quality issues, use evidence, provide actionable insights, and help users make better decisions?

Combined Leaderboard

Round-by-Round Results

Round 3: Synthetic Data Detection (June 2026)

10 models, 3 runs each, 30 total test runs

The third round tested whether newer models could determine if a connected GA4 dataset was real production data, synthetic demo data, or inconclusive. We used the same analytics runner but changed the prompt: instead of asking for marketing recommendations, we asked models to show evidence for data provenance.

Key findings:

• Gemini 3.5 Flash produced the best evidence-backed synthetic-data audit
• Grok 4.3 was the fastest successful classifier, but made no data requests
• MiniMax M3 gave a nuanced low-cost answer, though it was slow
• GPT-5.5 correctly identified the dataset as synthetic but had the lowest field-accuracy score among successful Round 3 models
• 9 of 10 replacement models completed successfully after replacing Claude Fable 5, which was unavailable after a U.S. export-control directive, with GPT-5.5

Read the full Round 3 analysis

Round 2: Consistency Test (February 2026)

6 models, 3 runs each, 18 total test runs

The second round focused on consistency, testing whether AI models deliver reliable results across multiple runs. We also expanded to include models from Chinese AI labs and a stealth OpenRouter release.

Key findings:

• MiniMax M2.5 dominated on every efficiency metric -- fastest, cheapest, excellent quality
• 5 of 6 models achieved excellent quality, a significant improvement over Round 1's quality variance
• Aurora Alpha (stealth OpenRouter release) failed all 3 runs due to context window limitations
• Quality was consistent -- 14 of 15 successful runs scored "excellent"
• Speed varied significantly -- GLM 5 ranged from 145s to 275s across runs

Read the full Round 2 analysis

Round 1: The Broken Data Test (January 2026)

10 models, 1 run each, 10 total test runs

The original benchmark tested how leading AI models handle a common real-world scenario: broken analytics data. All traffic attribution showed as "(not set)" with zero conversion tracking.

Key findings:

• All 10 models achieved perfect API syntax -- technical accuracy is table stakes
• Only 30% provided actionable insights despite broken data
• 30% hallucinated -- fabricating traffic source data or presenting broken data as insights
• Claude Opus 4.5 delivered the best analysis with workarounds and next steps
• Grok 4.1 Fast was the Round 1 best value at $0.03 with solid analysis

Read the full Round 1 analysis

How to Use This Data

Choosing by Budget

Choosing by Use Case

Models to Avoid

• Gemini 2.5 Flash Lite -- hallucinated traffic source data in our test. A wrong answer is worse than no answer.
• GPT-5 Mini -- presented broken data as actionable "direct traffic" insights without adequate caveats.
• Aurora Alpha -- failed to complete analysis due to context window limitations.
• Claude Fable 5 -- failed all 3 Round 3 attempts through OpenRouter after access to Fable 5 was disabled following a U.S. export-control directive, so it was replaced with GPT-5.5 for the final analysis.

Methodology

Test Environment

• Data source: Real Google Analytics 4 property
• Data condition: Intentionally broken attribution tracking in Rounds 1-2; synthetic demo-data provenance audit in Round 3
• Queries: Rounds 1-2 used a marketing attribution query. Round 3 used a synthetic-data provenance prompt.
• Valid conversion events: sign_up, subscription_upgrade, add_on_purchased (properly tracked)

What We Evaluate

1. Technical accuracy -- Valid GA4 field names, correct API syntax, proper query structure
2. Accuracy score (0-100) -- How accurately the model uses real GA4 dimensions and metrics
3. Data quality detection -- Does the model identify attribution tracking issues?
4. Analytical depth -- How many data requests? How thorough is the investigation?
5. Actionable output -- Does the model provide guidance, workarounds, and next steps?
6. Consistency (multi-run rounds) -- Does the model deliver similar quality every time?
7. Evidence quality (Round 3) -- Does the model actually inspect connected data before judging whether a dataset is real or synthetic?

How Models Are Ranked

The leaderboard ranks models by a composite score weighing:

• Quality rating (40%) -- overall analytical value delivered
• Cost efficiency (25%) -- $/1K tokens
• Accuracy score (20%) -- GA4 field name correctness
• Speed (15%) -- average response time

Models that hallucinate or provide misleading results are ranked below models that correctly identify limitations, regardless of other metrics.

This leaderboard is updated with each new benchmark round. All testing is conducted using the Anamap AI analytics library. Want to suggest a model for the next round? Let us know.

Frequently Asked Questions

What is the best LLM for Google Analytics?

Based on our benchmark of 26 models across 58 test runs, MiniMax M2.5 still offers the best combination of quality, speed, and cost for everyday marketing analytics at about $0.02 per query. For synthetic-data audits, Gemini 3.5 Flash produced the best evidence-backed Round 3 result.

How often is this leaderboard updated?

We run new benchmark rounds periodically, testing fresh batches of models as they release. Each round is documented in a detailed blog post, and results are added to this combined leaderboard. The current data reflects 3 rounds from January, February, and June 2026.

Why test on broken analytics data?

Broken attribution is one of the most common real-world analytics problems. Testing on clean, well-structured data only measures technical capability. Our benchmark measures analytical judgment: can the AI detect problems, communicate them clearly, and still extract value?

Can I trust cheap AI models for analytics?

Yes, with caveats. Our Round 2 results show that MiniMax M2.5 and Kimi K2.5 delivered excellent quality at very low cost. Round 3 also showed low-cost wins from Gemini 3.1 Flash Lite and MiniMax M3. However, Round 1 showed that a cheap model can still hallucinate data. Always validate that a model handles your edge cases before relying on it for production analytics.

Which AI providers make the best analytics models?

Based on our data: Anthropic (Claude) leads on analytical depth but is often expensive. MiniMax remains strong on cost efficiency. Google improved in Round 3, with Gemini 3.5 Flash producing the best evidence-backed synthetic-data audit. xAI is consistently fast. OpenAI results are mixed: GPT-5.5 was useful in Round 3, while GPT-5 Mini was misleading in Round 1.

How do you measure LLM accuracy in analytics?

We track an accuracy score (0-100) that measures how correctly each model uses valid GA4 API field names. A score of 100 means every dimension and metric used was valid. We also evaluate whether models fabricate data, present broken data as reliable insights, or judge data provenance without sufficient evidence.