How We Score: The 7-Category Methodology
Steiner Index Team · August 2026 · 6 min read
How do you score an AI platform? It's the question we get most often, and frankly, the one that matters most. A benchmark is only as good as its methodology. If the methodology is opaque, the score is just an opinion with a number on it.
So here's ours. Every decision, every weight, every data source: all public.
The Framework
Each platform is evaluated across seven weighted categories. Each category is scored 0–100, then combined into a single weighted composite: the Steiner Index (0–100).
The formula is simple:
Steiner Index = Σ (category score × category weight)
No black box. No proprietary algorithm. Just transparent math.
The Seven Categories
1. Capability & Performance (25%)
The heaviest weight, and the most scrutinized category. This measures output quality, task accuracy, and benchmark performance relative to category peers. We pull from published benchmark results, independent evaluations, and structured internal testing. This is the category most likely to be disputed by platforms, which is exactly why we require independent verification before it goes live.
2. Reliability & Uptime (15%)
Does the platform actually work when you need it? We measure historical uptime, latency consistency, incident frequency, and, critically, how transparent the platform is about outages. Data comes from public status pages and third-party monitoring services. A platform that quietly goes down and quietly comes back up scores lower than one that posts a public incident report.
3. Data Privacy & Trust (15%)
How does the platform handle your data? We evaluate published data-handling policies, training-data opt-out options, security certifications (SOC 2, ISO 27001 where applicable), and transparency reporting. This weight is likely to increase as AI regulation develops: privacy isn't just a feature anymore, it's a legal and ethical requirement.
4. Pricing Transparency & Value (10%)
Can you understand what you're paying for? We assess clarity of pricing, hidden costs, and value relative to the platform's capability tier. ChatGPT, for example, gets docked here for a six-tier structure that makes it confusing which model you actually get. Claude, by contrast, scores highest among frontier labs for stable pricing and clear tiers.
5. Ecosystem & Integration (10%)
How well does it play with others? We evaluate API quality, third-party integrations, and developer tooling. Gemini benefits here from deep Google Workspace integration. Midjourney loses points: no mainstream public API, runs largely through Discord. Copilot gains from Microsoft 365 ecosystem depth.
6. Update Velocity & Roadmap (10%)
Is the platform actively improving? We measure the frequency and substance of meaningful updates, plus responsiveness to user feedback. Changelogs, release notes, and product announcements are our primary sources. ChatGPT leads here with the fastest release cadence and largest plugin/integration footprint.
7. Community & Support (15%)
Can you get help when you need it? We look at documentation quality, support responsiveness, and community size and health. This includes support ticket response benchmarks, community forum and Discord activity, and documentation audits. A great platform with terrible support is a worse product than a good platform with great support.
The V1 Scorecard
Our rough v1 pilot scorecard, directional numbers from July 2026, not final scores, gives you a sense of where the landscape sits:
| Platform | Capability (25%) | Reliability (15%) | Privacy (15%) | Pricing (10%) | Ecosystem (10%) | Update (10%) | Community (15%) | Steiner Index |
|---|---|---|---|---|---|---|---|---|
| ChatGPT | 92 | 85 | 75 | 65 | 95 | 95 | 95 | 87 |
| Claude | 90 | 88 | 82 | 85 | 80 | 85 | 78 | 85 |
| Gemini | 88 | 85 | 78 | 70 | 90 | 88 | 82 | 84 |
| Midjourney | 88 | 85 | 70 | 92 | 55 | 75 | 85 | 80 |
| Perplexity | 80 | 82 | 75 | 88 | 70 | 80 | 72 | 78 |
| Copilot | 78 | 80 | 80 | 75 | 90 | 75 | 70 | 78 |
| Grok | 80 | 75 | 65 | 80 | 65 | 82 | 68 | 74 |
| DeepSeek | 82 | 70 | 55 | 90 | 60 | 85 | 65 | 73 |
These are directional. They're built from quick searches on pricing, positioning, and reputation, not the full benchmark/uptime/certification sourcing the methodology calls for. They're a starting point, not a verdict.
What Drove the Calls
A few notes on why platforms landed where they did:
ChatGPT leads on capability, ecosystem, and update velocity, the largest plugin/integration footprint and fastest release cadence, but pricing transparency is docked for that six-tier structure. Claude scores strongest on pricing transparency and privacy among frontier labs. Gemini benefits from Google ecosystem depth but lost points after multiple tier renames. Midjourney stands out for pricing stability (unchanged tiers for 3+ years) but scores lowest on ecosystem. Grok and DeepSeek score lowest on privacy/trust: both have less publicly documented data-handling policy, which is a real gap, not a subjective knock.
Before This Goes External
Three things need to happen before these numbers meet investors or the platforms themselves:
- Replace directional scores with sourced data: published benchmarks, verified uptime, actual security certifications.
- Get a second independent read on capability scores: that category is the most subjective and the easiest to dispute.
- Add a visible last-updated date and source citations per cell.
Governance
No platform can pay to influence its score. Any commercial relationship between Steiner Index and a scored platform must be publicly disclosed on that platform's score page.
Scores are refreshed on a rolling basis: monthly for fast-moving categories like capability, quarterly for slower-moving categories like privacy policy. A visible methodology changelog tracks any change to weights or criteria, so historical scores remain comparable and auditable.
And here's the part most benchmarks won't do: disputed scores get a public right of reply. Platforms can submit a rebuttal that appears alongside their score. If they think we got it wrong, they can say so, right there on the same page.
That's not weakness. That's integrity. A benchmark that's afraid of being challenged isn't a benchmark. It's a marketing tool.
We're building something different. And this is how.