The Oracle AI Fight Prediction Model: 2024 Accuracy Report

The Numbers: Overall Accuracy and Context

The Oracle AI fight prediction model delivered an overall accuracy of 69.8% across 53 verified MMA fight predictions. Out of these, 37 picks were correct, with no draws or no contests affecting the results. This places The Oracle AI in the upper tier of public MMA prediction models, as typical expert-level accuracy in this domain ranges from 60% to 70% over large samples.

A key metric for probabilistic models is the Brier Score, which measures the accuracy of predicted probabilities (lower is better). The Oracle AI achieved a Brier Score of 0.226. For context, a Brier Score below 0.15 is considered elite, below 0.20 is good, and below 0.25 is average. The Oracle’s score falls in the average-to-good range, indicating reasonable calibration but leaving room for improvement, especially in high-confidence scenarios.

Additionally, the model’s method accuracy—which reflects the accuracy of predicting not just the winner but also the method of victory—was 66%. This is a challenging metric in MMA, where method prediction is notoriously difficult, and the model’s performance here is in line with industry standards.

Confidence Calibration: How Well Did the Model Predict Its Own Success?

A transparent AI model should not only predict winners but also accurately assess its own confidence. The Oracle AI’s performance by confidence tier reveals important insights:

• Lock (85%+ confidence): 0/0 (no Lock picks made). This means the model did not identify any fights as near-certainties, so we cannot assess its performance in this tier. The expected win rate for Lock picks is 92.5%.
• High (70-84% confidence): 12/19 correct (63.2% actual vs 77% expected). Here, the model underperformed its own expectations by 13.8 percentage points, indicating overconfidence in this range.
• Medium (60-69% confidence): 11/12 correct (91.7% actual vs 64.5% expected). This is a significant positive edge (+27.2%), suggesting the model was too conservative and could have rated these picks higher.
• Low (50-59% confidence): 14/22 correct (63.6% actual vs 54.5% expected). Again, the model outperformed its own confidence by 9.1%, showing a positive edge in low-confidence scenarios.

In summary, The Oracle AI was well-calibrated or even underconfident in medium and low tiers, but overconfident in the high-confidence range. No Lock picks were made, so the model’s ability to identify true “sure things” remains untested.

What the Grade Means: Honest Interpretation

The Oracle AI’s B grade reflects solid, above-average performance, but not elite status. The model’s overall accuracy and Brier Score are competitive with leading public MMA predictors, but the underperformance in high-confidence picks is a clear area for improvement. Transparency is crucial: the model did not make any Lock picks, and its high-confidence predictions were less reliable than expected.

It’s also important to note the sample size—53 fights is a meaningful but not definitive dataset. Variance can still play a role, and continued tracking will provide a clearer picture of long-term reliability. Users should interpret these results as a strong but not infallible guide, especially when the model expresses high confidence.

Methodology: How the Oracle AI Works

The Oracle AI uses a 57-module ensemble engine, where each module specializes in a different aspect of MMA analytics. Predictions are made via a weighted voting system and are recorded before each fight, then verified against official results. Draws and No Contests are excluded from all accuracy calculations to ensure fairness. For a full breakdown of the model’s architecture and verification process, see our methodology page.