Global repeated permutation loss for a saved StackedEnsemble home-win probability classifier. This is separate from the training-pack SHAP margin model and EPA-OLS spread picks. No fitting occurs during this report.
Fit origin: Saved production-builder fit; original deployed object identity is not established.
Training seasons: [2016, 2017, 2018, 2019, 2021, 2022, 2023, 2024]; 4520 games, last kickoff 2025-01-21T00:30:00+00:00. Evaluation: 550 completed 2025 games, 2025-09-25T23:30:00+00:00 through 2025-12-13T20:00:00+00:00. Archive: 567 games; 17 explicitly excluded for missing continuous features. The private summary preserves excluded game IDs and missing-field reasons. No imputation is used. Both outcome classes are required.
Baseline Brier 0.185100; log loss 0.549005 (natural log; probability clipping at machine epsilon). Lower loss is better. 20 repeats, seed 235; each repeat uses the same row permutation across features.
Ranked by mean Brier delta. Positive means removing the feature's alignment with outcomes worsened loss; negative deltas are retained. One-hot columns for each category move as a block. The fixed model and feature order stay unchanged. SD and P05–P95 summarize permutation variation, not confidence intervals.
| Feature / category block | Brier Δ | Repeat SD | P05–P95 | Log loss Δ | Repeat SD | P05–P95 |
|---|---|---|---|---|---|---|
| home_elo | +0.025166 | 0.005303 | +0.015752 to +0.032374 | +0.058238 | 0.012172 | +0.037364 to +0.073123 |
| away_elo | +0.016098 | 0.002883 | +0.012564 to +0.020467 | +0.036375 | 0.006358 | +0.028883 to +0.045364 |
| away_adjusted_success | +0.008514 | 0.002623 | +0.004806 to +0.012154 | +0.019218 | 0.005809 | +0.010748 to +0.027983 |
| home_adjusted_success_allowed | +0.006861 | 0.002260 | +0.002516 to +0.010383 | +0.015818 | 0.005258 | +0.006160 to +0.024169 |
| away_adjusted_success_allowed | +0.003998 | 0.001273 | +0.002033 to +0.005716 | +0.009182 | 0.002757 | +0.004852 to +0.012495 |
| away_adjusted_epa | +0.003857 | 0.001997 | +0.000653 to +0.007266 | +0.008252 | 0.004398 | +0.000376 to +0.015302 |
| home_adjusted_passing_epa | +0.003444 | 0.001632 | +0.001193 to +0.006284 | +0.008895 | 0.003839 | +0.003491 to +0.016043 |
| home_adjusted_epa | +0.003301 | 0.001344 | +0.001386 to +0.005254 | +0.006967 | 0.002983 | +0.002973 to +0.010707 |
| away_talent | +0.003157 | 0.001192 | +0.001673 to +0.005062 | +0.006692 | 0.002644 | +0.002893 to +0.010820 |
| away_adjusted_epa_allowed | +0.003124 | 0.000794 | +0.002150 to +0.004263 | +0.006977 | 0.001847 | +0.004218 to +0.009140 |
| home_adjusted_success | +0.003103 | 0.002416 | -0.000461 to +0.006682 | +0.007857 | 0.005644 | -0.000613 to +0.015376 |
| away_adjusted_second_level_yards_allowed | +0.002201 | 0.001150 | +0.000691 to +0.003551 | +0.004722 | 0.002488 | +0.001165 to +0.007581 |
| home_adjusted_second_level_yards | +0.001870 | 0.000778 | +0.000518 to +0.002814 | +0.003907 | 0.001729 | +0.000781 to +0.006118 |
| home_adjusted_passing_epa_allowed | +0.001566 | 0.000523 | +0.000800 to +0.002304 | +0.003872 | 0.001166 | +0.002214 to +0.005545 |
| home_adjusted_line_yards | +0.001321 | 0.000856 | +0.000254 to +0.002682 | +0.003125 | 0.001817 | +0.000877 to +0.005828 |
| home_adjusted_standard_down_success_allowed | +0.001112 | 0.000866 | -0.000107 to +0.002378 | +0.002599 | 0.001960 | -0.000323 to +0.005581 |
| away_adjusted_passing_down_success_allowed | +0.001006 | 0.000418 | +0.000341 to +0.001674 | +0.002093 | 0.000848 | +0.000638 to +0.003521 |
| home_adjusted_epa_allowed | +0.000861 | 0.000718 | -0.000380 to +0.001478 | +0.001766 | 0.001587 | -0.001263 to +0.003239 |
| home_adjusted_rushing_epa_allowed | +0.000842 | 0.001001 | -0.000451 to +0.001791 | +0.002082 | 0.002211 | -0.001377 to +0.004602 |
| away_adjusted_passing_down_success | +0.000816 | 0.000560 | -0.000060 to +0.001569 | +0.001707 | 0.001294 | -0.000310 to +0.003529 |
| home_adjusted_pass_explosiveness_allowed | +0.000663 | 0.000467 | -0.000023 to +0.001383 | +0.001744 | 0.001026 | +0.000204 to +0.003301 |
| away_adjusted_standard_down_success_allowed | +0.000633 | 0.000370 | +0.000094 to +0.001098 | +0.001535 | 0.000829 | +0.000159 to +0.002562 |
| home_adjusted_rush_explosiveness_allowed | +0.000626 | 0.000674 | -0.000308 to +0.001539 | +0.001192 | 0.001497 | -0.001043 to +0.003234 |
| away_adjusted_pass_explosiveness_allowed | +0.000601 | 0.000166 | +0.000390 to +0.000769 | +0.001394 | 0.000365 | +0.000923 to +0.001752 |
| away_adjusted_rushing_epa | +0.000534 | 0.000716 | -0.000632 to +0.001488 | +0.001630 | 0.001648 | -0.001103 to +0.004183 |
| away_adjusted_line_yards | +0.000505 | 0.000279 | +0.000056 to +0.000842 | +0.001140 | 0.000678 | -0.000020 to +0.001951 |
| home_adjusted_passing_down_success_allowed | +0.000494 | 0.000395 | +0.000027 to +0.001257 | +0.001044 | 0.000847 | +0.000030 to +0.002792 |
| home_adjusted_open_field_yards_allowed | +0.000432 | 0.000423 | -0.000153 to +0.001064 | +0.001185 | 0.000926 | +0.000007 to +0.002603 |
| away_adjusted_explosiveness_allowed | +0.000347 | 0.000263 | +0.000074 to +0.000828 | +0.000818 | 0.000594 | +0.000179 to +0.001843 |
| away_adjusted_rush_explosiveness_allowed | +0.000235 | 0.000286 | -0.000117 to +0.000576 | +0.000477 | 0.000657 | -0.000397 to +0.001307 |
| home_adjusted_passing_down_success | +0.000199 | 0.000162 | -0.000067 to +0.000436 | +0.000501 | 0.000385 | -0.000086 to +0.001013 |
| home_adjusted_line_yards_allowed | +0.000168 | 0.000269 | -0.000210 to +0.000660 | +0.000428 | 0.000597 | -0.000366 to +0.001342 |
| away_adjusted_rush_explosiveness | +0.000142 | 0.000271 | -0.000322 to +0.000486 | +0.000138 | 0.000643 | -0.000976 to +0.001085 |
| home_adjusted_explosiveness_allowed | +0.000118 | 0.000244 | -0.000214 to +0.000399 | +0.000386 | 0.000539 | -0.000441 to +0.001042 |
| home_adjusted_rushing_epa | +0.000057 | 0.000926 | -0.001041 to +0.001492 | +0.000487 | 0.001992 | -0.002099 to +0.003543 |
| away_adjusted_explosiveness | +0.000049 | 0.000287 | -0.000466 to +0.000481 | +0.000244 | 0.000643 | -0.000829 to +0.001209 |
| away_adjusted_rushing_epa_allowed | +0.000030 | 0.000522 | -0.000828 to +0.000751 | +0.000347 | 0.001151 | -0.001424 to +0.001815 |
| away_adjusted_open_field_yards | +0.000020 | 0.000161 | -0.000209 to +0.000300 | +0.000034 | 0.000377 | -0.000441 to +0.000689 |
| home_adjusted_explosiveness | +0.000005 | 0.000387 | -0.000688 to +0.000591 | +0.000192 | 0.000847 | -0.001212 to +0.001305 |
| away_adjusted_second_level_yards | -0.000053 | 0.000292 | -0.000469 to +0.000457 | -0.000179 | 0.000659 | -0.001145 to +0.001058 |
| away_adjusted_passing_epa | -0.000079 | 0.000231 | -0.000439 to +0.000324 | -0.000039 | 0.000537 | -0.000791 to +0.000785 |
| away_adjusted_standard_down_success | -0.000096 | 0.000464 | -0.001005 to +0.000514 | +0.000005 | 0.001056 | -0.001746 to +0.001473 |
| away_adjusted_pass_explosiveness | -0.000100 | 0.000276 | -0.000594 to +0.000203 | -0.000196 | 0.000603 | -0.001267 to +0.000376 |
| away_adjusted_line_yards_allowed | -0.000114 | 0.000267 | -0.000428 to +0.000323 | -0.000144 | 0.000589 | -0.000927 to +0.000709 |
| away_conference | -0.000130 | 0.000473 | -0.000806 to +0.000671 | -0.000192 | 0.001056 | -0.001593 to +0.001569 |
| home_talent | -0.000142 | 0.000737 | -0.001285 to +0.000877 | +0.000052 | 0.001669 | -0.002549 to +0.002176 |
| away_adjusted_open_field_yards_allowed | -0.000146 | 0.000389 | -0.000712 to +0.000439 | -0.000358 | 0.000877 | -0.001575 to +0.000900 |
| home_conference | -0.000150 | 0.000426 | -0.000819 to +0.000538 | -0.000184 | 0.000960 | -0.001573 to +0.001442 |
| home_adjusted_standard_down_success | -0.000189 | 0.000303 | -0.000688 to +0.000210 | -0.000333 | 0.000762 | -0.001639 to +0.000813 |
| week | -0.000201 | 0.000410 | -0.000786 to +0.000317 | -0.000122 | 0.000984 | -0.001395 to +0.001081 |
| home_adjusted_pass_explosiveness | -0.000216 | 0.000129 | -0.000407 to -0.000055 | -0.000538 | 0.000300 | -0.000985 to -0.000118 |
| home_adjusted_rush_explosiveness | -0.000226 | 0.000163 | -0.000482 to +0.000092 | -0.000503 | 0.000373 | -0.001068 to +0.000233 |
| home_adjusted_open_field_yards | -0.000317 | 0.000357 | -0.000849 to +0.000122 | -0.000476 | 0.000749 | -0.001625 to +0.000427 |
| away_adjusted_passing_epa_allowed | -0.000365 | 0.000793 | -0.001413 to +0.001028 | -0.000723 | 0.001792 | -0.002809 to +0.002389 |
| home_adjusted_second_level_yards_allowed | -0.000389 | 0.000300 | -0.000810 to +0.000010 | -0.000866 | 0.000732 | -0.001967 to -0.000011 |
{
"fit_origin": "Saved production-builder fit; original deployed object identity is not established",
"bundle_manifest_sha256": "dbf988e9a5a23e56d7cca31ca30604031cf4fde947c4ba965e0f88736204a71e",
"fitted_bundle_sha256": "3bc7147298b52c56ecbbee78425bc0269c88d5159e59d4a73e4ec83a5b01638c",
"training_sha256": "0837fbbedb3a684b201340cc993067c34179ef0f7ea0e6cc49da8424378a4dd9",
"evaluation_sha256": "294b6aaa537a38b5cbbbea74df65d5c69087f3690fd3dd942fd01763d15b9696",
"source_files": {
"forecasts.json": "d602e4eeb530c9e7942579d13366993882e59b8ddf0714d9298be0cea997e17f",
"scripts__ensemble_receipt.py": "2099e9a233ee5220e0feff597b04e683363e58a28476774f51cae75fa6456f1d",
"training.csv": "0837fbbedb3a684b201340cc993067c34179ef0f7ea0e6cc49da8424378a4dd9",
"src__cfb_model_lab__export.py": "c28a54e6339df95b6649221235208993680764f57d1c58b867889f259c568603",
"inference.pkl": "3bc7147298b52c56ecbbee78425bc0269c88d5159e59d4a73e4ec83a5b01638c",
"uv.lock": "00537b95dd0b433432c30352ee6087b533718dc4173db0a4893168cb989b4bfe",
"scripts__build_ensemble_predictions.py": "0e254240613bb03277f2937469e1c3d55578b94d23d3b22231d7cabc450a0227",
"probabilities.npy": "d074afd76cc742c9721b5bf91f618606cfadb5a79d9260cd79cebd0f13d44444",
"src__cfb_model_lab__model.py": "15af71d671322e0b9e3c565d5d606b64c4986463a93ed64271d2e977c2f3e44d",
"src__cfb_model_lab__features.py": "a2b208fe95d3e4cf08d1bc6283d353956ffa1fdbf98d54f3d3ff803215252f7c"
},
"generating_revision": "e8e319a812dc8d64667f52591faff0179afcf37a",
"generating_checkout_dirty": false,
"report_source_sha256": "e15cf4af7c5d3683e975c1ad735dc97289e9142d4431bc373099c35bf2f6d25b",
"packages": {
"numpy": "2.3.5",
"pandas": "3.0.5",
"scikit-learn": "1.9.0",
"xgboost": "3.3.0",
"scipy": "1.18.0"
},
"python": "3.13.15",
"feature_order": [
"home_adjusted_epa",
"home_adjusted_epa_allowed",
"away_adjusted_epa",
"away_adjusted_epa_allowed",
"home_adjusted_rushing_epa",
"home_adjusted_rushing_epa_allowed",
"away_adjusted_rushing_epa",
"away_adjusted_rushing_epa_allowed",
"home_adjusted_passing_epa",
"home_adjusted_passing_epa_allowed",
"away_adjusted_passing_epa",
"away_adjusted_passing_epa_allowed",
"home_adjusted_success",
"home_adjusted_success_allowed",
"away_adjusted_success",
"away_adjusted_success_allowed",
"home_adjusted_standard_down_success",
"home_adjusted_standard_down_success_allowed",
"away_adjusted_standard_down_success",
"away_adjusted_standard_down_success_allowed",
"home_adjusted_passing_down_success",
"home_adjusted_passing_down_success_allowed",
"away_adjusted_passing_down_success",
"away_adjusted_passing_down_success_allowed",
"home_adjusted_explosiveness",
"home_adjusted_explosiveness_allowed",
"away_adjusted_explosiveness",
"away_adjusted_explosiveness_allowed",
"home_adjusted_rush_explosiveness",
"home_adjusted_rush_explosiveness_allowed",
"away_adjusted_rush_explosiveness",
"away_adjusted_rush_explosiveness_allowed",
"home_adjusted_pass_explosiveness",
"home_adjusted_pass_explosiveness_allowed",
"away_adjusted_pass_explosiveness",
"away_adjusted_pass_explosiveness_allowed",
"home_adjusted_line_yards",
"home_adjusted_line_yards_allowed",
"away_adjusted_line_yards",
"away_adjusted_line_yards_allowed",
"home_adjusted_second_level_yards",
"home_adjusted_second_level_yards_allowed",
"away_adjusted_second_level_yards",
"away_adjusted_second_level_yards_allowed",
"home_adjusted_open_field_yards",
"home_adjusted_open_field_yards_allowed",
"away_adjusted_open_field_yards",
"away_adjusted_open_field_yards_allowed",
"home_talent",
"away_talent",
"home_elo",
"away_elo",
"week_5",
"week_6",
"week_7",
"week_8",
"week_9",
"week_10",
"week_11",
"week_12",
"week_13",
"week_14",
"week_15",
"week_16",
"home_conference_American Athletic",
"home_conference_Big 12",
"home_conference_Big Ten",
"home_conference_Conference USA",
"home_conference_FBS Independents",
"home_conference_Mid-American",
"home_conference_Mountain West",
"home_conference_Pac-12",
"home_conference_SEC",
"home_conference_Sun Belt",
"away_conference_American Athletic",
"away_conference_Big 12",
"away_conference_Big Ten",
"away_conference_Conference USA",
"away_conference_FBS Independents",
"away_conference_Mid-American",
"away_conference_Mountain West",
"away_conference_Pac-12",
"away_conference_SEC",
"away_conference_Sun Belt"
],
"categories": {
"week": [
1,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16
],
"home_conference": [
"ACC",
"American Athletic",
"Big 12",
"Big Ten",
"Conference USA",
"FBS Independents",
"Mid-American",
"Mountain West",
"Pac-12",
"SEC",
"Sun Belt"
],
"away_conference": [
"ACC",
"American Athletic",
"Big 12",
"Big Ten",
"Conference USA",
"FBS Independents",
"Mid-American",
"Mountain West",
"Pac-12",
"SEC",
"Sun Belt"
]
},
"saved_forecast_replay": {
"status": "verified_saved_inference",
"games": 56,
"fit_calls": 0,
"fitted_bundle_sha256": "3bc7147298b52c56ecbbee78425bc0269c88d5159e59d4a73e4ec83a5b01638c"
},
"published_forecast_comparison": {
"payload_sha256": "f9656d509904085305f73134b6f2d01f69b34dae610d92c1c0c41e319973fc87",
"games": 56,
"all_rounded_forecasts_equal": true,
"original_deployed_object_identity_established": false
}
}
Raw training/evaluation rows, saved inference inputs and fitted objects stay private. Report generation does not write forecasts, change model features, select a betting strategy or constitute a model promotion.