Wednesday, May 6, 2009

Comparison of RRI, UPA Top 25 and Actual OpenFinals Participants


Nation,

After doing a comparison of women's UPA college finalists with the current RRI and UPA Top 25 rankings, 70% of the actual finalists were in the top 20 of RRI and UPA Top 25.

To my understanding (as per icultimate.com):

UPA TOP 25 RANKINGS- is an algorithm-based ranking system that ranks all teams with at least five games reported on the UPA Score Reporter Tool (SRT). The algorithm assigns a rating to each game based on the score, the strength of the opponent, and how long ago the game was and then averages each team's ratings.

RRI RANKINGS- The RRI rankings is another algorith-based ranking system that is based on the NCAA Hockey's KRACH ranking system. A Win Confidence Score is assigned to each game and represents how likely a team is to win a re-match of that game, based on the score. This allows the system to generate score predictions for teams in addition to the rankings.


Based on my last post, it seems clear:
  • These rankings are limited by geographic, season, and head to head limitations
  • These rankings don't affect seeding for UPA college finals, and people don't think they should due to their imperfect nature
  • These rankings are merely references for fun/trends during a given season

So, purely for fun. Here are the top 20 Open teams in terms of RRI, UPA Top 25 and Actual Finals Participants (Ranked by RRI, as seedings are not yet posted):

# RRI UPA Top 25 Actual Finals
1) Pittsburgh Carleton College Pittsburgh
2) Carleton College Colorado Carleton College
3) Cornell Oregon * Cornell
4) Colorado Florida Colorado
5) Florida Virginia Tufts
6) Oregon * Cornell Virginia
7) Tufts Wisconsin Wisconsin
8) Virginia Washington California
9) Middlebury Western Washington Stanford
10) Wisconsin Notre Dame Luther
11) California Pittsburgh Williams
12) Stanford California Michigan
13) Washington Middlebury North Carolina State
14) Notre Dame North Carolina State Minnesota
15) Harvard Michigan Kansas
16) Luther North Carolina-Wilmington Texas
17) Western Washington Kansas Georgia
18) Williams Stanford California-San Diego
19) Michigan California-San Diego California Santa Barbara
20) North Carolina State Texas Illinois

63% of RRI top 20 teams are in the actual finals 68% of the top 20 UPA Top 25 teams are in the actual finals The Fact that Oregan has been suspended from play has been taken into account.

1 comment:

jhaig said...

A somewhat interesting list. The biggest fault being that RRI and UPA Top 25 don't understand the qualification rules for UPA finals and are going to make mistakes. For example: having too many women's teams from the NW in their top 20. A far more interresting approach would be to test how well these systems predicted the outcomes of tournaments. Look at each regionals tournament and see from there how well either system predicted the nationals qualifiers.