Tournament Selection and Seeding and the RPI

This is my first try at a blog, so here goes.

If you're interested in how the NCAA Tournament selection and seeding process works, here's how they do it.

There are 30 conferences whose conference champions automatically qualify for the Tournament. That leaves 34 at large selections to fill out the 64 team draw. Here are the selection criteria:

Won-lost record

Strength of schedule; and

Eligibility and availability of student-athletes for NCAA championships (whatever that means, it probably is not relevant to us).

In addition or as a supplement, for women's soccer, the criteria include the following, not necessarily in priority order:

Primary criteria:

Adjusted Rating Percentage Index, which includes:

1. Won-lost record (25%) (A tie counts as half a win and half a loss. There is no recognition for overtime games.)

2. Strength of schedule (50%) (This is the average of a team's opponents' won-lost records against other opponents. In other words, it excludes the result against the team itself.)

3. Opponent's strength of schedule (25%) (This is the average of a team's opponents' opponents' won-lost records computed similarly to element 2.)

4. Bonus/penalty system (This system awards bonuses for wins away, neutral, home against teams in the RPI top 40; bonuses for wins away, neutral, home against teams 41-80; penalties for losses away, neutral, home to teams 135-205; and penalties for losses away, neutral, home to teams 205-314. The penalties are on a sliding scale. The NCAA does not reveal the amounts of the bonuses and penalties or the formula for computing them.)

Head-to-head competition

Results versus common opponents

Secondary criteria:

These are used only if use of the primary criteria does not result in a decision. If these criteria are used, they all are evaluated.

Results versus teams already selected to participate in the field (except for results against automatic qualifiers with an RPI of 76 or higher).

Late season performance - defined as the last 8 games including conference tournaments (strength and results). [Note: This was increased this year from a lesser number of games, probably to take account of players returning from World Cup competition.]

The selection committee considers recommendations provided by regional advisory committees. The selection committee explicitly does not consider coaches' polls and/or any other outside polls or rankings.

What this means is that if you are interested in Tournament selection and seeding, the polls are irrelevant. The RPI is what's relevant. You can find it through the following link: http://www.ncaasports.com/soccer/womens/ On the webpage this takes you to, in the upper right hand corner, in the "Rankings" box, click on RPI. This will give you the NCAA's latest version of the RPI covering games through October 21. The next update of the RPI will cover games through next weekend and should be on the NCAA website by Tuesday, November 5. This is the last version of the RPI the NCAA will publish until after the Tournament, although it will not be based on the next weekend's games and is not the version the Selection Committee will be using since they will use a version based on all games through the end of the regular season.

Finally, when using the RPI, the selection committee can consider compared data of individual teams, including but not limited to overall record, Division 1 record, overall RPI rank, non-conference record and RPI rank [I'm not sure what that means -- perhaps RPI rank with conference games excluded], conference regular season record, and conference tournament results.

Please note that contrary to a suggestion in one posting, there is no pre-seeding in the RPI. It starts from scratch each year. This is different than Albyn Jones' SoccerRatings, which starts with a seeding based on past history. Over the course of a season, this seeding filters out although it is not completely gone even by the end of a season. According to Jones, this produces a statistically more reliable predictor of game outcomes and I assume he is correct. Nevertheless, the NCAA explicitly does not consider past history. This does not mean, however, that there is not a bias built into the RPI. There is, because it starts with the assumption that the teams in each region of the country, on average, are equal. In other words, their is a built-in weighting towards distributing the teams to be selected and the seeding regionally on a pro rata basis in relation to the number of teams in each region. The second and third factors of the RPI are an attempt to offset this weighting if in fact the underlying assumption of regional equality is incorrect. There is a very significant question, however, whether there are enough inter-regional soccer games to fully make a correction if in fact there is not regional equality. (I may say more about this some other time.)

With that in mind, I have been running a parallel version of the RPI so I can understand how it works and what its strengths and weaknesses are. The one difficulty I have is figuring out exactly what the bonuses and penalties are since there are 12 different bonus amounts and 12 penalty amounts and since the NCAA carefully maintains its secrecy about what those amounts are. (It also is difficult because the NCAA's data are not always absolutely complete and correct, so even if I had the right amounts my results might not match theirs.) Nevertheless, I am close enough usually to be within a place or two of where a team fits within the NCAA's RPI ranks.

I've run the RPI numbers with games through this past weekend (i.e., through October 28, including the Pilots' wins over San Diego and Santa Clara). My numbers indicate that if tournament selection and seeding were based only on the RPI, the top 47 RPI teams would be in the tournament plus 17 other automatic qualifiers (conference champions) that are outside the top 47. The top 47, which include 13 automatic qualifiers as conference champions plus all the 34 at large positions, would be -- remember, however, that there are two weekends of games left and there still will be a bunch of shifting around:

1. Penn State
2. UCLA
3. North Carolina
4. USC
5. Tennessee
6. Stanford
7. Texas A&M
8. Portland
9. Wake Forest
10. Virginia
11. Georgia
12. San Diego
13. James Madison
14. West Virginia
15. Notre Dame
16. Boston College
17. Indiana
18. Purdue
19. South Carolina
20. Santa Clara
21. Florida
22. Missouri
23. Louisville
24. Clemson
25. Texas
26. William and Mary
27. LSU
28. UNC Greensboro
29. Florida State
30. Central Florida
31. Hofstra
32. Connecticut
33. Ohio State
34. Charlotte
35. Duke
36. Illinois
37. California
38. Memphis
39. Miami FL
40. Dayton
41. Arizona State
42. Colorado
43. BYU
44. Long Beach State
45. Oklahoma State
46. Utah
47. Georgetown

Next in line at #48 is Washington State, which has a tough schedule ahead of it. The ACC, SEC, Big 10, Big 12, and Big East, however, have their conference tournaments, so they'll all be knocking each other off too. Oregon is at #58; Loyola Marymount at #61; and Pepperdine at #67.

Bottom line is, the Pilots can just focus on winning against Pepperdine and Loyola Marymount. The others will do what they'll do and hopefully beat each other up. If the Pilots win out, we look pretty good though not absolutely assured for home games through the round of 16 (unless we run into an NCAA travel rule problem that forces the first and second rounds elsewhere). Most likely, the quarterfinals will be elsewhere unless one of the top four teams loses in the Tournament before the quarters -- which is what happened to two of the top four last year.

WOW!!! Best first post ever!!! Nice work, UPSoccerFanatic.

Thanks for the post... quite an entry for your first! A lot of research...

The moral of the story here is that we need to win our final 2 games. Period.

Secondarily, hope that the Coogs can win a few of their remaining tough games to boost their resume to help secure some home playoff games for us.

Nice explanation. We've talked about this before, UPSF (I sit behind you at the games but will honor your nom de post!).

There is no perfect system for these rankings etc. but one flaw with the current systemis that it makes little to no disctinction between UP going to USC and losing in OT and playing the same game at home and losing by 4. We all know there's a big difference.

I don't want a system that rewards teams for running upthe score but maybe they could build in a factor that took into account margin of victory and capped it at 3 or four goals.

Welcome and great post, UPSoccerFanatic. It's nice to have your knowledge and energy added to the board, as well as a 2nd Fanatic!

Two BIG matches left.

Go Pilots!!!

One possible way to recognize an overtime game would be for the RPI to treat it as 3/4 a win and 1/4 a loss for the winning team and 1/4 a win and 3/4 a loss for the losing team. This would be similar to the RPI's treating a tie as 1/2 a win and 1/2 a loss for both teams. The computer could be programmed to do this pretty easily. I suspect the issue for the NCAA is that collecting data about ties is one more set of data where there can be errors and therefore creates one more place where poor data can produce poor results. (One think I know is that NCAA's data are not always complete and also are not always correct.)

WOW!

If you have cracked the NCAA adjusted RPI, I'd love to see it. nobody could ever crack the basketball one when it was done this way. I think you'd be the first.

I'm a bit puzzled how the RPI can statistically link 314 teams with 20 games in a meaningful fashion without preseeding. I'd love to see how you are weighting the rankings in the adjusted part.I've seen speculation that the adjustments are based on a version of the coaches poll.

Do you have your RPI rankings compared to the two the NCAA has posted this year? it would be an interesting comparison.

It sure would be neat if you got your next one out before they did, and if you posted a season-ending RPI.

Don't forget that the NCAA counts shootout wins as ties, except in the Championship game. Playoffs are coming up. As such, are you counting ties as 1/2 win, or 1/3 win. or just not counting them in the RPI? many leagues , including the WCC, count wins 3 points and ties 1 point.

I think the eligibility/availability clause refers to teams on probation or suspension, schools that don't allow post-season play (the service academies didn't in the past-some other schools, too). Provisional D1 Schools (First 4 years of D1), and schools found to have ineligible players. You're right, it doesn't apply to UP.

You don't mention how the Provisional D1 schools are treated when it comes to RPI and win/loss record. How are you treating that?

The RPI has interesting quirks. Have you noticed that 0-14 Lamar has a better RPI that 18 schools- some of whom have wins?

Is that part of the adjusted RPI?


Lastly, where did you find the adjustment criteria?

Great job!

Lots of good questions from Purplegeezer. Here are my answers:

1. I haven't cracked the RPI adjustments. I'm close on the bonuses, which are what really matters for NCAA Tournament selection and seeding purposes, but haven't yet spent a lot of time on the penalties. Even with the bonuses, however, I'm sure I'm not all the way there. One of the biggest problems is trying to come up with bonuses and penalties that give me a complete match with the NCAA's RPI when our data are not exactly identical. I can identify when the NCAA and I have different won-loss-tie records for teams and when the NCAA and I have different home-neutral-away records. But, if either they or I have made erroneous entries of team names, there can be problems. I know that the NCAA's data have had some mistakes, of which I've advised them so they can make corrections, but it is possible I may have some that I can't correct through references to the NCAA's RPI reports. Once the season is over and I have more time, I'll go through a further "vetting" process for my data and see if I can get the adjustments just right. By the way, the NCAA's data from which they develop the RPI (game outcomes, etc.) are not publicly available.)

2. The adjustments are not based on polls. The bonuses are specific amounts that are added to the unadjusted RPI for certain wins based on the unadjusted RPI rank of the opponent and whether the game was home, away, or neutral. The penalties follow a similar format but are deductions for certain losses. If you go to the NCAA website and find the Division 1 Women's Soccer Championship Manual, there is a description of the general selection and seeding criteria near the front of the Manual and additional information in Appendix I. The information in Appendix I includes how the adjustment process works, but it does not provide the amounts of the bonuses and penalties. That is information that the NCAA intentionally keeps confidential. I have been told that the reason for the confidentiality is to avoid people who don't know what they're doing publishing their own incorrect RPIs and thereby creating confusion and also to avoid outsiders trying to sell their own RPI reports to the NCAA schools. (Of course, an alternative approach to those problems would be to publish the complete RPI formula and publish the NCAA's own RPI report weekly.)

3. I have done rankings each Monday for a number of weeks. I'll email to you my weekly RPI reports.

4. I am planning on doing an RPI report next Monday, which should precede the NCAA's by one day. I'm also planning on doing a report the following Monday, which I believe is the day the NCAA will be announcing the Tournament bracket.

5. I know that the NCAA treats end of season conference tournament shootout games as ties and I will treat them that way too. In computing the RPI, the NCAA treats a tie as half a win and half a loss for each team. I know that from the NCAA's chief statistician.

6. The NCAA includes in its computations the games of teams playing provisionally at the Division 1 level. This also is from the NCAA's chief statistician. My computations include the same schools as the ones the NCAA uses (all 314 of them).

7. I've followed the Lamar situation because they played both Oregon State and Loyola Marymount in an early season tournament at Loyola Marymount. It was really dumb of LM to include them in the tournament. If either Oregon State or LM was close to being included in the NCAA Tournament, the Lamar game could have been a killer for them, because if you play and beat a team with a really low RPI, notwithstanding the win you can end up with a lower RPI than if you hadn't played them at all. I was wondering if those games would hurt the Pilots' RPI, since we played Oregon State and will play Loyola Marymount. So, I deleted those games from my data base and re-ran the RPI numbers. As it turns out, the Oregon State game did not negatively affect the Pilots' RPI, and I assume this means the Loyola Marymount game won't either. The Pilots' RPI still benefited slightly from the Oregon State win, which means that Oregon State's better record helped the Pilots in the second element of their RPI and Lamar's very poor record didn't hurt the Pilots that much in the third element of their RPI. The same should be true once we've played Loyola Marymount. However, the same was not true for Oregon State and Loyola Marymount's RPIs. If Loyola Marymount had not played Lamar, its RPI ranking would be #54; with the Lamar game, it is #61. Similar, without the Lamar game Oregon State is at #152; but with the game it is at #160. This is because Lamar's being one of the teams included in the computation of the second element of their RPIs hurt them more than the inclusion of the win in the computation of the first element of their RPIs helped them.

Regarding how Lamar can have lost all its games (by a combined score of 91-2) and still be ranked above other teams, that is due strictly to the second and third elements of the RPI. These are the strength of schedule elements. Lamar has played a tougher schedule than teams below it. This reveals an interesting feature of the RPI. If a team loses all its games, the RPI tells you nothing about how it relates to other teams. All it tells you is how its strength of schedule relates to the strengths of schedule of other teams. The same is true if a team wins all its games: the RPI tells you nothing about how that team relates to other teams, but only tells you how its strength of schedule relates to the strengths of schedule of other teams.

Could this imply that the Pilot's RPI would have been better had we not played Portland State?

Here are the results for the Pilots with and without the Portland State game, Portland State being ranked at #187 according to my calculations:

Without Portland State game: Adjusted RPI = .661039
With Portland State game: Adjusted RPI = .659188

Difference: .001851

In other words, the Pilots' adjusted RPI would have been .001851 higher if the Pilots had not played Portland State. This difference does not affect the Pilots' #8 position in the rankings, although if the nearby teams' RPIs were closer, it could. The amount of .001851, in any event, is significant, although not affecting the Pilots so far this year.

Does this mean the Pilots should not be playing Portland State? I think that the Clive Charles ethic would say that it's important to promote soccer development in the Pacific Northwest, so we should continue to play Portland State, Washington, Oregon, and Oregon State, year in and year out. Love of the game, and promoting it in our area, is really important. Plus, we need teams from our area to become better and do better in Division 1 so that they can get into the Tournament bracket and we can have Tournament games here in Portland. In any event, whether the Pilots should or should not play Portland State every year, given the costs and the benefits, is a good discussion topic.

As a side matter, the NCAA has a rule that if teams are within a certain distance of a Tournament game site, then they must go by bus rather than by plane. In previous years, the distance was 300 miles. This year, for all sports but basketball, the distance was increased to 350 miles. To determine distance, the NCAA uses mappoint.msn.com. According to that web-site, depending on exactly where you are going between U of P and Washington State, the distance is +/- 350 miles. So, if WSU were to get into the Tournament, if they are considered within 350 miles of U of P, it might be really helpful to the pitch to have early rounds here. So, let's all send good vibes to WSU and bad vibes to the teams at the low end of the current likely qualifiers. Maybe we'll manage to get the early rounds here.

I asked my wife about whether the Pilots should play Portland State and she surprised herself by immediately saying, "No." Her thinking is that at Portland State's current level of play, their playing the Pilots does neither team any good. If they had kept Tara Erickson and continued to develop into a really good team, then it would have been a good thing, but at this point they should not play each other. Perhaps, as an alternative, bring in a good team like BYU or Utah.

Interesting. If the roles were reversed, what would we say? Or, say, what about men's basketball playing Oregon? How would we feel if UO stopping playing us?

I think there needs to be more to the overall picture than simply looking at RPI numbers. There are regional rivalries and interest games that should be played. If we get too focused on RPI and ignore all other considerations, we are losing a very important part of sports.

If we were to stop playing PSU due to RPI concerns, we are no better than the BCS basketball schools who have stopped playing road games at mid major schools.

We should play PSU, OSU and UP every year!

(I will step off the soap box now).

UPSoccerFanatic,

If PSU doesn't make more of commitment to women's soccer, I'd say I have to agree with your wife.
It's a shame, really. I think the rivalry would be good for both schools, and It would be a good draw to get schools from outside the region to come here and play two good Programs. Right now, If we want teams to come in for double-headers, we have to team up with U of O or OSU.

There is an alternative, and that's Seattle U. They are making the transition to D1, are a potential conference rival down the road, and have a top flight program at their level (D2) right now. They will be transitional D1 school next year , which means they will not count as a D1 opponent for any schools except for 2 sports which they can designate as "Fast Track" D1 teams. I assume they will pick a men's and a women's program, and if they do, the soccer program for women is a logical choice. My guess is that if they entered D1 right now, they'd be a better team than the Vikings.

Who knows, it could turn into a real rivalry.

I'm ambivalent myself and at least slightly tend to agree with PilotNut. On the other hand, as a high school coach (not soccer), I've had teams that have been way beyond their opponents, and the matches were not good for either team. Yes, a coach can use a game as a chance to put other players in, but there are not that many games in a season, and the starting players really love to play.

Perhaps the answer is for U of P, Oregon, and OSU to put pressure on Portland State: If you will make a major commitment to women's soccer and help turn this State into a soccer Mecca for college players, then we'll keep you on our schedules while you have time to develop your program. But if you won't do that, then let us know and you'll have to find other non-conference teams to play. I think that would be only fair.

One other consideration we haven't discussed. Suppose the Pilots' playing Portland State ended up resulting in the Pilots dropping enough based on whatever the criteria are so that a Tournament game that otherwise might have been home ends up being away. Besides the effect on the team, what about the Pilots' fans?

As with many things, the answers about what to do are not that clear to me. There are lots of considerations and competing values.