RPI- Paper on Its Use for DI Women's Soccer

I just have finished a long and technical paper on the use of the RPI for Division I Women's Soccer. It covers how the Championship Committee uses the RPI, how the NCAA computes it, and what the NCAA's rationale is for using the RPI for DI women's soccer. It then demonstrates that there are two very significant problems with the RPI:

(1) It has a "standard error" that makes it quite inaccurate: for example, it can't tell the difference between the teams it ranks as #16 and #41 (which is important for seeding purposes) and it can't tell the difference between the teams it ranks as #47 and #90 (which is important in selecting teams for at large bracket positions).

(2) Of particular interest to us, it cannot properly compare teams from different regions if the regions are of different strengths. If the regions are of significantly different strengths, it significantly under-rates many teams from the stronger regions, significantly over-rates many teams from the weaker regions, and also over-rates some teams from the other regions. And, in fact, the regions are of different strengths. Specifically, for 2007, the West Region was much stronger than the other regions, as demonstrated by the RPI itself and confirmed by other statistical rating systems. The Championship Committee, however, either is not aware that, in this situation, the RPI cannot accurately rate teams from different regions in relation to each other; or, the Committee does not have the commitment to fairness and the will to deal with it. I hope it's the former.

If you are wondering why Oregon did not get an at large selection in 2006, there is no doubt in my mind that this is why. I think if you read the paper, you'll agree. I believe the same is true at least for Washington State in 2007 -- and that this therefore is the real reason why we did not get Round 1/2 games here -- and possibly also for other West Region teams.

After demonstrating that the RPI has these problems, the paper suggests limitations the Committee must hold itself to if it is going to continue using the RPI (which it almost certainly is going to do) and rules it should follow to protect strong-region teams against the RPI's defects at least to some extent. In an appendix, the paper also discusses some other less serious problems with the RPI and suggests some changes to address those problems.

The paper is long (52 pages) and dry unless you are into this kind of thing, with lots of scenarios and tables. It also has an additional appendix that is 5 pages (FAQs on the DI Women's Soccer RPI, issued by the NCAA staff). I don't think I can post the paper here as an attachment -- administrators, if I can, let me know how and I'll do it. But, I would be happy to email it as an attachment to anyone who wants it. If you do, you'll need to send me your email address. You can do it through this site's email function. I really would like to get feedback on the paper, including criticisms, suggestions for changes, etc. My plan is to distribute it to West Region coaches, starting with those whose teams have gotten burned by the RPI; to selected media starting with our friend at ESPN.com; and to the DI Women's Soccer Committee and the NCAA Championships/Competition Committee that is a level up from the Soccer Committee. If you get a copy from me, you also are free to use it as you see fit. If you think the paper seems credible and have the time and desire, any help in getting it out would be great. I have no idea whether we can shake things up at the NCAA enough to get some fairer treatment, but I think it's worth trying for it.

Hip, Hip Hooray UPSoccerFanatic. It's wonderful to have an attorney with guts and the knowledge to be able to throw a big stick at these buggers. Great post! Makes perfect sense to me.

I would love to have a copy of your paper. I'll email you through the website.