What Exactly is the NCAA Tournament Committee Looking for?

With less than a week until Selection Sunday and Oklahoma State officially on the bubble, I think it’s time to take a look at what the NCAA selection committee even wants to see from at-large teams.

I suspect a lot of this is like the CFP — the committee will retroactively use whatever data it wants to justify its selections (because you can find data going in all manner of directions), but let’s try and whittle it down to a few key metrics.

Here is what the NCAA website says about its criteria.

The committee selects the 36 best teams not otherwise automatic qualifiers for their conference to fill the at-large berths. There is no limit on the number of at-large teams the committee may select from one conference. [NCAA]

OK, go on.

The Rating Percentage Index (RPI) is one of many factors used by NCAA sports committees when evaluating team selection, seeding and bracketing.

The basic RPI consists of a team’s Division I winning percentage (25 percent weight), its opponents’ winning percentage (50 percent weight) and its opponents opponents’ winning percentage (25 percent weight). The RPI is one of many factors the committees use for selecting and seeding teams. [NCAA]

Well you’re listing it first and it sure sounds like the most important thing! This is bad for Oklahoma State as its current RPI of 88 would be the worst ever admitted as an at-large bid into the Big Dance. More from the NCAA …

Other criteria the committee considers in the selections process are:

• An extensive season-long evaluation of teams through watching games, conference monitoring calls and regional advisory committee rankings;
• Complete box scores and results;
• Head-to-head results and results versus common opponents;
• Imbalanced conference schedules and results;
• Overall and non-conference strength of schedule;
• The quality of wins and losses;
• Road record;
• Player and coach availability; and
• Various computer metrics.

Each of the 10 committee members uses these various resources to form his or her own opinion, resulting in the committee’s consensus position on selection and seeding. [NCAA]

Yeah it sounds like you’re just looking at RPI. Why is that a bad thing? Well 538 took a close look at it in 2017, and this was the part that stuck out.

Until the NCAA adopts a new metric, the committee is stuck with the RPI. Developed in 1980 by statistician Jim Van Valkenburg, the RPI was originally intended to adjust a team’s record for its strength of schedule, a noble cause in a sport where 351 teams face opponents of wildly dissimilar quality. Critics of the RPI — and there are many — are less concerned about its goal and than its execution. It’s an arbitrary formula that mashes together a team’s winning percentage with those of its opponents (and opponents’ opponents), and as a result, it amplifies the importance of a strong schedule at the expense of everything else. [538]

They also interviewed KenPom (whose altar I worship at), and he voiced my concerns in a much more distinguished and nuanced way (also, SVP went on a rant about RPI six years ago!).

“In discussions with other committee members, they always stress to me, ‘Hey, we’re not just relying on the RPI, we’re allowed to use whatever we want,’” Pomeroy told 538. “But obviously it’s much more convenient to use the RPI, because the RPI is on their computer screen, in front of their face.

“In terms of the how the committee should select teams, it actually says in [the NCAA guidelines] that they need to select the best 36 at-large teams — based on results,” Pomeroy said. “Best” and “based on results” don’t always line up, though. “It’s like a contradiction right in that sentence.”

That last part is the college football argument we always have. Of course Alabama is better than UCF … but that’s not what the records say! It’s an endless circle.

The thing that really struck me is how deep RPI runs. As Jay Bilas recently pointed out, even the Quadrant 1-4 silliness is fundamentally based on RPI. There’s no getting away from it!

The Committee’s quadrant system is just more shaky data to organize information for less discerning minds. All of the quad data is based upon RPI. Garbage in, garbage out.

— Jay Bilas (@JayBilas) February 11, 2018

For example, a road win over Rider is valued the same as a home win over Villanova. Both are Quadrant 1 wins. That makes no sense. Based upon RPI.

— Jay Bilas (@JayBilas) February 11, 2018

John Gasaway takes that a bit further.

The RPI’s primary operational weakness is revealed by the fact that the single most statistically revelatory moment in the entire season is the day when a team releases its schedule. [JohnGasaway.com]

The best thing teams like Oklahoma, Texas and Baylor did to beef up their NCAA Tournament resumes … follow me closely here … schedule teams in the nonconference that they lost to. That’s it. They didn’t have to beat anybody different than OSU (the Cowboys have more and better top-level wins than either OU or Texas). They just had to schedule losses against top teams or schedule more teams between RPI 150-200 (powerhouses like Colgate and Grand Canyon instead of patsies like Charlotte and Pepperdine). Think about this logically for a second. It is insane.

This begs the question of what you should use if not RPI. I don’t know the answer to that, although other folks have suggestions. What I do know is that OSU’s RPI is incredibly low, and it would appear that it is being sort of exponentially deflated because it played too many cupcake games. Even though its top nonconference wins and losses compare favorably with other teams that are included in the NCAA Tournament!

So who knows how all of this is going to play out, but regardless of it all, consider me an advocated for the eradication of the RPI. If OSU doesn’t make it because they didn’t schedule enough losses to the Dukes and Kentuckys of the world, then that’s a crazy system and should go the way of the BCS. Oh hey, another system OSU inadvertently ended. Maybe this one will go the way of that one in the near future.