Rankings 101: NET, Ken Pom, and the AP Poll

Selection Sunday is exactly two months away, March 17th. If you’re like me you’ve already started diving into all the relevant data surrounding the teams that will likely be heading to the dance. But even if you’re just the casual fan, you’re going to start hearing all about how these teams are ranked, and what that means for their seeding.

It used to be that a team’s “ranking” referred to one thing: The AP Poll. Since 1949 members of the media have been voting on college basketball, and today it’s the most visible “Top 25” in the sport. However, as advanced analytics have become more popular, other ranking systems have been on the rise. By now you’ve likely heard about Ken Pom or the NET, especially when people are talking about how a team will be seeded in the NCAA Tournament.

While many will debate which of these ranking systems are more accurate or useful, the funny thing is they aren’t really comparable. These systems all take very different approaches to ranking college teams, and they’re not simply ranking “who’s the best?”. Some use quantitative analysis while others rely on visual intuition. Some are based on a team’s proven record, while others are concerned with what’s most likely to happen next. Simply put, it’s not simple at all.

Today I’ll take a stab at breaking these rankings down and explaining what they mean and how they work. It’s easier said than done when you consider that the three most prominent polls can all have the Kentucky Wildcats ranked 12th, and yet the Florida State Seminoles are ranked 11th, 24th, and outside the Top 30 (as of yesterday at least). To figure this out we’re going to first identify the three types of ranking systems, and then answer three questions for each poll: What data is it ranking, what does that ranking mean, and how does it effect the NCAA Tournament?

The Three Types Of Ranking Systems

To start, not all rankings are trying to accomplish the same thing. Sure, generally they’re going to list teams from best to worst, but best at what? Before we even get into what’s in the ranking, let’s start with what they’re trying to rank. For the most part it’s one of three things: How good has a team been, how good will a team be, or how good is a team right now? To put labels on those categories, we’re talking about cumulative rankings, predictive rankings, and power rankings. And to oversimplify it: what we know, what we think, and what we feel.

Cumulative rankings, also known as result based rankings, are a measure of how a team has performed so far. They usually take in data (wins, losses, scoring margin, etc.) from the games played in the current season and use that to rank teams solely based on their performance this season. Some of these can be weighted to give more importance to strength of schedule or location of the games, but they’re still based on what has happened. These are the most merit based rankings and can take some time to start looking accurate based on their small sample sizes early on. A perfect example of that is the initial release of the NET earlier this season, which came off as ridiculous initially before balancing out over time.

Predictive rankings are based on how a team will perform in their next game. While they do rely on much of the same data as cumulative rankings, they’ll often reach beyond the current season into previous years. They’ll also often use more complicated algorithms that take into consideration offensive and defensive strategies favored by teams as well as individual player analysis. The goal of predictive rankings isn’t to determine the best team, but rather who is more likely to win. While this doesn’t necessarily reward teams for what they’ve accomplished on the court, it makes for a for more useful tool in trying to predict a games outcome. In other words, it’s primarily for betting games and predicting upsets.

Power rankings, while the least reliant on data, often are most reflective of what the general public thinks of these teams. As we’ve seen many times before, that doesn’t mean they’re accurate. In fact, there’s no guarantee that data of any kind is even being used when these rankings are being created. It’s all about gut feelings or truthiness for some of these polls, and that’s why they tend to shift more frequently than the others late in the season. It’s also why everyone seems to be up in arms about them as they’re just opinions.

Now that we know what each of these polls are trying to accomplish, let’s go back and answer those three questions for each of them.

Cumulative Rankings: The NET

There are a number of cumulative rankings out there today, but only three of them are presented to the NCAA selection committee when they’re looking at each team’s resume. First is the KPI, or Kevin Pauga Index. It’s a ranking system developed by Kevin Pauga (duh), and has been one of the more tenured ranking systems used by the committee. Also included is the SOR, or Strength of Record. This ranking determines how difficult a given team’s record was to achieve, taking in factors like strength of schedule. But by far the most prominent of all the rankings presented to the selection committee is the NET.

The NET, or NCAA Evaluation Tool, is the newest and most influential of all the prominent ranking systems out there today. It was created after careful deliberation by the NCAA to replace the RPI as the main ranking system used for the Team Sheets presented to the committee during the seeding process. It’s used to populate the Quadrant System that further weights the ranking based on where each game was played, not just who was playing. But we’ll have more on the Team Sheets later. For now, let’s get into those three questions.

What data is it ranking?

The NET is calculated based on a number of factors including game results, team efficiency, winning percentage, scoring margin, and much more. While the “secret sauce” hasn’t been completely divulged by the NCAA, they have provided us with this (helpful?) graphic to explain what goes into the NET:

To simplify, it’s taking everything it can about the games that have taken place this season in order to best evaluate every team in the closest thing to “common ground” as it can. While it’s not a perfect system (nothing ever will be), it’s a big step in the right direction from its predecessor, the RPI.

What does the ranking mean?

Cumulative rankings try to represent a picture of how well teams have done so far in the current season. It’s easier said than done for a number of reasons, but the main one is that not every team plays each other. Not only would that be a logistical nightmare, it would greatly increase a team’s schedule from 30+ games to 350+ games. It’s a problem faced by all ranking systems, but it tends to hit the cumulative rankings the hardest.

As I mentioned earlier, the NET came under fire early on because its small sample size of games was producing rankings that just didn’t look or feel right. But the more games that get played and the more data that’s included in the formula, the closer the NET can come to synthesizing that “common ground” that would only happen if everyone played each other.

So when you look at the Top 25 of the NET, it’s representing the teams that the formula has determined have accomplished the most so far this season. Again it’s based on a number of metrics, many of which are weighted differently by other ranking systems, but it’s difficult to argue with the results.

As of Jan. 14th (the sample date I used for this evaluation) the Top 2 teams in the NET were the Virginia Cavaliers and the Michigan Wolverines, the only two undefeated teams in the country. The other two major rankings can’t say the same, but that’s also because only the NET is trying to rank teams by how they’ve performed so far this season.

How does the ranking effect the NCAA Tournament?

The NET was designed to be the primary tool for the selection committee to use in seeding the NCAA tournament. It, and the other cumulative rankings, are featured prominently throughout the Team Sheets used as a quick reference for every team:

Everything boxed in red, both the NET and Quad system data, all stems from the NET and other cumulative rankings. So basically the entire sheet. The NCAA didn’t sink a significant amount of time and effort researching this thing for it not to be used as the primary tool for the committee.

And arguably, that’s how it should be. Cumulative rankings are strictly merit based, and that’s how teams are supposed to be seeded. It’s not necesarily about who the best team is, but who’s the most deserving. However that doesn’t mean the other systems don’t have their place, and that brings us to the predictive rankings.

Predictive Rankings: Ken Pom

In the early 2000’s a perfect storm was brewing when it came to college basketball analysis. There was a growing frustration with the shortcomings of the RPI system which had been in place since 1982. Advanced analytics in sports were becoming more and more popular. And the internet was creating a demand for more intuitive ways to predict who would win a college basketball game (aka sports betting).

The following decade would produce the three major predictive ranking systems the selection committee uses today. The Sagarin Ratings, created by Jeff Sagarin, were one of the first of these predictive models. Later, ESPN produced their College Basketball Power Index, or BPI. But the one that seemed to take hold as the most popular and frequently most accurate of the three were the KenPom Rankings, produced by Ken Pomeroy. His will be the predictive ranking system we focus on for our three questions.

What data is it ranking?

Similar to the cumulative rankings, the predictive models use the data generated in the current season to create their rankings. However, there are two major differences. First, predictive analysis is more interested in how a team is trending than cumulative performance, so recency becomes a bigger part of the equation. If you’re looking for a deep dive into Ken Pom specifically, he suggests this article as a good place to start. If you’re looking for one sentence to summarize the whole thing, “It looks at who a team has beaten and how they have beaten them.”

The second big difference is that the predictive rankings also draw data from the previous season, at least initially. KenPom doesn’t become completely reliant on data from the current season until about three months in, or February-ish. While that does help alleviate the small sample size issues that face the cumulative rankings, it comes with it’s own setbacks.

College basketball experiences a lot of turnover, especially for some of the more elite teams. Just look at the 2018 Nation Champions, the Villanova Wildcats. They lost their four best players to the NBA draft, all of which would have been eligible to return for this season. This year’s Villanova team isn’t quite at that championship level. However, for much of the early season they were still benefiting from the success of last year’s team. They’re in the KenPom Top 25 on their own merits now, but it took awhile for everything to balance out.

What does the ranking mean?

Ken Pomeroy is pretty clear on what his system is and what it isn’t. It isn’t a ranking of how “good” a team is. It is a ranking of how strong a team’s performance would be if they played a game today without taking into consideration injury or emotion. Or to put that another way, it’s ranking how likely a team is to win a game.

But that doesn’t mean that the #1 ranked team would be automatically favored in the rest of their games for the season. Currently the Virginia Cavaliers, KenPom’s #1 ranked team, is actually the underdog for this weekend’s game against the #2 ranked Duke Blue Devils. That’s because there are other factors (location, offensive/defensive schemes, etc.) that are a part of the formula. When those same two teams play at Virginia in February, the Cavaliers are six point favorites.

Because Ken Pom is solely concerned with how well a team will play in its next game, it’s became a major aspect of the sport. Not only is the ranking frequently used when analyzing upcoming games, it’s also a fairly good tool for sports betting. And considering online and in person gambling is becoming largely available throughout the country, predictive rankings should continue to grow in popularity.

The one note I’ll make is that I’ve seen many instances in which predictive rankings are used incorrectly to evaluate a team’s current resume. Saying that a team has beaten X number of Ken Pom Top 25 teams, or lost to X number of teams outside the Ken Pom Top 100 aren’t really appropriate. That may be fine if the premise is that a team is only as good as what it will do from this point forward, but it’s using a ranking system that’s solely trying to predict the next game to give weight to past performances. While those are aspects of the predictive analysis they’re by no means the focus, and for that reason the predictive rankings play a far less prominent role in determining NCAA Tournament seeding.

How does the ranking effect the NCAA Tournament?

Predictive rankings like KenPom are probably more helpful in filling out brackets than they are in actually seeding them. That isn’t to say that don’t have a useful purpose. The three major predictive rankings, which I should note are no longer averaged together on the team sheets, help show how a team is trending heading into the tournament. It’s not the main factor in seeding, but it does help to paint the picture.

To that end, here’s how the predictive rankings are displayed on the team sheets:

That’s it, just one small strip at the top of the sheet. And that’s really how it should be. Predictive rankings shouldn’t be ignored entirely, but a team’s NCAA Tournament seed should have much more to do with what they’ve accomplished than what they may do next. That doesn’t mean committee members won’t choose to give those rankings more weight or use them as determining factors, but at least they know what the predictive numbers represent. The power rankings on the other hand, are another thing entirely.

Power Rankings: The AP Poll

Despite all of the numbers, facts, and figures that I’ve shoved into this article, it’s still an opinion piece. I’m giving you my opinion on how these ranking systems should or shouldn’t be used to determine seeding in the NCAA Tournament. And despite sometimes having records and other data to back them up, power rankings are also just opinion pieces.

The most popular power rankings come in the form of two polls that each use the same formula. A group of pollsters is asked to rank the Top 25 teams in college basketball each week, and each of those rankings is given a certain number of points. When all the ballots are collected, the points are totaled. The team with the most points is ranked number one, and so on down the line. One of these is the Coaches Poll, comprised of a group of active college coaches that rotates each season. But the most well known of all the ranking systems, and one of the oldest, is the AP Poll.

The Associated Press Poll is comprised of 65 members of the media who cover college basketball. It’s grown significantly in both members and number of teams ranked since its start in 1949. There’s no questioning that it’s the most popular ranking used today. When you’re looking up rankings or if you’re watching on TV, the AP Poll rank is what’s posted next to any team’s name. But when it comes to the validity of that ranking, it’s highly debatable.

What data is it ranking?

There are no guidelines, or restrictions for that matter, beyond ‘rank the Top 25 teams’ for these pollsters. That means it’s highly likely that several of these media members are using completely different metrics to determine which team is “best”. It’s a very subjective system, and we’ve had some of the participants admit to some completely arbitrary methods of ranking teams.

But that’s the whole point of this type of ranking system. It’s trying to combine a number of different opinions and methods to come up with which teams are the best in the country. When you’re deliberately taking a new poll each week without any consistent sample sizes or metrics, then there really is no relevant data that should apply to everything. Sure, most people will use team record and other basic data points so that their ballots aren’t completely random, but there are certainly some differing opinions that keep the poll diverse.

This ranking system likely relies more on the “eye test” and recent wins or losses than any relevant data. Teams that are fun to watch or have star players often get artificial bumps over teams that may be better but don’t pop off the TV screen. Even the tried and true ‘if they win move them up, if they lose move them down’ doesn’t hold across the board when the AP Poll comes out every week. It doesn’t make for a completely random poll, but it can often times be unpredictable and change radically.

What does the ranking mean?

Opinion polls rank how we feel about at a team. For example, despite the fact that there are only two undefeated teams in the country right now there are still two teams with losses that are receiving first place votes. That’s because those pollsters feel that the Tennessee Volunteers or the Duke Blue Devils, even with their losses, are better teams than those without any losses. The problem with the poll is that “better” isn’t defined.

When the AP Poll ranks its Top 25 teams, it’s not giving any context to what they’re Top 25 in. Are they the Top 25 teams right now? Are they the Top 25 since the start of the season? Will they be the Top 25 at the end of the season? Honestly, there’s probably at least one pollster approaching the poll from each of those angles, maybe even a combination of them. And the poll is then interpreted just as diversely as it’s composed. Fans will often complain that their team isn’t ranked high enough based on how they think the rankings should be viewed.

So bottom line, the meanings of these power rankings are vague at best. However, the AP Poll is the most popular and well known of all the ranking systems. I think there are two reasons for that. First, the general public doesn’t trust a formula. By their creators own admission, formulas and advanced algorithms can’t account for every variable and every nuance of the game. And while a human ranking is also far from perfect, it can account for things like injuries, non-basketball factors, and public perception.

But the second reason is likely why college basketball rankings are dominated by the AP Poll. Quite simply, it’s controlled by the media. TV, print, internet, every outlet where you can consume the game of college basketball other than being there in person is brought to you by the media. So of course they’re going to use their influence to promote their ranking system within the game. It doesn’t hurt that they’ve been doing it for about 80 years now either, so it’s established a base of influence. And that influence is how the AP Poll can effect the NCAA Tournament.

How does the ranking effect the NCAA Tournament?

On paper, the AP Poll or any other power rankings have zero influence on the selection committee. I don’t even need to put up a graphic because they aren’t a part of the Team Sheets reviewed leading up to Selection Sunday. That said, it doesn’t mean they haven’t already influenced the committee members before they walk into the room.

Throughout the season, specifically early in the season, the power rankings “feel” more accurate than cumulative or predictive models because they’re built for small sample sizes. There isn’t a lot of data to go on, so the public’s idea of how good a team is are largely based on the eye test or expectations. Since that’s exactly how the AP Poll is voted on, it matches public perception. Before the other rankings are even really established, the AP Poll already seems like the most accurate.

Then throughout the season, any time you or a committee member is looking for a good game to watch, you’re immediately drawn to games where two Top 25 teams are playing each other. Again, it’s the AP Poll Top 25, not one of the other ranking systems. In fact, often times the AP Top 25 teams will be the only games listed or promoted, giving those teams prominence for the casual fan.

And lastly, power rankings tend to be biased. They’ll often favor traditionally successful programs or the more well known conferences. Sure, no committee member is completely without bias, but it doesn’t help when a poll that isn’t based on any set data is the most used source material for the media and general public. The power rankings may not be in the room with the selection committee, but their influence is most certainly there.

How To Use The Rankings

I hope that long winded explanation of all these ranking systems has helped you understand what they actually mean. I know that in just researching and writing this piece I can point to a few times when I’ve incorrectly referenced KenPom to try to make a point in an article. So let’s summarize with a quick review of the best way to use these rankings to argue with friends, win a bar bet, or become champion of your office bracket pool in March.

First is the cumulative rankings like the NET. This is your best resource for determining how well a team has done so far this season. That said, if you’re trying to predict where a team will be seeded in the NCAA tournament, you’re going to need more than just the NET. It doesn’t account for the next two months of basketball, and that’s a lot of data to leave out.

That’s where predictive rankings like KenPom can help. That’s not to say KenPom has all the answers either. If you’re using that ranking to review how a team has performed to date, then you’re doing it wrong. In Pomeroy’s own words his system is “designed to be purely predictive”, and that’s how we should use it. If you use his model to predict the outcomes of the next two months for a team and combine that with the actual results currently in the NET, that would be a good estimate of how the selection committee is going to view them on Selection Sunday.

And while the AP Poll may not have much of an effect on Selection Sunday, it’s certainly a major part of college basketball. It influences which teams get prominent screen time, which in turn influences the perceptions of fans, or more importantly recruits. It’s also the most debated ranking system, which can often raise the public awareness of the sport before February when football season officially ends. While they may not be based on metrics or data, the power rankings certainly have their place in college basketball.

All that said, the best thing about the rankings is that on Monday, March 18th you can throw them all out the window. Sure the predictive rankings may be able to help you fill out your bracket, but I don’t know any of them that would have suggested you pick UMBC to upset Virginia in the first round last year. The rankings are purely a way to get us to the Tournament and fairly set up the most entertaining and perfect post-season in all of sports. We can be as ordered and measured in our rankings as we’d like for the next two months. But there’s a reason they call it March Madness.