Your Perfect Ranking System

[No_Skill]

I've wanted to try to make my own ranking system for a while now, so why not give it a shot? I know it's a daunting task, so I'm asking for a little help.

What factors do you feel are the most important when calculating rankings?

Straight win/loss?
Upset victory?
Magin of victory?
Home vs Away victory?
Previous week's ranking?
Strenght of schedule?
Win loss record?
Other...


I also need to assign a "percentage of importance" to each factor. Basically a weight range.
Here's what I've got so far from most important to least important. Keep in mind this is VERY rough.

1. Previous ranking 50%
2. Straight win/loss 25%
3. Margin of victory 10%
4. Upset victory 10%
5. Home vs Away victory 5%

Ideas/thoughts?

[McNeese_beat]

Define "upset victory"

[dpd]

well an upset victory would be where a lower ranked team beats a higher ranked team and you can use your ranking system to decide when the upsets occur and so therefore you can use upsets as a factor in creating your ranking system and oh no wait i see where this is going you think you're pretty clever don't you

[Casey_Orourke]

There is no perfect ranking system, because no matter how hard you try there will always be a bias based on location, conference, past reputation, evaluation of strength of schedule and whether a team is actually liked by the raters.

[ngineer]

Margin of victory, to me, is meaningless. Some teams will run it up, others will play their second and third deep, as well as go conservative. Other factors go into the margin as well, such as turnovers, weather impact, etc. So unless you now the actual details of the outcome the margin can be very misleading.
Strength of schedule, to me should carry a lot of weight in determining a quality win. Beating up some teams that are clearly from an inferior program mean nothing. Winning on the road is also significant. The traveling team is restricted as to number of players available, plus, some stadiums are more adverse to the opposition than others in terms of conditions, noise, etc.

[Ronbo]

As stated there is no perfect ranking system. But this system would be the most accurate for pre-season. Once we get into the season 3-4 games it becomes alot easier. Pre-season is the hardest.

By order of importance.

1. Returning lettermen - indicates depth of experience.

2. Returning starters - indicates quality of experience.

3. Returning skill and All Conference players - indicates quality and experience in the critical and important areas.

4. Teams prior year record - indicates how skilled the returning players are by their prior years record.

5. Coaching staff's overall record of success - never underestimate the importance of the Coaches. There is a reason some win year after year after year.

6. 2007 schedule - if a team has scheduled too many FBS or top ranked FCS teams they will hurt their chances to make the playoffs.

7. Transfers - least important factor, but a factor. There are as many transfers that wash out as become starters. Hardest factor to gauge.

Use these predictors and you'll be close as possible to the best system for pre-season rankings.

[JohnStOnge]

There is no perfect ranking system but if your objective is to rate teams in terms of relative strength I think that won loss record should mean almost nothing. I think it's all about strength of opposition and how well a team does against them. Suppose, for example, that one team is 0-2 while the other is 2-0. But the 0-2 team lost by one point each time in consecutive games to LSU and USC. The 2-0 team had two one point wins over Prairie View and Savannah State.

I think that if you decide that the 2-0 team is stronger because of record, you're nuts.

Also, I think that the guys that do power rating models say that the best ones focus strictly on how well teams do in terms of point differentials in the context of how strong their opposition is. Those are the models that do the best jobs of predicting winners. The objective of such models is to get as close as possible to the actual margin of each game.

[JohnStOnge]

Margin of victory, to me, is meaningless. .

I think that if you pay attention to what guys who do power ratings say, you'll find that the models that consider margin of victory are better than the ones that don't. Now, they will include "diminishing returns" characteristics so that a team doesn't get too much credit for really blowing out a very weak opponent, but they'll say that the best type of model is one that focuses on trying to get "0" for the difference between actual margin and predicted margin and ignores won/loss entirely in terms of what it uses to "predict" or "rank."

Now, of course, models that are good at predicting margins will also be good at predicting won/loss. But the idea is that won/loss isn't what goes in to the model.

You can see a reference to the outlook on Sagarin's site, which includes the following language:

"In ELO-CHESS, only winning and losing matters; the score margin is of no consequence, which makes it very 'politically correct'. However it is less accurate in its predictions for upcoming games than is the PURE POINTS, in which the score margin is the only thing that matters. PURE POINTS is also known as PREDICTOR, BALLANTINE, RHEINGOLD, WHITE OWL and is the best single PREDICTOR of future games. The ELO-CHESS will be utilized by the Bowl Championship Series(BCS)."

[Franks Tanks]

The FCS already has the perfect end of the year ranking system, actually playing games. Thats all the ranking I need :D

[kardplayer]

I agree with both ngineer and John on this one...

1. I don't like margin of victory on its own - I don't want to encourage running up the score
2. Looking at how much Team A beats Team B by should have some input

What I try to do in "mismatch" games is look at the halftime scores. If Team A "should" beat Team B and is winning by 4 TD's at halftime before putting the backups in, and then the backups let Team B back in the game a bit, I don't count that against Team A.

I also think road wins against good teams are an excellent indicator of how good a squad is.

[McNeese_beat]

well an upset victory would be where a lower ranked team beats a higher ranked team and you can use your ranking system to decide when the upsets occur and so therefore you can use upsets as a factor in creating your ranking system and oh no wait i see where this is going you think you're pretty clever don't you

The point is that the rankings have to be established before a game can be considered an upset. So including upsets in a rating system is problematic by nature. Maybe "quality win" would be a more appropriate term.

[ngineer]

I also don't think rankings mean anything until a few games (i.e. 3) have been played...I really dislike all ths preseason rankings and POY awards.

[terrierbob]

What the heck is an Elo Chess?

[DaveK]

Margin of victory, to me, is meaningless. Some teams will run it up, others will play their second and third deep, as well as go conservative. Other factors go into the margin as well, such as turnovers, weather impact, etc. So unless you now the actual details of the outcome the margin can be very misleading.
Strength of schedule, to me should carry a lot of weight in determining a quality win. Beating up some teams that are clearly from an inferior program mean nothing. Winning on the road is also significant. The traveling team is restricted as to number of players available, plus, some stadiums are more adverse to the opposition than others in terms of conditions, noise, etc.

Totally agree. Great teams find a way to win the close games.

[dpd]

Sagarin uses an ELO-Chess model in computing his rankings:
http://www.usatoday.com/sports/sagarin/fbt06.htm

Here is the Wikipedia article: Elo Rating System (http://en.wikipedia.org/wiki/Elo_rating_system)

[Keeper]

All of the above make some sense.
But in the end the RANK is still subjective,
meaning that the results "appear" fairly
correct by personal or popular opinion.

The fairest assessment of a system is
one that is able to out-predict the others,
presumably by wins-losses, over the
greatest number of related teams.

It also depends on how much input you want
(that is: endless tedious hours of statistical
wrangling, like me) or you construct a
transparently short equation to do the work.
like Al Einstein.

[JohnStOnge]

All of the above make some sense.
But in the end the RANK is still subjective,
meaning that the results "appear" fairly
correct by personal or popular opinion.

The fairest assessment of a system is
one that is able to out-predict the others,
presumably by wins-losses, over the
greatest number of related teams.


I don't know if a model like Sagarin's is "subjective." The effort is to do the best possible job of coming as close as possible to actual margins. If there is subjectivity, it's in the design of the system. Somebody has to decide on what basic approach to take and what "variables" to consider.

But once it's designed, it's purely objective. There is no judgement involved in the output at all. The numbers go in and the results come out.

I agree that how well a system predicts is the fairest or best assessment of the quality of a system. And you can see a presentation of that at http://tbeck.freeshell.org/ . Go to "Results" then click on "NCAA" for info on how well systems have done in predicting I-A/FBS games or on the "I-AA" years to see how well they've donein predicint I-AA/FCS games.

[JohnStOnge]

Totally agree. Great teams find a way to win the close games.

The problem is that models that consider margin of error do better at predicting winners than models that just consider wins and losses do. One of the big problems with the BCS is that they insist on using models that consider only wins and losses...so they have bad relatively bad models as a result.

There are models that take an entirely different approach...like the Thomspson one that finished on top of prediction rankings for I-A last season (http://collegefootballsprs.com/). But when it comes to comparing systems that just consider wins and losses vs. systems that just consider what the scores (margins of victory were), the systems that consider scores are better.

[dpd]

when i used that freeshell.org link (using Open in New Window) my anti-virus software deleted a trojan horse that the website tried to load onto my system

[JohnStOnge]

when i used that freeshell.org link (using Open in New Window) my anti-virus software deleted a trojan horse that the website tried to load onto my system

Now I want to know what anti virus software you use! I use Norton and it's updated and I never got a "hit" at that site. Wondering if you've got something better.

[dpd]

symantec 9.0.0.38, anti-virus file last updated 8/24/2007

it could have been a coincidence but after opening that window symantec went nuts, quarantining and then deleting about a half-dozen apps, reminding me of the old gator lurkware - it might not have been that site, but then the only other option would be something being somehow broadcast from the ags site, and we would surely hope for that to not be the case

[travelinman67]

I agree with both ngineer and John on this one...

1. I don't like margin of victory on its own - I don't want to encourage running up the score
2. Looking at how much Team A beats Team B by should have some input

What I try to do in "mismatch" games is look at the halftime scores. If Team A "should" beat Team B and is winning by 4 TD's at halftime before putting the backups in, and then the backups let Team B back in the game a bit, I don't count that against Team A.

I also think road wins against good teams are an excellent indicator of how good a squad is.

A declining curve after a two possession lead. Don't know about the halftime lead...seen some teams hold it together for two, then fall apart after halftime.

Road win against good squad is a great measure. IMHO, a sign of good coaching, tight execution.

[JohnStOnge]

symantec 9.0.0.38, anti-virus file last updated 8/24/2007

it could have been a coincidence but after opening that window symantec went nuts, quarantining and then deleting about a half-dozen apps, reminding me of the old gator lurkware - it might not have been that site, but then the only other option would be something being somehow broadcast from the ags site, and we would surely hope for that to not be the case

Well...that's Norton. But I checked to make sure my Norton definitions were updated today then did a full system scan and got nothing. Wonder what happened.