Sagarin Results - Week 11

[thebootfitter]

I pulled these statistics for another exercise and figured I'd post them here too. Lest anyone get their undies in a bundle about the shortcomings of computer ratings, I am simply posting for comparison to the human polls. Take from it what you will.

Sagarin FCS Top-25



Rank SagRank RATING Team SOS SOSRank Conference
1 38 76.40 North Dakota State 56.50 133 Missouri Valley
2 58 71.46 Jacksonville State 50.26 175 Ohio Valley
3 62 70.13 Illinois State 55.41 136 Missouri Valley
4 66 69.56 Northern Iowa 59.75 116 Missouri Valley
5 77 66.49 New Hampshire 47.89 188 Colonial
6 81 66.11 South Dakota State 64.98 79 Missouri Valley
7 84 65.86 Chattanooga 52.83 156 Southern
8 85 65.68 Harvard 40.35 226 Ivy League
9 86 65.49 Coastal Carolina 44.57 204 Big South
10 88 64.90 Villanova 48.71 182 Colonial
11 95 62.75 Southern Illinois 59.14 122 Missouri Valley
12 99 62.24 SE Louisiana 46.81 195 Southland
13 100 62.18 Eastern Washington 50.77 168 Big Sky
14 102 61.63 Youngstown State 54.47 141 Missouri Valley
15 104 61.12 Indiana State 62.89 93 Missouri Valley
16 106 60.68 Fordham 41.86 219 Patriot
17 107 60.56 Sam Houston State 53.08 153 Southland
18 110 60.38 Liberty 51.31 165 Big South
19 111 60.37 Samford 50.69 170 Southern
20 115 59.44 Richmond 48.68 183 Colonial
21 117 58.62 McNeese State 47.38 192 Southland
22 120 58.29 Missouri State 60.04 110 Missouri Valley
23 121 58.12 Idaho State 54.21 143 Big Sky
24 124 57.47 Eastern Illinois 52.45 159 Ohio Valley
25 127 56.04 Western Illinois 63.14 91 Missouri Valley





Sagarin Conference Ranking



Group Conference CentralMean ConfMax
P5 SEC 85.64 98.84
P5 BIG 12 78.98 93.20
P5 PAC-12 78.11 91.29
P5 BIG TEN 74.10 89.44
P5 ACC 74.07 87.43
NA I-A INDEPENDENTS 69.66 83.85
G5 AMERICAN ATHLETIC 63.99 74.05
G5 MWC 63.29 77.70
FCS MISSOURI VALLEY 63.02 76.40
G5 CONFERENCE USA 58.84 78.97
G5 MAC 57.29 67.43
G5 SUN BELT 55.38 70.37
FCS BIG SOUTH 50.13 65.49
FCS COLONIAL 49.60 66.49
FCS SOUTHERN 49.44 65.86
FCS SOUTHLAND 48.93 62.24
FCS BIG SKY 47.76 62.18
FCS OHIO VALLEY 47.47 71.46
FCS PATRIOT 45.03 60.68
FCS IVY LEAGUE 43.99 65.68
FCS NORTHEAST 42.92 54.39
FCS I-AA INDEPENDENTS 42.37 42.37
FCS MID-EASTERN 35.52 51.36
FCS SWAC 32.07 51.65
FCS PIONEER 28.58 45.97

[rokamortis]

I know Sagarin is highly respected but I really like the idea of the composite ranking vs just one methodology. Although I think there may be a couple of WTF? options - the Massey composite seems to be pretty reasonable. I think it could potentially be improved if he tossed the single highest and single lowest score.

http://www.masseyratings.com/cf/compare1aa.htm

[jmrepak]

As far as computer polls and composites go I really like Massey and GPI. I'm not much of a fan of the others.

[UNH72Plus]

I'll be the first to admit that I don't think UNH is the #1 team in the country, but when you look at some of the rankings in the Massey system you really have to scratch your head! Three of the systems rank UNH 20th, 22nd, and 24th, respectively, behind such teams as 4-6 Western Illinois, 5-5 Eastern Illinois, and the real doozy 3-4 Wofford that one ranking has them at #16. I guess we'll have to sit back and see what the playoffs have in store.

[mainejeff]

I know Sagarin is highly respected but I really like the idea of the composite ranking vs just one methodology. Although I think there may be a couple of WTF? options - the Massey composite seems to be pretty reasonable. I think it could potentially be improved if he tossed the single highest and single lowest score.

http://www.masseyratings.com/cf/compare1aa.htm

I bet that Maine would beat a lot of the 56 teams ranked ahead of them.

[UNH_Alum_In_CT]

I'll be the first to admit that I don't think UNH is the #1 team in the country, but when you look at some of the rankings in the Massey system you really have to scratch your head! Three of the systems rank UNH 20th, 22nd, and 24th, respectively, behind such teams as 4-6 Western Illinois, 5-5 Eastern Illinois, and the real doozy 3-4 Wofford that one ranking has them at #16. I guess we'll have to sit back and see what the playoffs have in store.

In theory a compilation (Massey) is better than Sagarin, but when the compilation contains so much garbage and doesn't include a proven accurate poll (AGS), I'm not sure how anyone can use Massey as a definitive ranking. You can argue where UNH sits in the top ten, but to rank them 20th, 22nd or 24th is ludicrous. If Massey wants to be taken seriously, they should toss those three rankings! And to not include AGS when absolute garbage is included gives me even more reason to totally reject Massey's rankings.

[URMite]

Look at fourth from the right. Massey is too wide for the screen.

[URMite]

And I still think too many ranking, ratings, & polls put too much or too little emphasis on SOS.

[UNH_Alum_In_CT]

Look at fourth from the right. Massey is too wide for the screen.

Okay, I believed someone on AGS who said the AGS Poll wasn't included. xmadx

Wow, for a level of football that no one cares about, there sure are an awful lot of rankings!!!! Does anyone with a computer who creates a ranking get included within Massey????? And I still maintain that some of them are absolute garbage that result in Massey rankings being flawed.

[knucklehead]

I know Sagarin is highly respected but I really like the idea of the composite ranking vs just one methodology. Although I think there may be a couple of WTF? options - the Massey composite seems to be pretty reasonable. I think it could potentially be improved if he tossed the single highest and single lowest score.

http://www.masseyratings.com/cf/compare1aa.htm
Of course you like the one that puts CCU higher. I would too. But LU is 18 either way.

[WileECoyote06]

But until Sagarin includes non-Division I games, it's flawed as well.

I like the Dunkel Index too.

[rokamortis]

Okay, I believed someone on AGS who said the AGS Poll wasn't included. xmadx

Wow, for a level of football that no one cares about, there sure are an awful lot of rankings!!!! Does anyone with a computer who creates a ranking get included within Massey????? And I still maintain that some of them are absolute garbage that result in Massey rankings being flawed.


You can't say "Keep the ones that rate me higher and toss all the lower ones because they are garbage" They just have a different computation. That's why I said they should add an outlier provision and drop the single highest and lowest scores. You take the good with the bad ina system like this, but by shear volume you help eliminate the biases.

[rokamortis]

Of course you like the one that puts CCU higher. I would too. But LU is 18 either way.

You really are a numbskull. I don't like single rankings and choose composite. If I only wanted to highlight the ones that have Coastal highest I'd choose the numerous ones that have us at 1 or 2. So slink back into your cave under Liberty mountain.

[knucklehead]

You really are a numbskull. I don't like single rankings and choose composite. If I only wanted to highlight the ones that have Coastal highest I'd choose the numerous ones that have us at 1 or 2. So slink back into your cave under Liberty mountain.
Hah, I hear ya. Mostly messing with ya, calm down. I agree that the Massey is a better look because it's a composite. I just thought it was interesting that Liberty was 18 either way.

Me and mama bear are headed back to the cave now. What?

[rokamortis]

Hah, I hear ya. Mostly messing with ya, calm down. I agree that the Massey is a better look because it's a composite. I just thought it was interesting that Liberty was 18 either way.

Me and mama bear are headed back to the cave now. What?

You are going to hibernate with your mom? Ew and I don't mean Eastern Washington.

[knucklehead]

Dude I'm old and thus papa bear. Hah. I can't wait for next week, if course after we dispatch the Bucs.

[rokamortis]

Dude I'm old and thus papa bear. Hah. I can't wait for next week, if course after we dispatch the Bucs.

Of course? xeyebrowx CSU is probably tougher than you think. Monmouth certainly was.

[UNH_Alum_In_CT]

You can't say "Keep the ones that rate me higher and toss all the lower ones because they are garbage" They just have a different computation. That's why I said they should add an outlier provision and drop the single highest and lowest scores. You take the good with the bad ina system like this, but by shear volume you help eliminate the biases.

That's not what I said at all! I said toss the entries whose rankings were garbage. Can you seriously justify any ranking system that has UNH #20, 22 and 24? I'm sorry, that's an obvious quality issue. I would say the same thing about any ranking that had CCU that low.

[rokamortis]

That's not what I said at all! I said toss the entries whose rankings were garbage. Can you seriously justify any ranking system that has UNH #20, 22 and 24? I'm sorry, that's an obvious quality issue. I would say the same thing about any ranking that had CCU that low.

No, they just measure things differently. They may be wrong with what we would contend is a traditional rating but they aren't wrong in their own analysis. They are analyzing data and comparing all teams against the same measurement (except for the polls) so I'm ok with it. If you look at it in isolation you can tear it apart, but once you see all of the various opinions come together I think it makes quite a bit of sense.

[dewey]

I'll be the first to admit that I don't think UNH is the #1 team in the country, but when you look at some of the rankings in the Massey system you really have to scratch your head! Three of the systems rank UNH 20th, 22nd, and 24th, respectively, behind such teams as 4-6 Western Illinois, 5-5 Eastern Illinois, and the real doozy 3-4 Wofford that one ranking has them at #16. I guess we'll have to sit back and see what the playoffs have in store.

Amazing that anyone would have UNH in the 20's. If you want to see something else crazy go and look at how Harvard is ranked in numerous polls. BOOM!

Dewey

[skinny_uncle]

That's not what I said at all! I said toss the entries whose rankings were garbage. Can you seriously justify any ranking system that has UNH #20, 22 and 24? I'm sorry, that's an obvious quality issue. I would say the same thing about any ranking that had CCU that low.

I think the concept of the composite is that including 29 different ones waters down the effects of the ones that are off-the-wall.

[chattanoogamocs]

Okay, I believed someone on AGS who said the AGS Poll wasn't included. xmadx

Wow, for a level of football that no one cares about, there sure are an awful lot of rankings!!!! Does anyone with a computer who creates a ranking get included within Massey????? And I still maintain that some of them are absolute garbage that result in Massey rankings being flawed.

You think FCS has a lot (usually about 40ish)...go check out the FBS, there are over 100!

I like the Massey Composite overall (and respect that he doesn't pick and choose which one's he thinks are worthy and which one's are not), though I do have a suggestion for tweaking it in one way...for every 20 rankings included, drop the high and low (so if there were 40, you'd drop two highs, and two lows, etc)...that would edit out some of the weirder results...like a poll(s) having Harvard 1st.

[rokamortis]

You think FCS has a lot (usually about 40ish)...go check out the FBS, there are over 100!

I like the Massey Composite overall (and respect that he doesn't pick and choose which one's he thinks are worthy and which one's are not), though I do have a suggestion for tweaking it in one way...for every 20 rankings included, drop the high and low (so if there were 40, you'd drop two highs, and two lows, etc)...that would edit out some of the weirder results...like a poll(s) having Harvard 1st.

I suggested just dropping the single highest and lowest scores as I figured that would get rid of the potential true outliers - but I'm wondering why you propose one high/low for every 20. Is it based on some mathematical law, experience, or just sounded good? What I'd be afraid of is that by selecting out 10% of the options then it lessens the effect of the communal rating system. Perhaps 10% is a good number - I don't know.

[rokamortis]

One thing I kind of question, although it will have little effect on the whole, why he adds the GPI in. The composite already includes the same rankings in the GPI just that the GPI has hand selected certain ratings, so he is essentially adding in the same rankings more than once.

[JaxSinfonian]

Wow, for a level of football that no one cares about, there sure are an awful lot of rankings!!!!

You should see the number of rankings used in the FBS compilation: http://www.masseyratings.com/cf/compare.htm


Does anyone with a computer who creates a ranking get included within Massey?????
Massey publishes his standards for inclusion in the composite here: http://www.masseyratings.com/cf/aboutcomp.htm


And I still maintain that some of them are absolute garbage that result in Massey rankings being flawed.
Some terms are getting tossed around a bit here and in other posts; I'm not sure what folks in this thread understand and don't understand about Massey, so forgive me if you're caught up on this: Ken Massey has his own mathematical system that rates all college football teams at all levels. You can see his ratings/rankings for just FCS teams broken out here: http://www.masseyratings.com/rate.php?s=cf2014&sub=11605

That system is itself just one of dozens Massey includes in the composite rankings that some folks in this thread seem to be referring to as "the Massey rankings." The composite is not, in Massey's description, an attempt to be definitive about the proper ranking of teams, rather it's a way to compare both the teams and the different ranking/rating systems. Put another way: He's not trying with his composite to do what Nate Silver did in the presidential elections, carefully parsing lots of data to predict a winner; he's just throwing it all out there.

I find the composite handy as a way to balance the views of teams we see in the polls and in the well-known Sagarin rankings. In short, it's a reminder that there are countless ways to rate teams, and the only way to really find out who would win a game between two teams is to let them play that game. Also, the composite page is a handy reference. Click on any team's name, and it'll take you to the page for that team in Massey's rating system, with the ratings of every other team on the schedule, a calculated SOS rating, plus results for games played and score predictions for future games, based on Massey's system.

Again, for those who knew all that already, my apologies. I thought others might find it useful.

[Hammerhead]

Massy has "only" 7 MVFC teams in the top 25 compared to Sagarin with 9.



As far as computer polls and composites go I really like Massey and GPI. I'm not much of a fan of the others.

[rokamortis]

Massy has "only" 7 MVFC teams in the top 25 compared to Sagarin with 9.

Not quite. Massey has 9 MVFC in the top 20 in his ratings. There are 7 in the top 25 in the composite.

This is a good example of why I think you shouldn't look at just one rating. I know the MVFC is good, but having nearly 50% of the top 20 be from one conference seems a bit much to me especially when one is 4-6.

[thebootfitter]

But until Sagarin includes non-Division I games, it's flawed as well.

I understand the premise behind this idea, but I haven't yet seen any evidence to suggest that the results are significantly skewed by treating all sub-DI teams equally. Intuitively, sure, it seems valid. But what do the numbers actually indicate?

FWIW - I'm gathering data to answer this question, among a few others. Hopefully will have some preliminary results to look at by the end of this season.

[thebootfitter]

I know Sagarin is highly respected but I really like the idea of the composite ranking vs just one methodology. Although I think there may be a couple of WTF? options - the Massey composite seems to be pretty reasonable. I think it could potentially be improved if he tossed the single highest and single lowest score.

http://www.masseyratings.com/cf/compare1aa.htm
One reason that I follow some of the computer rating systems is for their use in predictions. The composite rankings can't be applied directly to predicting the outcome of future games, except by suggesting that a team with a higher ranking should win more often than not against a team of a lower ranking. (Lower being a bigger number here, obviously.)

That being said, I do like the composite rankings as a gut check against individual rating systems or polls.

[thebootfitter]

I bet that Maine would beat a lot of the 56 teams ranked ahead of them.
Quite possible. It typically takes a little while for computer ratings to catch up to teams that are on the upswing or downswing.

If the season had more match ups with fewer variables skewing the results (injuries, team dynamics, coaching changes, etc.), it would be a better predictor of match ups than the 80% that it is now, with lower standard deviations.

[thebootfitter]

And I still think too many ranking, ratings, & polls put too much or too little emphasis on SOS.
Generally speaking, SOS is more of an output than an input for computer ratings. It is typically a function of the scores of games and records of opponents, and their opponents' scores and the records of their opponents' opponents, etc. After the numbers are crunched and a SOS is determined within a system, it is sometimes plugged back into part of the equation, but not always.

Generally, you'll find that the computer ratings that use margin of victory are better predictors of future results rather than those that just use W/L records. Some that use margin of victory also diminish the value of margins above (or below) a certain threshold. This helps to alleviate the different dynamics of blowout wins and losses.

[Wallace]

Look at fourth from the right. Massey is too wide for the screen.Yes, you have to look hard because the human polls are so disconnected from computer ratings, last place. The GPI takes only a few ratings and mixes them with accountable polls to get a result... always on top of the Massey composite. Kinda remarkable.

[chattownmocs]

But the question is this. Is sagarin above average? If so wouldn't that make his better than an average of all the polls.

[rokamortis]

Yes, you have to look hard because the human polls are so disconnected from computer ratings, last place. The GPI takes only a few ratings and mixes them with accountable polls to get a result... always on top of the Massey composite. Kinda remarkable.

What I find remarkable is that Massey gets the composite updated on Sunday and it continues to add to around 40 data sources. The polls are added in Monday night / Tuesday morning and we have solid data. GPI isn't released until Tuesday, maybe Wednesday with a small portion of the data sources.

[rokamortis]

But the question is this. Is sagarin above average? If so wouldn't that make his better than an average of all the polls.

That's the wrong way to look at it. That's like saying: Hey guys, I like John's poll submissions - he generally does a good job. Why don't we forget about all of the other people and just let his ballot be the only one?

Think of it this way, Sagarin is one opinion. So you could rely on one opinion or get the opinion of many others and combine that to forma consensus. The Massey Composite is like a poll for computers (although he does have the human polls in there as well, the computers are the vast majority).

[UNH Fanboi]

That's the wrong way to look at it. That's like saying: Hey guys, I like John's poll submissions - he generally does a good job. Why don't we forget about all of the other people and just let his ballot be the only one?

Think of it this way, Sagarin is one opinion. So you could rely on one opinion or get the opinion of many others and combine that to forma consensus. The Massey Composite is like a poll for computers (although he does have the human polls in there as well, the computers are the vast majority).

The efficacy of a rating system can be measured by its prediction of future games. Not all ratings are created equal.

[rokamortis]

The efficacy of a rating system can be measured by its prediction of future games. Not all ratings are created equal.

I agree, not all ratings are created equal. But where do you come up with that ratings are only measured with how well they predict future games?

[UNH Fanboi]

I agree, not all ratings are created equal. But where do you come up with that ratings are only measured with how well they predict future games?

What other objective, measurable criteria is there?

[THE HERD]

Sagarin seems to be a pretty good predictor of single games, especially once your later in the season like this.

[thebootfitter]

Yes, you have to look hard because the human polls are so disconnected from computer ratings, last place. The GPI takes only a few ratings and mixes them with accountable polls to get a result... always on top of the Massey composite. Kinda remarkable.

What do you mean by "last place" here?

And what do you mean by "always on top of the Massey composite"?

[CasualFan]

What other objective, measurable criteria is there?
Massey looks at ranking violation percentage (teams ranked below a team they have beaten) as a flag for a poor model. If that percentage becomes large, then the poll is out.

[robsnotes4u]

Something that would be awesome, and will never happen.

Imagine if you could use the 5 most important criteria of a great football team, in your opinion, and match it to the best rating system, algorithm.

[thebootfitter]

Something that would be awesome, and will never happen.

Imagine if you could use the 5 most important criteria of a great football team, in your opinion, and match it to the best rating system, algorithm.
Tell me more.

You mean select measurables that you think are most important, say defense, measured by points allowed, and overall strength, measured by margin of victory combined with some measure of schedule strength, for example? And so on? It might be complicated to put together, but I don't see any reason why that couldn't happen.

[robsnotes4u]

Tell me more.

You mean select measurables that you think are most important, say defense, measured by points allowed, and overall strength, measured by margin of victory combined with some measure of schedule strength, for example? And so on? It might be complicated to put together, but I don't see any reason why that couldn't happen.

Exactly.

I think some people would be surprised at the outcome of a ranking system that fits their criteria. It would be a fun tool

[rokamortis]

Exactly.

I think some people would be surprised at the outcome of a ranking system that fits their criteria. It would be a fun tool

My team is always on top!