New cfbdatawarehouse.com

[Lehigh Football Nation]

FBSchedules.com @FBSchedules
(https://twitter.com/FBSchedules)

New CFB Data Warehouse is coming soon, not related to the old one. Follow them at @cfbdatawh (https://twitter.com/cfbdatawh/), new URL will be thecfbdatawarehouse.com (https://t.co/mGbazobJn6)

[bonarae]

xchinscratchx

What the future holds... maybe... hopefully they're a better version of the old one.

[ST_Lawson]

FBSchedules.com @FBSchedules
(https://twitter.com/FBSchedules)

New CFB Data Warehouse is coming soon, not related to the old one. Follow them at @cfbdatawh (https://twitter.com/cfbdatawh/), new URL will be thecfbdatawarehouse.com (https://t.co/mGbazobJn6)




Awesome. I was looking into trying to put something together to do the same thing, but what they're doing looks significantly better than anything I could put together (my database design skills are somewhat lacking to be able to put something like that together).

[Lehigh Football Nation]

Awesome. I was looking into trying to put something together to do the same thing, but what they're doing looks significantly better than anything I could put together (my database design skills are somewhat lacking to be able to put something like that together).

I was trying to get something together as well through Wikimedia, but all I kept finding was that depending on old media guides wasn't always the best way to get the information. The amount of data that held was staggering, and it's looking like it's built on the foundation of the database of the old (or at least they tell me). We'll see.

[ElCid]

Will it have the same historical data?

[RootinFerDukes]

http://s2.quickmeme.com/img/f1/f17ff7c5ef2b35e22d4df1d255bdf97bacd5fce6feffa4a0ae afe1cff9e2e39f.jpg

[clenz]

Will it have the same historical data?
From questions I asked on twitter it will have all the same FBS and FCS data - no answer beyond FBS and FCS. It sounds like a mostly "smooth" transition through 2013 as CFBDatawarehouse was up to date through there.

It looks like it will have a greatly different look.

http://www.anygivensaturday.com/attachment.php?attachmentid=25818&stc=1

[Go Green]

Terrific news!!!!

[GreenGlasses]

xhurrayxxhurrayx I used CFDWearhouse more than I probably should have xhurrayxxhurrayx

[ST_Lawson]

xhurrayxxhurrayx I used CFDWearhouse more than I probably should have xhurrayxxhurrayx

Pretty sure I was on there hourly during football season. Maybe only daily during the off season.


Sent from my iPhone using Tapatalk

[GreenGlasses]

The new CFBDataWarehouse is now up and running. I haven't looked at it that much but what I have it looks more streamlined than the older version.

http://thecfbdatawarehouse.com/home

[ursus arctos horribilis]

God damn this is great news!

[GreenGlasses]

God damn this is great news!

Looks like its only going to be FBS and FCS right now. And it looks like most FCS schools aren't updated to the 2016 season yet, but at least there is a reference.

[ursus arctos horribilis]

Looks like its only going to be FBS and FCS right now. And it looks like most FCS schools aren't updated to the 2016 season yet, but at least there is a reference.

Trudat. They could probably crowd source information and roll it up quickly if they wanted but then it' goes into verifying stuff etc.


I'm just happy it is back.

[dgtw]

I didn't see any seasons other than 2015 for FCS or 2016 for FBS. Is there historical data on there somewhere?


Sent from my iPhone using Tapatalk

[ASU33]

Great news!

[cfbinfo]

Hi Guys. I'm the owner of theCFBDatawarehouse. What's currently active is more of a beta site. I'm working on getting a FCS data set for 2015 & 2016 so please be patient there. I will hopefully have something up there this week for that. Please feel free to ask me any questions. I'll try to drop by here daily the next couple of weeks to answer any questions you may have. Thanks and here's to a good football season for all of us!

xthumbsupx

[GreenGlasses]

Hi Guys. I'm the owner of theCFBDatawarehouse. What's currently active is more of a beta site. I'm working on getting a FCS data set for 2015 & 2016 so please be patient there. I will hopefully have something up there this week for that. Please feel free to ask me any questions. I'll try to drop by here daily the next couple of weeks to answer any questions you may have. Thanks and here's to a good football season for all of us!

xthumbsupx

I understand this is just the beginning and all the bugs and everything isn't worked out yet. But later on do you or your staff (if you have one) plan on adding DII, DIII and NAIA. Also do you plan on adding discontinued teams. I like many other is just glad there is a place for research on scores again.

[cfbinfo]

I understand this is just the beginning and all the bugs and everything isn't worked out yet. But later on do you or your staff (if you have one) plan on adding DII, DIII and NAIA. Also do you plan on adding discontinued teams. I like many other is just glad there is a place for research on scores again.

Yes. Hopefully over the course of the next year or so we will be adding DII, DIII & NAIA games. But we wanted to get the FBS & FCS data up first and get all the bugs worked out.

[RootinFerDukes]

I will personally update stuff for the caa if they don't have time. I'm sure you could find a volunteer to put in the leg work for each conference.

[ursus arctos horribilis]

Yes. Hopefully over the course of the next year or so we will be adding DII, DIII & NAIA games. But we wanted to get the FBS & FCS data up first and get all the bugs worked out.

Hey, since you are new (and a very welcome member btw) you have a 10 post thing that keeps your posts from showing up. Look for an email coming at you from me and I'll try to unchain it for you.

[ursus arctos horribilis]

Yes. Hopefully over the course of the next year or so we will be adding DII, DIII & NAIA games. But we wanted to get the FBS & FCS data up first and get all the bugs worked out.

BTW, there are many resources and people around here that would be willing to help you I will bet so keep this in mind to help wiht your workload. We also have connections to the D2 board via several members here as well.

[GreenGlasses]

BTW, there are many resources and people around here that would be willing to help you I will bet so keep this in mind to help wiht your workload. We also have connections to the D2 board via several members here as well.

I'm sure he could find a lot of help with the DII people at http://www.d2messageboard.com/forumdisplay.php?1-Football. Also each conference at http://www.d2football.com/ has a beat writer there that would probably help in anyway possible.

[ursus arctos horribilis]

I'm sure he could find a lot of help with the DII people at http://www.d2messageboard.com/forumdisplay.php?1-Football. Also each conference at http://www.d2football.com/ has a beat writer there that would probably help in anyway possible.

Yeah, Laker was brought over here by OB 55 years ago from one or both of those boards.

[cfbinfo]

Thanks for the suggestions guys. I'll work on getting your admin something for crowd sourcing the missing data. I was able to find a 2016 data set and am working to get that uploaded. I think I'll just need 2015 if I can't find a data set somewhere.

[WestCoastAggie]

let me know if you any info from NC A&T.

[RootinFerDukes]

I only looked at it via a phone, but will the pages consist of more than just their season schedules?

[aceinthehole]

Supid question, but is the new owner actually trying to rebuild and research all the results from old site from scratch?

Or do they have the scorce information from the old site provided by the previous owner and and they are just reformatting this data on a new website?

Seems crazy to rebuild and verify new data from the public, that took years to ccomplete. Also, do we know exactly what happened to the old website owner/researcher?

[ST_Lawson]

I only looked at it via a phone, but will the pages consist of more than just their season schedules?

At this point, it has the season schedules and the series information (for example, here's the history of Western Illinois vs Illinois State: http://thecfbdatawarehouse.com/series/western-illinois-leathernecks/illinois-state-redbirds
Once the team data is in the database, there should be quite a bit more info and relationships that can be displayed...records vs certain conferences for example...should be just a matter of setting up a database call where you get wins and losses from the team's database where the opponent's conference is X (it's a little more than that, but shouldn't be much more difficult than what they've already got set up).



Supid question, but is the new owner actually trying to rebuild and research all the results from old site from scratch?

Or do they have the scorce information from the old site provided by the previous owner and and they are just reformatting this data on a new website?

Seems crazy to rebuild and verify new data from the public, that took years to ccomplete. Also, do we know exactly what happened to the old website owner/researcher?

From what I heard, the old site got hacked and the database was corrupted. I don't know why he/they decided to just shut it down rather than trying to rebuild it or something (and if the backups were also somehow corrupted...assuming he had backups). My guess is that the new site's creator(s) scraped the data from a backup of the old CFB Data Warehouse site like the archive.org wayback machine...but that's just based on the fact that the newest backup for most of the old site had up to 2015 data in it, and the new site creator said that they were in the process of uploading 2016 data currently (which they probably have to do relatively manually).

So, starting from scratch...no...probably scraping from a slightly older version of the old site.
Still tricky to make sure everything imported/converted over correctly to the new site's database, which is why it's taking a while.
Site is essentially still in beta while they're bringing in the most recent data and getting everything updated to present day in terms of schedules, coaches, conferences, etc.
Since they have a much cleaner, more modern interface, and seem pretty dedicated to getting this thing going, I'd imagine that eventually we'll see quite a bit more info/data/capabilities than we saw on the old CFBDW site...it'll just take a little while to get there.

[Lehigh Football Nation]

If you need crowdsourced stuff from the Patriot League and before, I'm your guy.

In terms of Lehigh information, one of the more valuable info the old CFB Data Warehouse had was Lehigh's "Middle Three" conference information (basically back when Lehigh/Lafayette/Rutgers was considered the "Middle Three"). AFAIK the CFB Data Warehouse was the only place where that was kept. I see the conference information for Lehigh is all "Patriot League" even for the days before the PL.

[cfbinfo]

If you need crowdsourced stuff from the Patriot League and before, I'm your guy.

In terms of Lehigh information, one of the more valuable info the old CFB Data Warehouse had was Lehigh's "Middle Three" conference information (basically back when Lehigh/Lafayette/Rutgers was considered the "Middle Three"). AFAIK the CFB Data Warehouse was the only place where that was kept. I see the conference information for Lehigh is all "Patriot League" even for the days before the PL.

I have to manually change all of the conference information and it's a tedious process. I'm through Albany right now. The way the database was set up the current conference is the default for all years until it's changed. I'm going to try to do a few teams a day and hope to have it all updated by the end of September.

[cfbinfo]

Supid question, but is the new owner actually trying to rebuild and research all the results from old site from scratch?

Or do they have the scorce information from the old site provided by the previous owner and and they are just reformatting this data on a new website?

Seems crazy to rebuild and verify new data from the public, that took years to ccomplete. Also, do we know exactly what happened to the old website owner/researcher?

We were able to get most of the data off of the web archives. It took a few days but we got it all.

[grizband]

Will you have historical information for teams, beyond scores? The site previously had coach information, conference associations, national champions, superlative stats , etc. All of this was very cool information.

Sent from my SM-N920V using Tapatalk

[cfbinfo]

Will you have historical information for teams, beyond scores? The site previously had coach information, conference associations, national champions, superlative stats , etc. All of this was very cool information.

Sent from my SM-N920V using Tapatalk

Yes. If you go through the schedules you will see the schools conference and coach are listed. It's not accurate yet as I have to manually update all the conferences and coaches so that will take a bit.

[Serpentor]

Yes. If you go through the schedules you will see the schools conference and coach are listed. It's not accurate yet as I have to manually update all the conferences and coaches so that will take a bit.

We appreciate you guys providing this site, it's really useful!

[ST_Lawson]

Recommendation for anyone using the site, especially those of us who use it regularly (or plan on using it regularly...I used the old one daily)...
If you're using an adblocker, you might want to whitelist the website. He's got a few google ads on the pages that probably help pay for the hosting and stuff to keep the site running.

I'm not going to tell anyone that they have to, but if you are so inclined and would like to help support their site, this is a fairly easy way to do so.

(I'm not affiliated with the site, but I have my site setup the same way...minimal ads that pay for hosting and stuff)

[ursus arctos horribilis]

Recommendation for anyone using the site, especially those of us who use it regularly (or plan on using it regularly...I used the old one daily)...
If you're using an adblocker, you might want to whitelist the website. He's got a few google ads on the pages that probably help pay for the hosting and stuff to keep the site running.

I'm not going to tell anyone that they have to, but if you are so inclined and would like to help support their site, this is a fairly easy way to do so.

(I'm not affiliated with the site, but I have my site setup the same way...minimal ads that pay for hosting and stuff)

This is a good point. It probably took a good deal of cash hosting and so forth so let these get put some cash in the till so as not to see it go away.

It does take so much work setting a site like that up and working and worrying constantly about getting things kept up and making sure it doesn't fall to an attack so either pitch in with some cash or shut the ad block off on that site. They are also obligated to not discuss this sort of thing but since I am not financially involved, I'm not.

[smilo]

This is amazing stuff, love the series data. Unfamiliar with the old site, but it would be great to centralize all-time records, etc. on the front page for all levels as sports-reference does solely for FBS. Any player data expected? I suspect that might be a lot of excess. But anything that includes extensive sortable data for wins and records by school all-time is already amazing

[Catbooster]

It's great to have this back. Thanks xthumbsupx

[AmsterBison]

I'm interested in this process - especially the process of getting getting good data!

I've been maintaining an online fan guide for NDSU that includes all-time records for teams and stuff. I relied heavily on the CFB Data Warehouse data initially but found that it was incorrect for NDSU, included data for club teams/rugby teams, and was missing most D2 teams and sereral FCS schools (at least it was back when I started.)

I tried to reconcile the warehouse information with media guides and other third party information, but there are still discrepancies (like Lehigh and Lafayette not really agreeing on who won a particular game.) Not to mention that media guides aren't very accurate. I've spent years cleaning up the data and really should have taken notes on discrepancies. There's still quite a bit of clean up left to be done too.

If others are interested, I could make the data available even though it's a work in progress.

Here are some examples:

All FCS teams: http://www.bisonville.com/fanguide/teamsfcsbyname.html
All FBS teams: http://www.bisonville.com/fanguide/teamsfbsbyname.html
Playoff results: http://www.bisonville.com/fanguide/playoffbyname.html

Of course, I've set most of the pages up to be NDSU-centric, but the underlying data is not.

[Lehigh Football Nation]

I tried to reconcile the warehouse information with media guides and other third party information, but there are still discrepancies (like Lehigh and Lafayette not really agreeing on who won a particular game.) Not to mention that media guides aren't very accurate. I've spent years cleaning up the data and really should have taken notes on discrepancies. There's still quite a bit of clean up left to be done too.

I tried to start the process of doing this for Lehigh and Lafayette and some PL teams, and I was stunned how many discrepancies there were, even today. I knew it was a daunting task, but realizing how much cross-referencing I would need to do really dissuaded me from following up. Plus, I was thinking of building a DB based on the Wikimedia platform - which has its own issues of some teams having extensive records, and others having almost none. Wikimedia was a horrible choice.

[clenz]

I tried to start the process of doing this for Lehigh and Lafayette and some PL teams, and I was stunned how many discrepancies there were, even today. I knew it was a daunting task, but realizing how much cross-referencing I would need to do really dissuaded me from following up. Plus, I was thinking of building a DB based on the Wikimedia platform - which has its own issues of some teams having extensive records, and others having almost none. Wikimedia was a horrible choice.
This is why I stopped doing the historical sheets I was doing about 2 years ago for MVFC teams. I could get my information from one place (most of it CFBDataWarehouse) and cross references it against a school program which I would cross reference vs the other school program and then against the MVFC program and have at least 2 or 3 different sets of numbers, locations, even who won the game.

Thus errors were on the sheets and I was willing to fix them but people, I only shared them here, were pretty ****ty how they went about letting me know about them. Mostly ungrateful. So I stopped, because I was doing it for nothing in my free time. I had better **** to do than that.

[AmsterBison]

LFN, hit the nail on the head. Most data for scores is completely separate by team so you could have the page for Team A showing that they beat Team B in 1996 34-14, but when you go to the 1996 Team B page, you might not see the game at all or the score might be different. And, cripes, the NCAA waving their magic wand on vacating/un-vacating wins doesn't help.

And clenz, yeah, I ran into the same problem, if you check CFB data warehouse versus SIU's media guide versus WIU's media guide, there are going to be discrepancies that need to be resolved. Most of them can be puzzled out (even if it means using the google news archive search to find what newspapers reported.) Heck, there are a surprising amount of media guides that have year-by-year WLT that don't match up the total on the bottom which kind of boggles my mind.

Anyway, I have some tips:

http://prwolfe.bol.ucla.edu/cfootball/ <- this site has the schedules for every team playing college football this year. Once the season starts rolling, he updates the scores page with every score. If you can write code to read the page in, you are good to go. However, I've been using that site since I started the Bisonville fan guide back in 1999 which makes me wonder which will be the first to go: that site or me? One of my buddies is CEO of a sports score aggregating site that mines the internet for HS sports scores in real time... I'm considering using that as my backup.

Anyway, I started with the college data warehouse scores and reconciled them with another site.... I believe it might be this: http://www.jhowell.net/cf/scores/byName.htm

One nice thing about that is that he has a notes page where he shows his work on bad data: http://www.jhowell.net/cf/scores/Notes.htm

Here's an example of one of his notes: "1935 - I am not including the triangular game involving Colgate, St. Lawrence, and Amherst in which each team played two quarters against each of the other two teams on the same day." WTF, Colgate? :)

Er, although, it seems like the source I'm thinking about had scores back to 1949 an wasn't just limited to mostly FBS schools.

[ST_Lawson]

Here's an example of one of his notes: "1935 - I am not including the triangular game involving Colgate, St. Lawrence, and Amherst in which each team played two quarters against each of the other two teams on the same day." WTF, Colgate? :)

Man...football scheduling was weird back in the day.

[Lehigh Football Nation]

Man...football scheduling was weird back in the day.

The game was set up for Colgate in 1935 as a part of their homecoming celebration, where the starters would play three "halves" of football: Colgate vs. St. Lawrence, Colgate vs. Amherst, and then Amherst vs. St. Lawrence. Colgate had a tremendous team in 1935 and the game was in effect an exhibition, with most of the starters being subbed out early. The following week they had a game vs. Iowa.

Incidentally, those results are in the new CFB Data Warehouse!

http://thecfbdatawarehouse.com/team/colgate-raiders/1935

[The Boogie Down]

Hi Guys. I'm the owner of theCFBDatawarehouse. What's currently active is more of a beta site. I'm working on getting a FCS data set for 2015 & 2016 so please be patient there. I will hopefully have something up there this week for that. Please feel free to ask me any questions. I'll try to drop by here daily the next couple of weeks to answer any questions you may have. Thanks and here's to a good football season for all of us! xthumbsupx

Looking forward to the finished product but am already all shades of impressed. Even w/the old website as a starting point, I can't imagine doing all that work. Just thinking about it is worthy of a migraine so again kudos to you and your team! Best of luck going forward!!!

[The Boogie Down]

I'm interested in this process - especially the process of getting getting good data!

I've been maintaining an online fan guide for NDSU that includes all-time records for teams and stuff. I relied heavily on the CFB Data Warehouse data initially but found that it was incorrect for NDSU, included data for club teams/rugby teams, and was missing most D2 teams and sereral FCS schools (at least it was back when I started.)

I tried to reconcile the warehouse information with media guides and other third party information, but there are still discrepancies (like Lehigh and Lafayette not really agreeing on who won a particular game.) Not to mention that media guides aren't very accurate. I've spent years cleaning up the data and really should have taken notes on discrepancies. There's still quite a bit of clean up left to be done too.

If others are interested, I could make the data available even though it's a work in progress.

Here are some examples:

All FCS teams: http://www.bisonville.com/fanguide/teamsfcsbyname.html
All FBS teams: http://www.bisonville.com/fanguide/teamsfbsbyname.html
Playoff results: http://www.bisonville.com/fanguide/playoffbyname.html

Of course, I've set most of the pages up to be NDSU-centric, but the underlying data is not.


Holy cannoli, this is pretty damned great too! Wish I had known about it the second the original CFB Data Warehouse went down. I only root for NDSU when you guys are taking on the FBS and usually hate all forms of green-&-yellow but you have some great info and I do like how your color scheme is NDSU-centric. Many thanks for all the work. Would be cool if you and "thecfbdatawarehouse" work together going forward!!!

[grizband]

Yes. If you go through the schedules you will see the schools conference and coach are listed. It's not accurate yet as I have to manually update all the conferences and coaches so that will take a bit.What I really meant was a page for each team, that listed the record and coach for each session in a teams history. It was really cool information to have in one place.

Sent from my SM-N920V using Tapatalk

[cx500d]

What I really meant was a page for each team, that listed the record and coach for each session in a teams history. It was really cool information to have in one place.

Sent from my SM-N920V using Tapatalk

Go look at the bisonville data linked above. That info is there.

[cx500d]

I'm interested in this process - especially the process of getting getting good data!

I've been maintaining an online fan guide for NDSU that includes all-time records for teams and stuff. I relied heavily on the CFB Data Warehouse data initially but found that it was incorrect for NDSU, included data for club teams/rugby teams, and was missing most D2 teams and sereral FCS schools (at least it was back when I started.)

I tried to reconcile the warehouse information with media guides and other third party information, but there are still discrepancies (like Lehigh and Lafayette not really agreeing on who won a particular game.) Not to mention that media guides aren't very accurate. I've spent years cleaning up the data and really should have taken notes on discrepancies. There's still quite a bit of clean up left to be done too.

If others are interested, I could make the data available even though it's a work in progress.

Here are some examples:

All FCS teams: http://www.bisonville.com/fanguide/teamsfcsbyname.html
All FBS teams: http://www.bisonville.com/fanguide/teamsfbsbyname.html
Playoff results: http://www.bisonville.com/fanguide/playoffbyname.html

Of course, I've set most of the pages up to be NDSU-centric, but the underlying data is not.

There are some issues....Case in point:



Jacksonville (FL) (http://www.bisonville.com/fanguide/team000458.html)
PFC (http://www.bisonville.com/fanguide/2016/conf000126.html)
John Grass (http://www.bisonville.com/fanguide/coaches/coach008880.html)


Jacksonville State (http://www.bisonville.com/fanguide/team000143.html)
OVC (http://www.bisonville.com/fanguide/2016/conf000041.html)
John Grass (http://www.bisonville.com/fanguide/coaches/coach008880.html)

[AmsterBison]

There are some issues....Case in point:



Jacksonville (FL) (http://www.bisonville.com/fanguide/team000458.html)
PFC (http://www.bisonville.com/fanguide/2016/conf000126.html)
John Grass (http://www.bisonville.com/fanguide/coaches/coach008880.html)


Jacksonville State (http://www.bisonville.com/fanguide/team000143.html)
OVC (http://www.bisonville.com/fanguide/2016/conf000041.html)
John Grass (http://www.bisonville.com/fanguide/coaches/coach008880.html)




Thanks! Gave me an idea to search the database to see if any coaches were assigned to more than one team in a year. Found 54 and fixed all the ones from the end of WWII except for 1988 (I have a William Spencer as coach for both Livingstone and Clark Atlanta.)

Anyway, the John Grass problem is fixed. Jacksonville State/Jacksonville, Southern Ilinois/Southern, all the freaking Concordias/Concords are hard to keep straight. And keeping up with coaching changes is practically a full-time job.

Only interesting find was the UC Davis and Nevada media guides both claim that Jack Glascock was their coach in 1917... maybe it's two different guys? Maybe the same guy coaching two different teams? Twist: UC Davis played Nevada that year.

BTW, if anybody knows where to find historical information for JC Smith, Livingstone, Clark Atltanta, and Southern U, please let me know.

[cfbinfo]

Do you guys have a good data set for 2015 FCS games? I was able to get 2016 updated.

[Lehigh Football Nation]

Do you guys have a good data set for 2015 FCS games? I was able to get 2016 updated.

ESPN has the data, use the dropdown to select 2015. It includes FCS as well as FBS and has the games weekly.

http://www.espn.com/college-football/scoreboard/_/group/81/year/2015/seasontype/2/week/1

[AmsterBison]

Do you guys have a good data set for 2015 FCS games? I was able to get 2016 updated.

http://bisonville.com/test/scores2015.txt <- the second team is the home team (unless there is a location listed at the end of the line - that indicates a neutral-site game.)

Edit: Rats, that only goes up to Dec. 19th. I can generate the data though. What format would you like? Would .csv work? And information do you want? Is the date format OK?

Note: That's not my work, it's Peter Wolfe's.

[cfbinfo]

Hey guys. Wanted to let you know I have all the FCS games up now. Also had to rebrand to cfbinfo.com. I'm trying to finish up the conference and coach history but that's tedious.