Sorry for the slow response, my day job as a security consultant here in Chicago was pretty crazy today.
So yes, we're building a scraper in Ruby that reads the raw PBP data from the HTML, JSON, and other files from stats.ncaa.org. That data gets parsed into CSV files that I will eventually be building into a relational database, front-ended by a Ruby on Rails application (maybe with some new hotness JavaScript framework that makes it easy to browse around).
Initial goals are just to recreate useful stats in a way that people can consume them without having to go through the hell of parsing the data like I have. Longer term goals are to come up with interesting queries that can be run against the data set, maybe create an API, etc. For those familiar with Rails, ActiveRecord relationships make this really nice and straightforward to do interesting things.
HOW CAN YOU HELP ME
A lot of you are already interested in this, asking how you can help, etc. first off, I've created an email that you can get me at for this project [email protected]... second, here's the list of things we need help with:
The biggest thing we need right now is some QA testing... essentially, people who are willing to take a set of CSV files, go to the corresponding game page, and make sure our data looks reasonably accurate. The more people help out here, the more accurate our data, the more things I can fix in the parser for future data, and the quicker I can get to building features that actually are interesting.
If you have interesting data sets in a not sucky format that I can consume to add to what we hope will be a pretty sweet setup, that's wonderful.
If you have Ruby/Rails experience and want to contribute, that's wonderful (frankly, I'm not a full time programer, I'm a computer security guy... I consult companies on security of their applications... read: I break applications, I don't build them).
If you have graphic or front-end experience, especially if you've done some of this in something like Ruby/Rails, or one of those fancy new single-page JavaScript frameworks that can do really nice single-page applications to make load times fast and what not, that could be interesting down the road.
If you have ideas on what would make this super awesome, well, we already have a lot of features to get done, but shoot them my way.
If you just think this is awesome and want to provide words of encouragement, hey sure, I'll take it... if you see this data raw, you will come to learn pain when you try to parse it :).
Side note, I don't really post on MGoBlog often, but I read it religiiously. I took this project on because I thought this would be a nice way to give something back... so I'm really hopeful that I can make that happen for everyone and that that will turn into more awesome content for the blog.
Ugh... that's going to be a problem I think. Guess some of the younger guys will have to step up... expectation for the position and other coach talk what not.
Wish we could have landed him, but not sure we even really wanted him that much. Part of my befuddlement with where all the DLinemen were going to come from on our team. We're fortunate that BWC and Q have been decent this year, or we'd be screwed.
I mean, you have to realize that this type of thing, it isn't like Phil Steele is reviewing a lot of game film. Jake probably could use a few more sacks, TFLs, and FFs to get his numbers up and then you'll see him there, but yeah, I do agree with you, he's been awesome.
Recent Comments
Anyone know where I can find full replays of the hockey games? I've missed all of them so far.
https://www.change.org/p/coaches-replace-the-ncaa-with-an-org-focused-a…
So, I did this while on my lunch break today. It's not wonderful, but hey, why not.
https://www.change.org/p/coaches-replace-the-ncaa-with-an-org-focused-a…
Please feel free to suggest edits and what not, as I through it together.
But if more of you would like to help in the effort, please hit me up at [email protected].
THANKS!!!
Hey all, Nate here.
Sorry for the slow response, my day job as a security consultant here in Chicago was pretty crazy today.
So yes, we're building a scraper in Ruby that reads the raw PBP data from the HTML, JSON, and other files from stats.ncaa.org. That data gets parsed into CSV files that I will eventually be building into a relational database, front-ended by a Ruby on Rails application (maybe with some new hotness JavaScript framework that makes it easy to browse around).
Initial goals are just to recreate useful stats in a way that people can consume them without having to go through the hell of parsing the data like I have. Longer term goals are to come up with interesting queries that can be run against the data set, maybe create an API, etc. For those familiar with Rails, ActiveRecord relationships make this really nice and straightforward to do interesting things.
HOW CAN YOU HELP ME
A lot of you are already interested in this, asking how you can help, etc. first off, I've created an email that you can get me at for this project [email protected]... second, here's the list of things we need help with:
Side note, I don't really post on MGoBlog often, but I read it religiiously. I took this project on because I thought this would be a nice way to give something back... so I'm really hopeful that I can make that happen for everyone and that that will turn into more awesome content for the blog.
from MGoBlogger to OC.
It's like getting pwn'd, but it's much, much worse.
MAAAAANNNNNBAAALL
Fuck Sparty. If I had a little brother, I'd kick him in the nads.
That is all.
F Sparty. But REALLY F OHIO.
Ugh... that's going to be a problem I think. Guess some of the younger guys will have to step up... expectation for the position and other coach talk what not.
Wish we could have landed him, but not sure we even really wanted him that much. Part of my befuddlement with where all the DLinemen were going to come from on our team. We're fortunate that BWC and Q have been decent this year, or we'd be screwed.
I mean, you have to realize that this type of thing, it isn't like Phil Steele is reviewing a lot of game film. Jake probably could use a few more sacks, TFLs, and FFs to get his numbers up and then you'll see him there, but yeah, I do agree with you, he's been awesome.
7 TDs and 2 FGs for Michigan...
I've got Denard with 150 rushing 200 passing four total touchdowns
I've got Fitz with 100 yards rushing and a touchdown
I've got Rawls with 75 yards and two touchdowns
Jake Ryan gets two sacks
feel like Hoke recruits good kids. Not that we didn't before, but just that some of these kids Hoke is bringing in are really feel good stories.