Home
i'm an actor, not a reactor

Primary links

  • About
    • $upport (lol)
    • Ethics
    • FAQ
    • Glossary
    • Privacy Policy
  • Contact
  • MGoStore
    • Hail to Old Blue
  • MGoBoard
    • MGoBoard FAQ
    • Michigan bar locator
    • Moderator Action Sticky
  • Useful Stuff
    • Depth Chart By Class
    • Hoops Depth Chart by Class
    • 2017 Recruiting Board
    • Unofficial Two Deep
    • MGoFlickr
    • Diaries, Windows Live Writer, And You
    • User-Curated HOF
    • Where To Eat In Ann Arbor
  • Schedule/Tix
    • Future Schedules (wiki)
    • Ticket spreadsheet
Home Diaries mikegros's blog

Navigation

  • Forums
  • Recent posts

User login

  • Create new account
  • Request new password

MGoElsewhere

  • @MGoBlog (Brian)
  • @aceanbender
  • @Misopogon (Seth)
  • @Aeschnepp (Adam)
  • @BISB
  • @EUpchurchPhoto
  • @FullOfTwitt (Fuller)
  • Hail to the Victors 2016
  • MGoFacebook
  • MGoPodcast
  • WTKA
  • Instagram

Michigan Blogs

  • Big House Blog
  • Burgeoning Wolverine Star
  • Genuinely Sarcastic
  • Go Blue Michigan Wolverine
  • Holdin' The Rope
  • MVictors
  • Maize 'n' Blue Nation
  • Maize 'n' Brew
  • Maize And Go Blue
  • Michigan Hockey Net
  • MMMGoBlueBBQ
  • The Blog That Yost Built
  • The Hoover Street Rag
  • The M Zone
  • Touch The Banner
  • UMGoBlog
  • UMHoops
  • UMTailgate
  • Wolverine Liberation Army

M On The Net

  • mgovideo
  • MGoBlue.com
  • Mike DeSimone
  • Recruiting Planet
  • The Wolverine
  • Go Blue Wolverine
  • Winged Helmet
  • UMGoBlue.com
  • MaizeRage.org
  • Puckhead
  • The M Den
  • True Blue Fan Forum

Big Ten Blogs

  • Illinois
    • Illinois Loyalty
    • Illinois Baseball Report
  • Indiana
    • Inside The Hall
    • The Crimson Quarry
  • Iowa
    • Black Heart, Gold Pants
    • Fight For Iowa
  • Michigan State
    • The Only Colors
  • Minnesota
    • GopherHole.com
    • The Daily Gopher
  • Nebraska
    • Corn Nation
    • Husker Max
    • Husker Mike's Blasphemy
    • Husker Gameday
  • Northwestern
    • Sippin' On Purple
    • Lake The Posts
  • Notre Dame
    • The House Rock Built
    • One Foot Down
  • Ohio State
    • Eleven Warriors
    • Buckeye Commentary
    • Men of the Scarlet and Gray
    • Our Honor Defend
    • The Buckeye Nine
  • Penn State
    • Slow States
    • Black Shoe Diaries
    • Happy Valley Hardball
    • Penn State Clips
    • Linebacker U
    • Nittany White Out
  • Purdue
    • Boiled Sports
    • Hammer and Rails
  • Wisconsin
    • Bruce Ciskie

Links of Note

  • Baseball
    • College Baseball Today
    • The College Baseball Blog
  • Basketball
    • Ken Pomeroy
    • Hoop Math
    • John Gasaway
    • Luke Winn/Sports Illustrated
  • College Hockey
    • Chris Heisenberg (Class of 2016)
    • College Hockey Stats
    • Michigan College Hockey
    • Hockey's Future
    • Sioux Sports
    • USCHO
  • Football
    • Smart Football
    • Every Day Should Be Saturday
    • Matt Hinton/Grantland
    • Football Study Hall
    • Football Outsiders
    • Harold Stassen
    • NCAA D-I Stats Page
    • The Wizard Of Odds
    • CFB Stats
  • General
    • Sports Central
  • Local Interest
    • The Ann Arbor Chronicle
    • Arborwiki
    • Arbor Update
    • Ann Arbor Observer
    • Teeter Talk
    • Vacuum
  • Teams Of The D
    • Lions
      • Pride of Detroit
    • Pistons
      • Detroit Bad Boys
      • Need4Sheed
    • Tigers
      • Roar Of The Tigers
      • Bless You Boys
      • The Daily Fungo
      • The Detroit Tigers Weblog
    • Red Wings
      • Winging It In Motown
      • On The Wings
    • Michigan Sports Forum

Beveled Guilt

Site Search

Diaries

  • New
  • Popular
  • Hot
  • This Month in MGoBlog History - April 2008: No Spring Game at the Big House! Hockey loses to ND in the Frozen Four!
    Maize.Blue Wagner - 3 days ago
  • Thirteen unlucky minutes (TL;DNR-This is a bit of rant about the refs)
    docwhoblocked - 2 weeks ago
  • Fan Satisfaction Index End of Season Bball Survey
    OneFootIn - 3 weeks ago
  • How likely are we to revert to the mean?
    Bo Glue - 3 weeks ago
  • It's time to avenge Villanova's 1985 NCAA tourney upset over Michigan
    Communist Football - 3 weeks ago
  •  
  • 1 of 2
  • ››
more
  • This Month in MGoBlog History - April 2008: No Spring Game at the Big House! Hockey loses to ND in the Frozen Four!
    Maize.Blue Wagner - 1,303 views
  • This Month in MGoBlog History - April 2008: No Spring Game at the Big House! Hockey loses to ND in the Frozen Four!
    Maize.Blue Wagner - 8 comments

MGoBoard

  • New
  • Recent
  • Hot
  • MBB: Wake Forest grad transfer PG Keyshawn Woods commits to OSU
    2 replies
  • Harmoniously OT: UM Men's Glee Club in AZ
    4 replies
  • Coach B w Andy Katz podcast
    1 replies
  • Michigan Alumni Club Scholarship Golf Tournament featuring Honorary Chair Glen Rice - This Saturday in Miami!
    4 replies
  • Hello? Boring Tuesday POSbang Happy Hour Thread
    107 replies
  • OT: UCL Semi-final First Legs
    14 replies
  • More evidence of awful culture at MSU, volleyball this time
    59 replies
  • OT: Rick Pitino Rumored to be Candidate for Detroit Mercy Job
    77 replies
  • New Jersey DE Aeneas DiCosmo: Prospect we should all want at UM
    55 replies
  • In-state recruiting rankings update
    124 replies
  • A piece of U-M broadcast history ends tonight.
    28 replies
  • Crootin': Joey Velazquez
    78 replies
  • OT: College Football video games coming back
    90 replies
  • LaMarr Woodley Opening K-8 School in Saginaw
    43 replies
  • SIAP: Jay Feely prom "controversy"
    36 replies
  •  
  • 1 of 6
  • ››
  • Harmoniously OT: UM Men's Glee Club in AZ
    4 replies
  • MBB: Wake Forest grad transfer PG Keyshawn Woods commits to OSU
    2 replies
  • New Jersey DE Aeneas DiCosmo: Prospect we should all want at UM
    55 replies
  • In-state recruiting rankings update
    124 replies
  • Hello? Boring Tuesday POSbang Happy Hour Thread
    107 replies
  • Coach B w Andy Katz podcast
    1 replies
  • More evidence of awful culture at MSU, volleyball this time
    59 replies
  • OT: Rick Pitino Rumored to be Candidate for Detroit Mercy Job
    77 replies
  • Michigan Alumni Club Scholarship Golf Tournament featuring Honorary Chair Glen Rice - This Saturday in Miami!
    4 replies
  • A piece of U-M broadcast history ends tonight.
    28 replies
  • Pep and Partridge Pressers
    11 replies
  • OT: UCL Semi-final First Legs
    14 replies
  • Crootin': Joey Velazquez
    78 replies
  • WBB Hello: 2020 G/W Makailah Griggs-Zeigler
    12 replies
  • LaMarr Woodley Opening K-8 School in Saginaw
    43 replies
  •  
  • 1 of 6
  • ››
  • Belleville coach Jermain Crowell mad at UM again
    244 replies
  • "Being Not-Rich at UM" Guide
    168 replies
  • The Evolution of Commerce - What Industries are Dying, What's Thriving?
    148 replies
  • Pep Hamilton on Shea: Can extend the play, make all the throws, plus other QB's
    129 replies
  • In-state recruiting rankings update
    124 replies
  • OT - Jalen Hurts possibly looking to transfer
    121 replies
  • Notre Dame Spring Game: analysis from M n B, video
    119 replies
  • No additional protest of Shea Patterson appeal by Ole Miss
    113 replies
  • Hello? Boring Tuesday POSbang Happy Hour Thread
    107 replies
  • OT: MSU digs hole deeper, Engler adviser: Nassar survivor's claims of payout 'fake news'
    106 replies
  • Nebraska football
    105 replies
  • OT: Map of college stadiums that sell alcohol
    96 replies
  • Karsen Barnhart - did we cool on him?
    92 replies
  • OT: College Football video games coming back
    90 replies
  • UCF Knights unveil 2017 championship banner
    89 replies
  •  
  • 1 of 6
  • ››

Support MGoBlog: buy stuff at Amazon

MGoUFRVisualization

By mikegros — November 12th, 2015 at 4:57 PM — 39 comments
Filed under:
  • football

A few weeks ago I decided I wanted to play with some web scraping and RShiny and thought building a toy for the MGoCommunity to be able to look at old players would be a fun project.

I now have the points to actually post a new message and it's all up-to-date, so here it is:

https://mikegros.shinyapps.io/ufr_analysis

I've put the project up on github:

https://github.com/mikegros/ufr_analysis

for anyone to grab and improve. Until I move it to a different host, I'll need to keep the live version updated if anyone makes changes.

Things it can do:

  • Make plots of individual players pluses, minuses, and total +/- from the charts of all the UFRs since the end of 2005.
  • Look at position group totals as well in the same way as above
  • Aggregate numbers to their mean by year for cleaner viewing
  • Compare players +/- side-by-side on the same scale
  • There is a sortable table for looking at best/worst career/season/game or for looking at the raw data for any players
  • RPS/other team metrics can be viewed in the Single Player Tab, their position is "TM" in the Sortable Table if that matters
  • Top 10/Bottom 10 lists - quick look at a wall of fame/shame to get context for particularly big or rough games.
  • Automatically checks for new UFR data on viewing. Should keep it up to date as long as it's used (probably at least by me) every 4 games or so. 

 

Things to fix:

  • Pressure metric for D is missing since 2010 due to a typo in my scraping. I'll fix it at some point but this is a side project for me so timeframe is unknown
  • iOS support bug - for some reason there is a resizing issue on iOS devices. No clue why; will try to fix it later
  • Possibly change hosting do to some of the restrictions on that current host. I'll assess that based on traffic

Things to add:

  • Panel for Top 10/Bottom 10 lists - I already have the functions to do that and it's easy for anyone to check on their own with the table, but it would still be nice to have a dedicated panel to quickly look at a wall of fame/shame to get context for particularly big or rough games
  • "Similar players" tool for trying to get an idea for who is grading out similar to players past - good and bad.
  • Add ability to adjust for # of plays in a game. Would be even better to adjust for playing time per person, but that might be ambitious. 

I've started sharing a few of the plots in UFR threads in order to finally get the mgopoints to be able to post (Been a lurker since forever...). Some examples are below. The x-axis labels should be more legible on the web app:

Morgan_bounceback

Cole v Lewan

I hope the community likes it, please give feedback! Feel free to "Save Image As..." and share images at your leisure, though obviously I'd appreciate the source being mentioned.

  • mikegros's blog
  • Login or register to post comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
November 12th, 2015 at 5:18 PM
#2
kevin holt
kevin holt's picture
Joined: 03/16/2010
MGoPoints: 16959
This is awesome. Can't wait

This is awesome. Can't wait to play around with it!

Top
  • Login or register to post comments
November 12th, 2015 at 7:07 PM
(Reply to #2) #3
carolina blue
Joined: 02/12/2014
MGoPoints: 10823
Twss

Twss




Sent from MGoBlog HD for iPhone & iPad

Top
  • Login or register to post comments
November 12th, 2015 at 5:32 PM
#4
markp
markp's picture
Joined: 12/24/2013
MGoPoints: 3426
Noice!

Noice!

Top
  • Login or register to post comments
November 12th, 2015 at 5:33 PM
#5
Spork
Spork's picture
Joined: 10/06/2015
MGoPoints: 238
Straight

Baller

Top
  • Login or register to post comments
November 12th, 2015 at 5:39 PM
#6
BiSB
BiSB's picture
Joined: 08/15/2009
MGoPoints: 44839
This

is fantastic.

Top
  • Login or register to post comments
November 12th, 2015 at 6:01 PM
#7
youn2948
Joined: 05/28/2010
MGoPoints: 3749
Game Blouses

Game Blouses

Top
  • Login or register to post comments
November 12th, 2015 at 6:11 PM
#8
SoDak Blues
Joined: 09/24/2015
MGoPoints: 15844
This is really cool. Thanks

This is really cool. Thanks for the work! 

Top
  • Login or register to post comments
November 12th, 2015 at 7:35 PM
#9
Woodstock Wolverine
Woodstock Wolverine's picture
Joined: 07/16/2015
MGoPoints: 10584
So, you have a lot of time on

So, you have a lot of time on your hands. must be nice. Anyways this is pretty cool stuff and thanks for your time!

Top
  • Login or register to post comments
November 12th, 2015 at 8:06 PM
(Reply to #9) #10
mikegros
mikegros's picture
Joined: 01/06/2011
MGoPoints: 175
Not really....

... but I could justify it to myself as "professional development" as it's good to practice web scraping and data cleaning; plus it was a good toy to use for learning RShiny. 

It's also just fun data to look at. 

Top
  • Login or register to post comments
November 12th, 2015 at 8:24 PM
(Reply to #10) #11
LBSS
LBSS's picture
Joined: 11/21/2009
MGoPoints: 6655
justify it however you want

justify it however you want man, this is awesome.

Top
  • Login or register to post comments
November 12th, 2015 at 11:35 PM
(Reply to #12) #12
IncrediblySTIFF
Joined: 03/26/2011
MGoPoints: 7182
he's got good reasons to justify it, even if

he's got good reasons to justify it, even if rshimy is just another

Top
  • Login or register to post comments
November 13th, 2015 at 8:25 AM
(Reply to #10) #13
VikingDiet
Joined: 09/02/2014
MGoPoints: 634
This is great and all...

But can you port it to ArnoldC?

Top
  • Login or register to post comments
November 13th, 2015 at 2:20 PM
(Reply to #10) #14
Noleverine
Joined: 10/06/2010
MGoPoints: 3735
People should remember to

People should remember to upvote his comments so he gets "point cred" for his work. If I remember correctly, OP votes don't count.

Top
  • Login or register to post comments
November 12th, 2015 at 8:10 PM
#15
jmblue
Joined: 11/07/2008
MGoPoints: 56299
This is very cool and

This is very cool and informative - definitely worth checking out.

(Also, kudos to you for earning your Mgopoints the old-fashioned way, instead of just visiting a posbang thread and getting 100 points in an hour.)

 

Top
  • Login or register to post comments
November 12th, 2015 at 11:11 PM
(Reply to #11) #16
justingoblue
justingoblue's picture
Joined: 11/16/2010
MGoPoints: 29685
Old fashioned way.

If someone has a solid, diary level contribution I think all of the mods are on board with just dumping some points into an account. To be clear this is an exception and people should post normally to acquire the ability to post, but if you're going to bring something unique to the table immediately (like this) comment in the mod sticky or get our attention on the board or tweet LSA and we'll make it happen.

Top
  • Login or register to post comments
November 12th, 2015 at 10:47 PM
#17
Cranky Dave
Cranky Dave's picture
Joined: 09/08/2012
MGoPoints: 17741
I'm very

Impressed and now know what a scraping is...sort of

Top
  • Login or register to post comments
November 13th, 2015 at 12:40 AM
#18
UMForLife
Joined: 11/04/2013
MGoPoints: 16031
MGoDifference right here my

MGoDifference right here my friend. Awesome work.

Top
  • Login or register to post comments
November 13th, 2015 at 1:01 AM
#19
EGD
EGD's picture
Joined: 09/16/2009
MGoPoints: 26584
UFR charts

I was impressed with these when you first put them out a few weeks ago and was really looking forward to seeing more. These are a great contribution, so thanks for doing this and by all means keep them coming!

Top
  • Login or register to post comments
November 13th, 2015 at 6:02 AM
#20
Other Andrew
Other Andrew's picture
Joined: 06/30/2008
MGoPoints: 1854
Gaps in early years?

Fantastic stuff. One question - looking up older players brought mixed resutls. e.g. Lamarr Woodley and Leon Hall have full data for 2006. But Mike Hart has only two games showing up. Are there some gaps in the earlier data?

 

Thanks a ton for putting this together. Really good!

Top
  • Login or register to post comments
November 13th, 2015 at 11:47 AM
(Reply to #19) #21
mikegros
mikegros's picture
Joined: 01/06/2011
MGoPoints: 175
Brian didnt....

.... chart +/- for offense for a long stretch. He only made the Hennechart and then at some point started the receiver chart before adding the +/-.

Top
  • Login or register to post comments
November 13th, 2015 at 9:00 AM
#22
jordraph
Joined: 01/31/2011
MGoPoints: 162
Most Impressive

Seeing this information graphically is wonderful. Performance trends are now so easily digestible. Many thanks!

Top
  • Login or register to post comments
November 13th, 2015 at 9:03 AM
#23
Yo_Blue
Yo_Blue's picture
Joined: 08/24/2010
MGoPoints: 83181
Love the "Compare Two Players" mode

Take a look at Koger vs Butt.  You may be surprised.

Top
  • Login or register to post comments
November 13th, 2015 at 12:09 PM
#24
wolfman81
wolfman81's picture
Joined: 12/17/2008
MGoPoints: 1337
Open Source

FTW...

Top
  • Login or register to post comments
November 13th, 2015 at 10:19 PM
(Reply to #24) #25
Qmich
Qmich's picture
Joined: 03/30/2011
MGoPoints: 192
for sure

posted a comment in the frontpage diary thread, but yeah I think there are lots of people that would love to collaborate on this.  Any way we can contact grozzy to lobby for this?

Top
  • Login or register to post comments
November 13th, 2015 at 11:14 PM
(Reply to #32) #26
mikegros
mikegros's picture
Joined: 01/06/2011
MGoPoints: 175
Consider your case lobbied for....

I responded in the other thread too, but sounds great. I will get on it soon (when I can scrape up the time). 

Thanks for the idea!

Top
  • Login or register to post comments
November 20th, 2015 at 2:14 PM
(Reply to #32) #27
mikegros
mikegros's picture
Joined: 01/06/2011
MGoPoints: 175
Code on github

Here: https://github.com/mikegros/ufr_analysis/

Have at it!

Top
  • Login or register to post comments
November 13th, 2015 at 3:16 PM
#28
DualThreat
DualThreat's picture
Joined: 05/04/2012
MGoPoints: 2297
I vote this "Diary of the Year"

For the usefulness, yet elegent simplicity of the tool.

Well done.

Top
  • Login or register to post comments
November 13th, 2015 at 4:18 PM
#29
MGoMatt30
MGoMatt30's picture
Joined: 11/12/2009
MGoPoints: 659
Cool!

It's awesome to be able to sort the game / season / career numbers for each player.

Lewis looks like he will end up with double the points of anyone else for career points by the end of this year and Peppers looks like he will blow that number out of the water by the end of his!  Progress!

Top
  • Login or register to post comments
November 13th, 2015 at 5:14 PM
#30
Minus The Houma
Joined: 09/04/2011
MGoPoints: 2251
I pretty much went through

I pretty much went through and gave everyone who posted a plus one. Even if I banged alone I think your work was worth my effort. Really cool stuff. Thank you.

Top
  • Login or register to post comments
November 13th, 2015 at 8:42 PM
#31
Unsalted
Unsalted's picture
Joined: 09/26/2014
MGoPoints: 1828
Cool stuff!

This is the coolest thing since the UFR viewer, may it RIP.

Just for fun I looked at Braden's numbers. As Brian wrote, he just isn't the same player we saw against Utah or in 2014 for that matter. Braden has transitioned from OL most likely to be replaced to pretty damn good. Looking forward to next years numbers. BTW I thought his block on the LB on Pepper's TD was outstanding, maybe worth a +2 IMO.

Thanks!!!!

Top
  • Login or register to post comments
November 13th, 2015 at 9:39 PM
#32
tsbilly
Joined: 08/14/2013
MGoPoints: 194
This is...

This is awesome!

 

Top
  • Login or register to post comments
November 13th, 2015 at 9:42 PM
#33
tsbilly
Joined: 08/14/2013
MGoPoints: 194
Charles Woodson

Does anyone want to go back and analyze Woodson's entire career so it can be added to the visualizer? I want to view awesomeness in graphical format.

Top
  • Login or register to post comments
November 14th, 2015 at 12:48 AM
#34
Chops
Chops's picture
Joined: 01/13/2015
MGoPoints: 577
Nice work!

I've been working on some similar stuff using d3js and node. I got a nice scraper working to pull all the player data into firebase. Hopefully I'll have something interesting to show soon.

Here's an example: Michigan Top 20 Rushers by Net Yards

Top
  • Login or register to post comments
November 14th, 2015 at 1:18 AM
(Reply to #34) #35
justingoblue
justingoblue's picture
Joined: 11/16/2010
MGoPoints: 29685
Made the type of exception I

Made the type of exception I was talking about above. Took a look at what you've got and I'd encourage you to post a diary with unique numbers and 200 words when you have something interesting to show.

Top
  • Login or register to post comments
November 14th, 2015 at 3:34 AM
(Reply to #34) #36
mikegros
mikegros's picture
Joined: 01/06/2011
MGoPoints: 175
Really nice!

I'd love to see more.

Top
  • Login or register to post comments
November 14th, 2015 at 4:42 PM
(Reply to #36) #37
Chops
Chops's picture
Joined: 01/13/2015
MGoPoints: 577
Workin' on it

Grozzy, I've got my code version controlled in git and spent some time dockerizing the node app this week so hopefully I can easily deploy it to a host in the future. Plus it was an excuse to learn more about how docker works.

Perhaps we should connect on github. My github id is twebb. Let me know what your github id is and if I get the repo up there I'll give you access.

Next step is to turn the d3js rushing visualization code into a working page in the node app (currently it's just a scraper) and connect it up to the firebase data. 

Top
  • Login or register to post comments
November 20th, 2015 at 2:13 PM
(Reply to #37) #38
mikegros
mikegros's picture
Joined: 01/06/2011
MGoPoints: 175
My code is up on github

Here: https://github.com/mikegros/ufr_analysis/

I recently "followed" you on github. I don't have a ton of extra time right now, but I would love to connect and really put together some good work as much as I'm able.

Take care!

Top
  • Login or register to post comments
December 3rd, 2015 at 1:58 AM
(Reply to #38) #39
Chops
Chops's picture
Joined: 01/13/2015
MGoPoints: 577
First Diary Post Up

First Post is Up: Top 200 Rushers Data Visualization

I've added you to the repo so you can see the code. Can't wait to play with all the data I scraped. Took quite a while to get it all in the form that I wanted it. I might try doing something with the Passing stats next.

Top
  • Login or register to post comments
Powered by Drupal, an open source content management system
Theme provided by Roopletheme; sidebars adapted from Chris Murphy.