Referee Statistics

Submitted by AC1997 on

This week Michigan's basketball team saw two extremes of Big Ten officiating.  Against Minnesota the TV Teddy Valentine crew set a record for fouls and free throws against a Beilein coached team at Michigan.  Then against Rutgers the Gene Steratore crew hardly called any fouls and generally let both teams play - resulting in long stretches without stoppage, no foul trouble for anyone on either team, and fewer foul shots.  The contrast in the way both games were called was alarming to me.

I set out to search for any sites that track referee data.  I didn't have much luck, so I'd love for anyone to comment if they know of a good site.  I did find this interesting piece (LINK) from two years ago that ranked the best and worst officials using foul margin between home/road teams.  It is an interesting read, though I'd love to see it updated. 

Interesting findings in that post:

  • TV Teddy makes no appearance...much to my surprise.
  • Gene Steratore was ranked the 6th best ref of those evaluated.
  • John Gaffney from Valentine's crew was ranked the 5th worst ref.
  • Lamar Simpson from Steratore's crew was ranked the 7th worst ref.

Given that we track so much in-depth data and statistics these days, you'd think there would be better data out there for referees - if noting else just for gambling purposes, let alone so conferences like the Big Ten could actually measure their performance and make decisions accordingly.  

MGoBender

February 23rd, 2017 at 2:43 PM ^

It should be noted that NCAA basketball officials don't have "crews" that they work with for every game. So, while John Gaffney might have worked on Steratore's crew one game, that doesn't mean he always does.

Also, there used to be a full database of NCAA basketball referees online somewhere, but I cannot find it after 5 minutes of googling.

While Home/away team ratio is the method used to judge officials in the above link, I don't know that it is the best (though valid) metric.  Unfortunately, it would be one of the few easily gathered metrics.  To really judge officials, you need to evaluate each call (and no-call!) on an individual basis as "correct" or "incorrect" which requires an intensive, subjective review process.  Officials' assignors do this, but there's no way they'd make that info public.

AC1997

February 23rd, 2017 at 2:50 PM ^

Your point about "crews" is of course accurate.  I was using the term more to identify who was on each crew for that game and reference the one recognizable name from each crew.  Valentine, despite his 30 years of terrorizing fans, wasn't even the lead official for that game.  The fact that they mix up refs so much is another interesting point of discussion.  I know if football games there are concerns when they break up "crews" for the playoff games in favor of an "all star team" of refs that aren't used to each other.

 

As for data, I agree that a comprehensive analysis of every play would be ideal...if impossible.  Not ony would you have to rate each call they made (fouls, travels, out of bounds, etc.) but you'd have to rate every non-call.  While there are highly complex tools/sites/techniques used for baseball that track events, doing it for refs seems nearly impossible.  If such data exists, it would be nice at least to share some summary of it for transparency - but I agree it won't happen.

 

Instead, I do think there are things that could be compiled based on the game stats.  The link I shared looked at fouls for home/road teams.  You could also look at trends for the two teams in question looking for outliers.  You could track things like game time, total fouls, fouls versus margin of score, etc.  I suspect you could also whether the foul differential changed throughout the game as one of the refs lookedat the scoreboard and was getting yelled at by the coach to make up the difference.  

charblue.

February 23rd, 2017 at 3:13 PM ^

effectiveness of a referee as a crew leader as opposed to just being a working official.

I mean Rutgers fans might have a different take on the calls last night than we do as Michigan fans. I mean clearly the crew that worked the Minnesota game in which Valentine was not the crew leader and injected himself by overtly responding to certain calls,(a bench technical, and a rather innocuously stupid call on Muhammad Ali-Abdul Rahkman touching a Minnesota player as he watched a ball deflect off him and then roll out of bounds) were hard to accept.

Steratore had an issue last night with a ball deflection call with uncertain posssession, and quickly brought in his partners for consultation before awarding the ball to Michigan, something he would regularly do as a football referee, and understanding from an NFL perspectove, that the referee is to be seen as a final arbiter not the center of attention. That is how Valentine's work is largely perceived as different and personally-focused. It's just a matter of his demeanor. It doesn't mean he is a bad official.

The question of consistency in fouls called against certain teams is more easily judged in football than basketball because of the differece in college crew assigning in both sports. In basketball, officials rarely find themselves working together more than two or three games per season. while in football, crews work full seasons as a team.

That said, you can measure how individuals call games just by the number of calls they typically make in a given game. And that stat is kept and known to coaches because they  typically want to know who is working their games in advance so they can plan accordingly.

AC1997

February 23rd, 2017 at 3:50 PM ^

In my opinion, the crew chief factor is largely irrelevant.  Unless the refs talk before the game or at halftime about how they plan to call the game and that falls on the chielf, then I think most calls are heat-of-the-moment judgments that anyone on the crew makes.  

 

I do like your comment about how Steratore seems to be more collabrative than Valentine and agree that is probably accurate.  I do think that's a better approach on tough calls and wish more refs did it.  

 

Where I will disagree is about Valentine being a bad ref.  I think he knows the game and generally is probably pretty accurate with his calls overall.  But I think he falls into the trap that Bill Simmons has pointed out as the true "conspiracy" behind NBA refs - Valentine is very easily sucked into the emotions of the game.  The big home crowd, the coach that's getting on his nerves, the dramatic responses from players, the momentum of the way the game is going - he gets wrapped up in it and wants to be part of it....and that affects his performance.  

Coach Carr Camp

February 23rd, 2017 at 3:40 PM ^

There is a book called Scorecasting which takes a Freakonomics-esque look at sports. Ultimately they determined that the only significant factor in home field advantage is referees, and that this the largest in NCAA basketball. Its been a while since I read it but their analysis looked at what were considered "judgement" calls for home and away teams, and they found that there was always a statistically significant favor on those calls towards the home team.  

Jangalang

February 23rd, 2017 at 6:02 PM ^

were called more consistently. I find it ridiculous that coaches need to know who's crew is reffing the game to know what to expect. I believe that if Valentine's crew was calling last night's game against Rutgers there would have been more fouls called for both teams...which is BS.

I realize that blocking vs a charge is most times a coin flip but the other hand checking fouls, etc. need to called with more consistency between crews.

...and get off my lawn!!

smwilliams

February 23rd, 2017 at 7:09 PM ^

Maybe a few of us with some spirit and some time could put together a database? Maybe B1G only at first for basketball and football. I mean, TV Teddy stands out as obvious, but I find it fascinating that we have a site that can use advanced math to determine any number of advanced observations about individual players, but no public data on referees.

B-Nut-GoBlue

February 23rd, 2017 at 9:48 PM ^

I don't know but I'm still pissed about Sunday night and the lack of anyone else caring to admit how terrible an officiated game it was. If anything fans think we get calls "cuz we're Michigan" when I can honestly say the past decade has seen me almost want to think there's a bias against Michigan, regarding both big-time sports. Yes we've had our share of calls go our way like everyone else, but overall I think we've been on the wrong side of calls compared to our opponents.

Especially in basketball. I feel our style of play is partially to "blame" but I think that's bullshit; because we're not a banging, physical team, when we do touch someone it's an auto-foul the last 6 years. I hate it and it's an unhealthy rage that's developed!

drzoidburg

February 24th, 2017 at 2:41 AM ^

I'll always think of refs are pure scum that should be replaced by techology ASAP, and the 3 examples of that i'll never forget are 2002 korea vs italy, nba refs, and "the game" 2016

mvp

February 24th, 2017 at 5:34 PM ^

If you are a KenPom subscriber, there is an attempt to look at officiating there:

http://kenpom.com/officials.php

I'm not at all sure about the validity of the methodology, but it is explained as such:

 

The problem with this idea is that I have no way of actually grading officials. But officials do get graded by someone and I’m operating on the assumption that the best officials get assigned to the best games. That is surely not precisely true on a daily basis, but over time, if an official is regularly working the biggest games on the national stage, the folks that make such decisions probably think he’s doing a solid job. In order to assess the quality of a game, I’m using the Thrill Score from FanMatch, imperfect though it may be. But it would seem to capture the things that are important here: opponent quality and game competitiveness. The only other thing we need to account for is the officials’ schedule. The busiest officials can top 100 games in a season. I’m going to value quality over quantity, so I’d prefer not to reward officials for bulk scheduling. In that interest, I’m taking average Thrill Score over the top 50 games that an official works in a season.