OT: the effect of firing coaches

Submitted by dnak438 on

Via the Daily Camera (the study is here):

two University of Colorado political science professors say statistical analysis indicates firing a coach for poor team performance is far from a surefire way to turn things around, and, in some cases, may actually harm a team's future performance.

 

Looking at results for four years after a coaching replacement, the study concluded bringing in a new coach, on average, had a negligible effect on a team's win-loss record.

"I had always watched these teams fire coaches, pay for a buyout and then hire more expensive coaches and I wondered, 'Are they actually getting anything out of this?'" said Adler, a University of Michigan alumnus and college football fan. "What we find is, as you go out to the fourth year, the difference between teams that did and didn't replace their coaches were just nonexistent. They were performing just about the same."

How a Michigan alumnus and fan could write a study concluding this is beyond me.

Seriously, though, Michigan is clearly an outlier -- we are a premiere program. I think this study is relevant to teams like Minnesota, who fire good coaches (Glen Mason) thinking that they are capable of being more than they are. But the new coach can't improve the facilities, can't change the amount of local football talent, etc.

woomba

November 15th, 2012 at 12:11 PM ^

Survivorship bias - they probably don't factor incoaches that were replaced before they were 4 years in so they will be analyzing 'good enough' coaches by definition

1464

November 15th, 2012 at 1:27 PM ^

Also, when presented with the same set of data, this study does not account for extenuating circumstances.  Example:

 

In year four of their contracts there were 3 coaches who went 2-10.

Coach 1 had a 48-3 record as a coach prior to taking this position.  The team was coming out of crippling sanctions.  The AD decides to keep him.

Coach 2 had a 14-26 record as a head coach coming in, and was replacing another crappy coach.  He got axed as his background did not suggest he could turn it around.

Coach 3 was an assistant to a legend.  He had no head coaching experience prior to taking over after the legend retired.  It's obvious he was in over his head, and was let go.

 

Now... coach 1 turns it around.  The other two schools are floundering with the aftermath of their coaching carousel (sp?).  This study is flawed as it just looks at numbers, and cannot evaluate the outlying conditions.  Not sure how so many people struggle with this kind of stuff...

david from wyoming

November 15th, 2012 at 2:25 PM ^

The paper makes it very clear that they do focus only on wins and loses and makes the point that, oh hell, let me just blockquote.

Though there are other goals that can also be part of the performance objectives in the sports enterprise—for college athletics, these might be alumnae contributions, media attention for the university, or the ability to recruit higher quality student-athletes—these are generally considered ancillary to the primary objective of team performance.

The study is not flawed at all. Now sure how so many people can't read the methodically before coming to conclusions that the authors don't try to make.

1464

November 15th, 2012 at 4:33 PM ^

Your ability to grasp my point is pretty weak, to the point where you are using my point to disprove my point.

The coaches that do all that ancillary stuff better, and have a better track record, will generally be kept on staff, while the less talented coaches, ancillarily, will be cut.  Past success, coupled with off the field wins, is a predictor for both job retention and future success.  Do you really not see how only examining some of the data would produce bad results?

david from wyoming

November 15th, 2012 at 12:17 PM ^

How a Michigan alumnus and fan could write a study concluding this is beyond me.

I'm guess that is the conclusion because that is what the data shows.

Yes, I know I'm answering sarcasm with sarcasm.

raleighwood

November 15th, 2012 at 12:18 PM ^

I live down the road from NC State and they are going through this particular situation now.  Tom O'Brien has been there for six years and he's pretty consistently been in the 8-9 win range.  NC State has a habit of beating good teams (Clemson in 2011, FSU in 2012) but losing to bad teams (Virginia in 2012).

I've always thought....."Who do you think you are?"  Not every program can be in the Top 25.  I think that programs hate the status quo and all think that they can be better than they are.  That's true to a large extent but most team have ceilings.  We've seen the rise of teams like Kansas State, Oklahoma State and Stanford in recent years.  I think that those teams are more of the exception than the rule....and I'm not convinced that they can remain at those levels.

In the case of NC State, I don't think that they'll benefit from firing O'Brien (they'll most likely still be in the 8-9 win range regardless of the coach is).  Now, if a team is consistently in the 4-5 win range, they probably do need to look at coaching options.  That's a status quo that you really can't accept.

swan flu

November 15th, 2012 at 12:27 PM ^

This is just bad science. It is impossible to scientifically determine the effects of firing coaches because you have no control group.

Never ever ever trust pop science bullshit like this.

Edit: to clarify, I'm not saying that a statistical analysis has o value, but you cannot draw distinct and specific conclusions from them.

david from wyoming

November 15th, 2012 at 12:33 PM ^

Well, this is peer reviewed, so in this field, it is about the best science possible. Based on the impact factor of the journal, I would trust the peer review process more than enough to trust the conclusions. I just skimmed the paper because I don't feel like working on my own science right now...

Any science has to make assumptions to be able to draw conclusions. They have a clear control group of teams that don't fire their coach under the same conditions.

Please note that the conclusions in the science paper are slightly different in the press article.

swan flu

November 15th, 2012 at 8:00 PM ^

When I say there is no control group I mean you cannot effectively reduce all confounding variables to the point of being negligible. Michigan replacing rich Rodriguez is not comparable to Tennessee replacing Phillip Fulmer is not comparable to USC replacing Pete Carrol is not comparable to Minnesota replacing Tim Brewster.

It is not actual science because you cannot isolate the variables, all you can do is analyze the data using statistics, which is good if you know what you are doing, but I guarantee that the sample sizes and quality of the linear regression are not good enough to warrant the conclusions that were made.

I get that this is a neat article with an interesting topic, but I would never in a million years agree that this study should, in any way, influence the firing and hiring of coaches.

profitgoblue

November 15th, 2012 at 12:30 PM ^

I don't think the guy is wrong with respect to Michigan yet - he states that the zero improvement result is after year 4.  Hoke is obviously only in year 2 at this point.  Regardless, interesting articles are interesting.

 

Don

November 15th, 2012 at 12:42 PM ^

their conclusion is probably borne out by their data.

However, to go from that to the belief that firing a lousy coach is guaranteed NOT to result in any improvement is idiotic, for the obvious reason that the history of college and pro football is filled with hundreds of examples showing dramatic and long-lasting improvement under a new coach.

From 1961 through 1978, Iowa had Jerry Burns, Ray Nagle, Frank Lauterbur, and Bob Commings as HC, with all of them having dreadful records. Using Adler's logic, Iowa made a dumb decision to fire Commings and hire Hayden Fry for the 1979 season.

david from wyoming

November 15th, 2012 at 12:49 PM ^

Whoa whoa, there is a huge difference in saying 'on average the data shows no results' and 'in this specific case, there was a change'. There point, using all data possible, is only to say that improvements are, on average, not going to happen.

To say it another way, if ten coach were fired this offseason, for every improvement based on the replacement at one school, there is on average going to be a team that gets worse. The net effect would be that those ten schools averaged together didn't improve.

French West Indian

November 15th, 2012 at 12:47 PM ^

...for one thing, firing coaches is not always a matter of won/loss record.  Although some alumni and especially many casual "fans" freak out over game results, the coach of a university's football team does have a role beyond the games as a mentor & role model.

But I would agree that firing coaches halfway into big contracts and then hiring new ones is counterproductive in terms of money directly (spent on coach salaries) and indirectly (lack of program stability probably does more harm to fundraising than any bump that might be derived from an exciting new hire).

It's early yet, and the jury is still out...but hopefully longterm the effect of dismissing Coach Rodriquez won't prove to be too disasterous.

jmblue

November 15th, 2012 at 1:05 PM ^

The problem with this study is that it's going to be unduly influenced by the fact that terrible programs, like Indana, are regularly firing coaches whereas the stronger programs (like Michigan) very rarely do so.  So you have lots of coaching changes at the nearly hopeless schools that distort the total picture.  If you focus on schools that are historically average or above average, the picture may be very different. 

I don't think Michigan regrets its most recent firing (Rodriguez).  Nor does MSU regret its most recent firing (JLS), nor does Wisconsin (Don Morton - the coach before Alvarez), Northwestern (the guy before Gary Barnett - I can't remember his name) or Iowa (Bob Commings - the coach before Hayden Fry).  LSU is probably okay with having fired Gerry DiNardo (the coach there before Saban), ND is probably fine with Charlie Weis being gone, and I imagine no one at Alabama is pining for Mike Shula.

 

dnak438

November 15th, 2012 at 5:10 PM ^

here (link), under the sections called "stratified matching" "nearest neighbor matching" and "weighted analysis."

The study does not claim to have valid results for all kinds of programs. Specifically, they claim that:

We find that for particularly poorly performing teams, coach replacements have little effect on team performance as measured against comparable teams that did not replace their coach. However, for teams with middling records—that is, teams where entry conditions for a new coach appear to be more favorable—replacing the head coach appears to result in worse performance over subsequent years than comparable teams who retained their coach.

 

 
 

aratman

November 15th, 2012 at 1:53 PM ^

Most teams are mediocre and they fire coaches to help but bring in another mediocre coach.  Do you think Urb would have come out of family time to coach Indiana or Saban would have left LSU for Iowa State?  Everyone tends to rise to their level of incompetence. When your team is a .500 for fifty years it is because your team is a .500 operation.  If a coach comes in and goes .750 he will be leaving to some place were he can go .500 until he gets fired.

Moleskyn

November 15th, 2012 at 2:58 PM ^

So now I have some actual research to back up my assertion that firing Jim Leyland in the middle of this past season would not have accomplished much, if anything. Awesome!

wile_e8

November 15th, 2012 at 5:28 PM ^

Just because schools on average don't improve by firing a coach does not mean you should keep a bad coach.

On average, most schools don't have any improvement after firing coaches, and firing a coach if far from a gurantee of improvement, but some schools *do* improve after firing the coach. After a coach proves to be bad, keeping him will just gurantee a bad future. Firing him might make things worse, or it might have no effect, but it might make things better. Once you know you have a bad coach, I'd rather take a chance at getting a good coach at the risk of getting an even worse coach than just stick with a known bad coach. Never going to improve by keeping a bad coach.

snarling wolverine

November 15th, 2012 at 5:57 PM ^

And as Alabama, ND and (to a lesser extent) MSU have shown, it doesn't really matter if you cycle through coaches for awhile as long as you end up with the right one.  Those programs went through stretches of coaching instability, but then they hired their current guys and things turned around.

 

 

Don

November 15th, 2012 at 7:53 PM ^

athletic directors who have no idea what they're doing when they conduct a HC search. How anybody could have fired Glen Mason and think that Tim Brewster was going to be an improvement is still a mystery.