Analyzing the Validity of the Hennechart
After reading Coach Schiano’s excellent diary on visualizing The Hennechart, I started to wonder about the validity of the HenneChart (damn you experimental psychology grad program!). Basically, does the HenneChart measure what it’s intended to – QB performance. So, I compared the HenneChart with 2 established metrics of performance: Completion percentage and NCAA QB Rating (not to be mistaken with NFL QB Rating).
To do this, I played around with assigning different weights to the Hennechart variables as well as which variables to include. Then, I assigned each variable into a Good category and a Bad category. From there, I took the percentage of points that fall into the Good category [ = Good points / (Good points + Bad points)].
As I mentioned, I manipulated the weights and which variables to include. I tried the following:
- Limited-1pt System: Good = (1 * DO) + (1 * CA). Bad = (1* IN) + (1 * BR)
- All-1pt System: Good = (1 * DO) + (1 * CA) + (1 * MA). Bad = (1* IN) + (1 * BR) + (1*TA) + (1 * PR)
- Limited-multi pt System: Good = (2 * DO) + (1 * CA). Bad = (1* IN) + (2 * BR)
- All-multi pt System: Good = (3 * DO) + (2 * CA) + (1 * MA). Bad = (2* IN) + (2 * BR) + (1*TA) + (1 * PR)
- 0 false 18 pt 18 pt 0 0 false false false /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-ascii-font-family:Cambria; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Cambria; mso-hansi-theme-font:minor-latin;} Modified 1pt System: Good = (1 * DO) + (1 * CA). Bad = (1* IN) + (1 * BR) + (1*TA) + (1 * PR)
From there, I ran a series of bivariate correlations. Of course, this is limiting given our low sample size (5 games). I don’t have the time to do previous seasons (maybe this weekend…).
Here is a table comparing these 5 systems of analyzing the HenneChart. Correlation r-values are provided with p-values (statistical significance) in parentheses.
For those that are interested, Completion % and QB Rating is r(3) = .56, p = .33.
As you can see, using the HenneChart with the Modified 1pt System (excluding MA throws) appears to offer the best validity. Now, this is not perfect. There may be yet a better way of assigning weights, deciding which variables to use, etc. Any advice is welcome. So, this would suggest that Coach Schiano's visual illustration of the HenneChart should have Marginal throws as not appearing in Blue or Red; Brian's suggestion of using MA as a zero-point is a good one.
October 9th, 2009 at 6:16 PM ^
October 9th, 2009 at 6:20 PM ^
October 9th, 2009 at 6:32 PM ^
October 9th, 2009 at 7:06 PM ^
October 9th, 2009 at 9:57 PM ^
October 10th, 2009 at 8:01 AM ^
October 10th, 2009 at 9:53 AM ^
October 10th, 2009 at 11:41 AM ^
October 10th, 2009 at 7:04 PM ^
Comments