This is exactly it. This is why defensive players don't win it (and a defensive player who only plays defense has never won it). You just don't have that big of an effect on your team as a defensive player unless you're just crazy. Te'o is not crazy, he's just a great linebacker. One of the best college linebackers in a while, for sure.
He has 7 picks (tied for second in the country) for a total of 35 yards, and has never been in the endzone. He has 1.5 sacks. Ed Reynolds, the safety from Stanford (on an equally good defense, and nearly as successful team) has 6 picks for 301 yards and 3 TDs. Where is his Heisman love? A kid from Fresno State has 8 picks, 3 TDs and 4 sacks (!) which are all better than Te'o.
You need to be dynamic to win the Heisman, you need to put the team on your shoulders. I don't think Te'o has done that. I can't see someone winning the Heisman without ever sniffing the endzone.