The NCAA can issue a "show-cause" penalty. This means that the school needs to a) fire Tressel or b) appear before the Committee on Infractions and show a cause to keep employing him. Basically they would need to make the case that he really isn't all that bad.
Nobody has ever gone through the appeal process, and even Tressel would probably have the dignity to resign before it got to that point. So yes, effectively the NCAA can end Tressel's (or any coach's) career.
Edited to add that this is the standard punishment for a 10.1 violation, the one the NCAA charged Tressel with today, and the one he admitted at the initial press conference.