For me personally Mattison gets a pass, because he's proven what a great coach and recruiter he is. Mostly the coaching is what I care about, and he took an absolutely dreadful unit with basically all the same players and turned them around immediately into a top-25 defense. The defense has continued to be solid for his tenure at Michigan.
I think the reason many people are frustrated with Borges is that the offense has only gotten worse and worse since he's been here. It's not one bad game, its a bad pattern and doesn't show any signs of changing.