Missouri- adjacent to Big Ten territory, some rivalry connection, football/basketball competence, decent institution
Nebraska- See Missouri, except not basketball. Added bonus of Big Ten style football, traditional powerhouse
Notre Dame- Because you have to invite them at least, though I don't really want them anymore
Rutgers? No where near the traditional Big Ten midwest territory, not a traditional powerhouse in basketball or football, and not a stellar enough institution to raise the academic level of the conference. Rutgers doesn't have a current major sports following in New York, so why would it add more to the Big Ten? By that logic, the Big Ten should add Army too. Rutgers does not have the marketing pull that people seem to think it will have for the conference.
Call me old fashioned, but to me a conference should embody a certain geographical area, if possible, and Rutgers/Texas are fun ideas, but they just don't fit. The Big Ten is a midwest conference, so personally I am not comfortable going and farther south than Cincy or farther east than Syracuse or farther west than Nebraska. Add Pitt before you add a school like Rutgers.