Auburn went the NC game 2 years after starting the year unranked. If a team puts up a great season, they will be recognized. If multiple teams do it, than the ones who pad their schedules with cupcakes (Baylor and TCU) will be left out.
TCU actually plays a respectable non-conference schedule. Unlike Baylor, they aren't afraid to schedule competitive non-conference games. Off the top of my head I know they've played Virginia, LSU, and Oregon St. in recent years.
For some reason Baylor is hesitant to schedule those types of games. When people challenge them on it their excuse is always "These schedules were put together 5 years ago". However if you look at their future schedule they haven't made much of an effort to improve their SOS. I think they finally play a game against Duke in 2021 I believe, and that's their toughest non-conference game for the foreseeable future.
So if Baylor keeps getting passed over for playoff spots they'll have no one to blame but themselves.