In this day and age of short attention spans and the availability of other sports on tv/internet, the baseball season is just too long.
In the 70's college football was strictly controlled by the NCAA, as to how much could be televised, once that ended and the conferences and teams started making their own deals, it become a free for all. Football is fast paced, action packed and fits into the tv schedules much better than baseball. Football viewership really took off with the move to widescreen digital TV too. It also comes at a time of the year when most people are inside more.
The MLB games are slow paced and usually run long on time. I personally have other things to do than devote 4hrs a night to watch baseball games and that's saying something as I pretty much have no life and am home most of the time.