Conan is doing better than Leno Did on the Tonight Show with younger male audiences (18-34), but the problem is that this audience is way too small to make up for the disenfranchisement of the rest of the Leno audience.
The 11:30 slot has way too much competition for the younger audiences so even though Conan is winning the battle in the plan to attact younger audiences to the network, he doesn't have the broad appeal (yet) needed to get a large overall audience.
Letterman is there for the older audiences. Many of the older audience that couldn't stomach Conan defected. This is one of the reasons that Letterman took over the #1 slot. It wasn't just because Conan lost audience. It was because letterman gained audience Conan lost!
Colbert is there for the political humor. You can see the difference in ratings for the Tonight Show when Colbert is in reruns or on Fridays.
And lets face it, much of the younger viewers have the internet as a viable entertainment stream. The largest competitor for yourng viewers is the internet, not other networks. Ask a 15 year-old what their favorite TV show is and you are likely to get an answer like: "TV?"
See ya
Tony