Not certain if I'd recommend a visit to CNN/Coke/Aquarium area this weekend. They are still cleaning up from the tornado down there although I do believe everything is open.
If you are into hiking and Confederate history, you might want to visit Stone Mountain. The view from the top is nice, and there is a laser show on the carving weekend nights. Unfortunately its become much more commercialized in recent years.
And if you are really into Civil War history, visit the Cyclorama. Its a huge 360 painting about the Battle of Atlanta. Pretty cool.
If you are into museums, the High Museum has an exhibit from the Louvre as well as a Georgia O'Keeffe exhibit currently. The Emory Carlos museum has a nice collection of Egyptian/Ancient World materials. Fernbank Museum is the natural history center with dinosaur displays, etc. None of these compare to larger museums found in places like New York, but they are a pleasant way to spend a few hours.
Atlanta has a small zoo that isn't much to talk about, but their gorilla and panda exhibits are excellent.
If you are going to be hitting several places, to save some money, get the City Pass which offers you cheaper admission to 5 major exhibits (Coke, Aquarium, CNN, Zoo, Coke)