Sorry to join this thread late. Looks like the debugging is going well.
Couple of points, though . . .
Any RCA cable should pass the digital signal over short distance.
The digital audio on the orange connector just uses coax (as you'd use for video), not junk shielded cable like often used for R/L audio.
So, if you had a yellow video cable, that should work for a test.
Does any signal on non - AC3 channels come through on your SPDIF (that's optical) cable?
You should get regular sound on it all the time.
If you do, and don't get AC3, that is a clue.
At least explore this matter and discuss it.
(I'd have another look through the Pansat menus for a feature or mode that controls the output, as Turbosat suggested)
Are there any options on your Sony receiver for processing the optical signal?
How does it react or display the status of AC3 when hooked to your working Coolsat FTA receiver/?