The PS3 is capable of passing both TrueDolbyHD and DTSMasterHD unproccessed thru the HDMI 1.3 to a HDMI 1.3 receiver that can do the decoding in the AVR. Hence the Denon's 3800 series to the rescue. The Denon can also accept a processed signal from either codec too. I am telling you these new Denons are absolutely going to rock!!!
Where to start. First of all, I agree with you about Denon products. I have an old 3200 that I use in the bedroom since all that I need is the optical connection for audio and a 4806 I use in the family room hooked up to the panasonic. Second, I'm not saying the PS3 is great, nor am I saying it's junk. It is a good BD player and is very fexible with its Cell processor.
That being said, you do have some misunderstandings in your post. Let's Start with DTS-HD Master Audio. DTS-HD and DTS-HD MA are two separate codecs. DTS-HD is similar to its Dolby counterpart DD+. Both are very nice improvements over their predecessors but still lossy. The PS3 does not support Master audio at this time. It is expected by users but has not been announced. If it does get added that would be great for the PS3. I honestly hope it does. It would be the only player that could properly handle Fox titles like X-men. Why they would add a DTS HD master track when it's not supported and not uncompressed 5.1 I don't know. Oh yeah, it's a 25 Gb disc.
Next is HDMI 1.3, You are correct in saying that HDMI 1.3 can pass Dolby TrueHD and DTS-HD MA. You have a misunderstanding on how the PS3 handles it. Let's start with advanced authoring of high definition discs. Advanced authoring requires that both formats do the decoding within the players. This is written onto the discs. This was done purposely since no receivers currently handle these advanced codecs. However, they do handle LPCM since this has been used for quite some time. Whether these alogorithms are authored onto the discs or not in the future, we'll have to wait and see. My bet is exactly what Dolby Labs said, some will, but only a minority.
According to the PS3 manual, AVS, etc. this is how the PS3 handles audio. It supports DD, DTS, 7.1 channel LPCM, Dolby TrueHD, and DTS-HD. If the master audio track on the disc is uncompressed 5.1 or 7.1, LPCM will take that bit for bit to the receiver for post processing. As for Dolby TrueHD and DTS-HD, the player does the decoding, and sends it out over PCM to the receiver for post-processing. Straight from the owner's manual. Again it will depend on the discs authoring whether or not in the future, receivers will have access to the raw HD bitstream.
*1 A device compatible with Linear PCM 7.1 Ch. is required to output 7.1 Ch. audio, supported by Dolby TrueHD or a similar format, from the HDMI OUT connector.
*2 This system does not support output from the DTS-HD 7.1 Ch. DTS-HD 7.1 Ch. audio is output from a 5.1 or lower channel.
As Dolby said, the sound quality will be essentially the same. When these new receivers come out, we'll talk more.
Enjoy your player.
S~