Roku has limits to the media formats it supports naively. The Plex channel knows what these formats are and will request the plex server to transcode the video on the fly to the best format supported on the Roku.
The quality of the image you see depends on the native resolution of your display, and the native resolution of your source material, as well as the native format of your source material.
If transcoding, the CPU utilization of your computer will be high. In addition, when using Plex, your PC needs to encode the image and stream it to the roku. There is also the data going to the roku over your network connection and your wifi. All of that can affect the quality of your viewing.
Here are the formats natively supported:
http://support.roku.com/entries/423...oes-the-roku-usb-media-player-channel-support
Note... I just did a test with some content that was pulled off of a site. So a music video in flash video format that would show up on youtube in 480p (1085Kbps, 854*480 (16:9) at 24 fps) to my 1080p DLP television. Image is soft overall with a lack of detail you would expect in HD output. This not HD, but the image is passable with no real digital artifacts except some blockyness in dark areas. To be expected with a low resolution video.
Compare to MPEG-4 video file (mp4) which would show up in youtube as 1080 (5974Kbps, 1920x1080 (16:9) at 23.976fps) to the same tv. Now this one I see fires up the Plex Transcoder which is spiking about 80 to 90% CPU utilization (appears multi threaded) on my older laptop (Intel core 2 Duo P8600 @ 2.4GHz) This has more detail, but is more like Dish HD on a bad day. Facial colors are mottled but there is more detail (i.e. sweat on a body, patterns in lace). This may be due to lack of some optimization the the transcoder. If I get more time I may play with that. I.e. I just downloaded a H.265 trailer for Superman, man of steel. Transcoder fired up. That is a format that the roku knows, so I'm not sure why it feels a need to transcode it.
p.s. Haven't had anything crash yet.
p.p.s. Plex has a forum with information, i.e. this post
http://forums.plexapp.com/index.php/topic/46386-roku-stuttering-even-mkv-and-h264/. Oh and you can change the plex channel preferences so it will ask for higher quality. By default it only is 3mbps 720p. I bumped it up to 8mbps 1080p and tested again. The 480p video looks about the same. the 1080 video looks so much better. I never new the config was on the roku channel, and not my pc. I have a 127Mbit wireless connection between my plex server and the roku. it's handling the 8mbit fine and it did not fire up the transcoder this time. This also resolve the issue with the H.264 trailer I downloaded for Superman. Looks better.