They are definitely warm. I recommend adding a set of spacers (RG6 connectors work good) behind the SWM16 to aide in cooling, keeping it about an inch off the mounted surface.
I just drill mine straight into siding or onto a panel board with screw clip screws, no spacers. Not that that is a bad idea at all, but Ive never done it. The only defective swm16 Ive ever pulled out of the field was one of the first generations with clear ports instead of orange, and it was a communications related issue. Not sure if heat related or not.