Actually it was depth 24 on my regular X server that solved the issue there. Specifying depth 24 for Xvnc doesn't have the same effect.