My little brain had got confused by all this kernel modesetting stuff. The only kernel compiling I have done is to enable the framebuffer console (for 301, 412 and a Wary kernel or two).
This explained it for me http://nouveau.freedesktop.org/wiki/KernelModeSetting/
I looked at /var/log/messages and noticed the video=640x480 line which gave the game away.