I do prefer the solution by thomasnabgelis, but other working solution would be to use lightdm display manager. To do this just runsudo apt-get install lightdmandsudo dpkg-reconfigure lightdmif not configured after install.
Why this happens? I spent a lot of time trying to understand... nvidia-drm provides anti-tearing solution, though I didn't find any difference with this turning that off. Citation from nvidia forum (topic):
nvidia-drm modeset=1 is needed for tear-free display but since this is enabling linux kms for the nvidia driver, gdm thinks it can use wayland. Having a parallel wayland session makes it somehow impossible for X to detect any outputs on the nvidia gpu
