Apparently new-kernel-pkg does not work for grub2 config with xen entries (new kernel isn't added to it at all). So regenerate grub2 config using grub2-mkconfig.