I've been told that running GUI application with sudo is not a good
thing.
What you've been told is a remnant of an old issue (tale) that is not an issue anymore. Some people will still warn you not to do so on Ubuntu without even knowing why!
Starting from Ubuntu 19.10 and later, you can run GUI applications with sudo.
The old issue was that sudo used to write and modify the applications configuration files in the current user's home directory changing their permissions and ownership from the current user to root which will render them inaccessible to the current user and thus mess up the current user's home directory's contents to the point that GUI applications might not run anymore or run with errors ... this, however, is not the case anymore. Starting from Ubuntu 19.10, using sudo with GUI applications will assume root's home directory and not the current user's home directory.
However, running applications (GUI or otherwise) with sudo gives them root's privileges and this might pose major security, stability and integrity problems and this one was and still the same so use precaution and common sense with it.
That said ... make sure your application needs sudo to function or otherwise you should refrain from using sudo.
I just installed qemu-kvm and virt-manager. But when I run
virt-manager by clicking the icon it is not connecting to kvm. But
when I run $ sudo virt-manager KVM connects.
virt-manager is a desktop user interface for managing virtual
machines through libvirt and clearly doesn't need sudo to run ... what you, probably, need to do is add your user to the two groups libvirt and kvm as stated in the installation instructions.