Contrary to the note in section 1, you CAN run virt-manager on a headless server.

On my Mac, I ran "xhost +my.server.ip".  Then "ssh -X my.server.ip".  Once there, "sudo apt-get install virt-manager".  This will install a bunch of stuff, including GTK libs and some supporting X apps, but it DOESNT actually require X, a Window Manager, or all the other X bloat you don't need if you aren't using the local monitor. 

Then make sure the DISPLAY variable is set: "echo $DISPLAY".  If not, set it: "export DISPLAY='my.mac.ip:0' ".  Then by just typing "sudo virt-manager" on the server (via the ssh session), virt-manager will RUN on the headless server, but DISPLAY on your Mac.

 This assumes you have X11 installed on the Mac.  If not, install it from the Leopard DVD.

 This should also work equally well on a PC, just use something like Putty and XMing to provide the SSH connection and an X Window (display) server on your PC.

Hope that tip helps someone out--managing KVM via command line was driving me nuts!