Running Qemu/KVM without libvirt

When I booted a VM yesterda, I noticed that there was a huge command line that showed up if I ran ps. I tried to run that by hand.  It is huge, so I wrapped it with a script, but the command is not too bad to understand:  everything that qemu needs to do needs to be passed in on the command line.

Complete command line is at the end of the article.
I first put SELinux into permissive mode, as it will not allow my user to create a VM.

I needed to adjust three values.

  1. First, the VM opens a domain socket, used for monitoring the VM. The path that this points to is in /var/lib/libvirt/qemu/ which my user does not have access to. I changed this to /home/ayoung/devel/qemu.
  2. The image is read out of /home/ayoung/devel/qemu which my user does not have access to. I copied it to /home/ayoung/devel/qemu and changed ownership. I also changed the path used in the call.
  3. How to connect to the network interface.  when libvirt kicks is off, it uses an fd argument, which indicates it should reuse an open file descriptor. Since that is a process specific value, we can’t use that, but need to link the VM up to a network some other way. I’m still fiddling with this.

The xml file that has that defined is in

/etc/libvirt/qemu/generic.xml.

The network is called default.  It is defined in

/var/lib/libvirt/dnsmasq/default.conf

And that maps to :

interface=virbr0

I first tried using

tap,id=hostnet0,ifname=tap0

but that seems to try to connect to

/dev/net/tun

which is not allowed. So instead I tried changing to a bridge device:

bridge,id=hostnet0,br=virbr0

If I run with that up, I see that ip addr reports:

12: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0 state UNKNOWN group default qlen 1000
    link/ether fe:2f:d3:35:da:91 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc2f:d3ff:fe35:da91/64 scope link 
       valid_lft forever preferred_lft forever

And that goes way if I kill the VM.

Once it is up I try to use netcat to talk to to the VM (thanks Kashyap)

$ nc -U monitor.sock
{"QMP": {"version": {"qemu": {"micro": 1, "minor": 6, "major": 2}, "package": " (qemu-2.6.1-1.fc24)"}, "capabilities": []}}

So the VM process is reporting status.

I can attach using SPICE:

remote-viewer spice://127.0.0.1:5900

but nothing shows. However, if the VM is not running, it just fails. So, I know this is working, but I still need a way to communicate with the VM.

Here is what I am using to run the VM.

#!/bin/sh

#VARPATH=/var/lib/libvirt/qemu/domain-2-generic
VARPATH=/home/ayoung/devel/qemu
#IMAGEPATH=/var/lib/libvirt/images
IMAGEPATH=/home/ayoung/devel/qemu
#NETDEV_PARAMS=tap,fd=29,id=hostnet0
NETDEV_PARAMS=bridge,id=hostnet0,br=virbr0

/usr/bin/qemu-system-x86_64 \
    -machine accel=kvm \
    -name generic,debug-threads=on \
    -S \
    -machine pc-i440fx-2.6,accel=kvm,usb=off,vmport=off \
    -cpu Haswell-noTSX \
    -m 1024 \
    -realtime mlock=off \
    -smp 1,sockets=1,cores=1,threads=1 \
    -uuid 6f6f9463-8b7e-401c-910e-d217e00816a1 \
    -no-user-config \
    -nodefaults \
    -chardev socket,id=charmonitor,path=$VARPATH/monitor.sock,server,nowait \
    -mon chardev=charmonitor,id=monitor,mode=control \
    -rtc base=utc,driftfix=slew \
    -global kvm-pit.lost_tick_policy=discard \
    -no-hpet \
    -no-shutdown \
    -global PIIX4_PM.disable_s3=1 \
    -global PIIX4_PM.disable_s4=1 \
    -boot strict=on \
    -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 \
    -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 \
    -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 \
    -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 \
    -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 \
    -drive file=$IMAGEPATH/generic.qcow2,format=qcow2,if=none,id=drive-ide0-0-0 \
    -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \
    -netdev $NETDEV_PARAMS \
    -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:4d:04:7d,bus=pci.0,addr=0x3 \
    -chardev pty,id=charserial0 \
    -device isa-serial,chardev=charserial0,id=serial0 \
    -chardev spicevmc,id=charchannel0,name=vdagent \
    -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 \
    -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on \
    -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 \
    -device intel-hda,id=sound0,bus=pci.0,addr=0x4 \
    -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
    -chardev spicevmc,id=charredir0,name=usbredir \
    -device usb-redir,chardev=charredir0,id=redir0 \
    -chardev spicevmc,id=charredir1,name=usbredir \
    -device usb-redir,chardev=charredir1,id=redir1 \
    -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 \
    -msg timestamp=on

4 thoughts on “Running Qemu/KVM without libvirt

  1. How did you conjure up this monstrosity? I’ve been trying to figure out qemu/kvm for months now, and I still have to use the virtual machine manager to configure things.
    I’ve searched every website / blog / howto that has ever had anything to do with qemu, and I still can’t figure out the networking.
    (I’m really thankful for virtualbox, otherwise I’d never get anything done at this rate. Their vboxmanage command actually makes sense and is really well documented.)
    QEMU documentation (whatever I can find) sucks whales – its usually dated or simply incomplete. Most of the suggestions are half-baked and don’t work.
    If there is something out there that is current and accurate, I have yet to discover it. This for me is the most disappointing opensource software I’ve ever encountered, and I’ve been using / following open since the mid-eighties.

    Just my two cents.

  2. I think part of the problem is that virtual machine manager has become the norm for people using Qemu/KVM that the rest of the documentation has bit rotted. It was hard to find anything documenting it from this side.

    I was, however, given the suggestion to look for documentation for people doing Kernel research, as many of them are doing direct KVM operations for developing bring-up.

    The monstrosity of a command line is, as I said, the one that VMM shows in ps…plus some trial and error.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.