Topic: Qemu with tap networking on FreeBSD Current

As people read and comment, I'm making a few changes.
--------------------------------------
Changelog
17 January, 2007.

Changed the qemu-ifup script, the 0.0.0.0 address is unnecessary.
Added some information about oddities with the bridge0 interface's IP address.
Fixed a typo, where I was showing the guest O/S' IP to be the same as the host's IP.  (Sorry folks)
Did some editing of the style
Added a link to the qemu networking page

19 January, 2007

Added more detail to doing ifconfig bridge0 addm tap0

21 January, 2007

Added possible (though kludgy) startup and shutdown scripts
Added better explanation of getting permission denied on tap interfaces

27 January, 2007

Added a bit on unloading modules

29 January, 2007

Added more on unloading modules

3 February, 2007

Added mention that slirp is working again

9 April, 2007

Added note that this howto is not working on CURRENT right now, then, later that evening, added
the fix for it (and took away the notice at the top that it wasn't working.)

10 April, 2007

Added information about net.link.tap.up_on_open

1 June, 2007

Added information about a discussion about bridging on freebsd-emulation.

2 June, 2007

Fixed information about configuring bridging--removed what was probably bad advice about giving bridge an IP address--many thanks to Per Hedeland.  Also added the exit 0 to the qemu-ifup script, again thanks to Per.   Lastly, made the qemu-ifconfig script more intelligent, once again, thanks to Per. 

-------------------------------------


Assuming you have read vermaden's excellent qemu howto you already have qemu installed with a guest O/S.  By default, qemu will start with what is called user mode networking--the guest O/S will automagically give itself an IP address of 10.0.2.x and although pinging doesn't work, most other things do--one can browse the Internet, ssh from the guest to the host, and so on.

There are various forms of networking with qemu.  How you do this will depend upon your needs.  I used
the default user mode until it stopped working in CURRENT.  For an overview of the different methods, one can check the unofficial qemu wiki's networking page.

As of early February, the latest qemu once again works with default user mode networking (more properly known as slirp).  However, once you get tap set up, it's probably more convenient.  The machine can simply be another node on your network.

The tap networking method adds a few features.  Pinging will work, and VPN networking seems to work better (at least on a Windows guest O/S using the Nortel VPN client).

This howto is only for FreeBSD-7.  The method of setting up tap networking has changed.  For FreeBSD-6.x see the article on the acidos site.

I want to give my most sincere thanks to Bakul Shah who saved me a great deal of googling and false starts.  All mistakes are, of course, my own.  Bakul's setup is far more sophisticated than the one I describe. 

This is a very simple setup.  We are going to have a qemu guest on the same subnet as the host, with a manually configured IP address and gateway.   However, it works for me.

So, assuming you have qemu installed with a guest O/S and can boot it, we have a few things to do first.

We have to allow tap networking for an ordinary user.  We'll call our user John, with a username of john.  I am assuming the reader is familiar with sudo.  The user john is in the sudoers file.   Much of the preparation will have to be done as root or with root privilege. 

Add the following line to /etc/devfs.conf

own     tap0 john:john

Restart devfs

/etc/rc.d/devfs restart

If these modules aren't loaded, load them.

kldload kqemu aio if_tap if_bridge

Enable a user to open tap devices

sysctl net.link.tap.user_open=1

(You'll see a message that net.link.tap.user_open is changed from 0 to 1.)



Create a bridge.




ifconfig bridge0 create

Add the physical interface to the bridge.  In my case, my nic is vr0 for a cheap onboard Via network card.  Change this according to your network card's designation. 

ifconfig bridge0 addm vr0 up

For a long time, this howto left out the "up" at the end of the above line and gave the bridge a specific IP address.  Per Hedeland pointed out the error.

If you haven't yet started qemu, there won't yet be a tap0 interface.  If you've had an earlier qemu session and then stopped it, you can add the tap0 interface now.

ifconfig bridge0 addm tap0

If this is your first qemu session since the machine was started, or if, after closing the last session, you've unloaded the tap module, etc, you will get an error message that tap0 doesn't exist.  Then, you will have to wait to add it as a member to the bridge after the guest has been started.   (This is covered below).
If you create the bridge at boot (by adding lines to /etc/rc.conf--this is covered below) you won't need to bring your nic up.  That is, you can simply use the addm line without the up at the end.


There was an interesting discussion on this between Bakul, Per Hedeland and myself about this on freesbsd-emulation's mailing list.  (Look for the week that ended 3 June, 2007).  Per pointed out my errors in the original howto along with the solution.


Back to our procedure.

Create an /etc/qemu-ifup script.  It should read as follows. 

#!/bin/sh
sudo /sbin/ifconfig ${1} up

Make the script executable

chmod 755 /etc/qemu-ifup

Interestingly enough, one didn't need the up at the end of the ifconfig line until early April 2007.  I thought that tap networking was broken, but all it needed was for that "up" to be added to the ifconfig line.  Many thanks to Tobias Grosser, who pointed me to this solution.

Rather than add up to the ifconfig line, you can change a sysctl variable.  There is a variable

net.link.tap.up_on_open

once you kldload if_tap.  It's set to 0 by default.  Changing it to 1 with

net.link.tap.up_on_open=1

will also work.  You can do one or the other, either one will work.   The important point is that if_tap is no longer automatically up after creation.  Many thanks to Bruce M. Simpson, who told me about the sysctl variable.  (Both he and Tobias responded to a post of mine to current and emulation@FreeBSD.org, asking why if_tap wasn't working.)
Start qemu--in this example, our qemu image is ~/qemu/win2k.img as per vermaden's article.

Decide what network address you're going to give the guest O/S.  In this case, on our 192.168.1.x network, we'll give it an address of 192.168.1.40.  The gateway on this network (which can always be checked with netstat -r) is
192.168.1.1. 

vermaden's howto gave various options for starting qemu, such as using -localtime, adjusting memory, sound, etc.  In this case, we won't bother with that, we're just testing networking.

qemu -net nic -net tap -hda ~/qemu/win2k.img

At times, I've gotten an error message that tap9 couldn't be opened.  If this happens it means that I've either forgotten to change the sysctl net.llink.tap.user_open variable to 1 or that devfs.conf has to be restarted.  (One has to load the if_tap module before the lines in devfs.conf will take effect.)

Unless you've added the NOPASSWD option to sudoers for the qemu-ifup script, it will also ask for your password--remember, the qemu-ifup script uses sudo.


As the system is booting, the tap0 interface is started (through the qemu-ifup scritp.  So, now add tap0 to the bridge.

ifconfig bridge0 addm tap0

As arun mentions, in the thread started by this post,  one can put the bridge0 addm tap0 in the qemu-ifup script.  (The ${1} in his sample script is the tap0 interface.   If you tend to frequently stop and start qemu, and don't unload the if_tap module each time, this can cause a problem, you'll get the error that the tap0 interface is already a member.  However, this message can be ignored.  However (thanks to Per for figuring this out for me) you HAVE to add the exit 0 that I show below.  Otherwise, you'll get the error message about tap existing, or already being a member, and the script will fail.
I'll cover this further in the startup script section.  If you almost always remember to unload the if_tap module when you're done with your qemu session,  or don't get annoyed by harmless error messages, then change the qemu-ifup script to read

#!/bin/sh
sudo /sbin/ifconfig ${1} up
sudo /sbin/ifconfig bridge0 addm ${1}
exit 0

One can also make the script more intelligent, (thanks again to Per Hedeland), checking if tap0 is already a member, and this will eliminate error messages.

#!/bin/sh

sudo /sbin/ifconfig $1 up
case "`/sbin/ifconfig bridge0`" in
    *" $1 "*) ;;      # already in the bridge
    *) sudo /sbin/ifconfig bridge0 addm $1 ;;
esac
exit 0

This eliminates the need to run ifconfig bridge0 addm tap0 as a separate command.  (Also, note that if you are using the sysctl net.link.tap.up_on_open way of making sure tap0 is up, you don't need the word "up" in the ifconfig ${1} line.)

Adjust the guest O/S' network. Hopefully, you know how to do this.  The IP address, as we said, is 192.168.1.40.  The netmask, DNS server(s) and gateway are the same as the host's settings. 

To review, the host's settings are as follows

IP address              192.168.1.50
Netmask                 255.255.255.0
Default Gateway      192.168.1.1
Nameserver            192.168.1.80   

So, we will give the guest O/S the following settings.


IP address              192.168.1.40
Netmask                 255.255.255.0
Default Gateway      192.168.1.1
Nameserver            192.168.1.80

In other words, it is being treated just as if it were another physical host on the LAN.




If you want to save the various settings then we will change a few configuration files.

For the sysctl variable add the following  to /etc/sysctl.conf

net.link.tap.user_open=1

The bridge can be set in /etc/rc.conf.    Again, with a vr0 network card, add the following to /etc/rc.conf

cloned_interfaces="bridge0"
ifconfig_bridge0="addm vr0"

Note that we are only adding the physical interface here, not the tap0 one.

Adding the above to /etc/rc.conf will also load the if_bridge module on startup.  To load kqemu and aio on startup add

kqemu_enable="YES"

to /etc/rc.conf.

To load the if_tap module on startup add

if_tap_load="YES"

to /boot/loader.conf 

I don't run qemu all that often, so for me, it's a bit easier to use simple scripts.  I keep meaning to make them a bit more sophisticated, for example, check if a module is loaded before loading it, but haven't yet gotten around to it.

I have two scripts, qemuprep.sh and qemudown.sh



The qemuprep.sh is

#!/bin/sh

#load modules if not loaded.  
sudo kldload if_tap if_bridge kqemu aio

#create the bridge and configure it.

sudo ifconfig bridge0 create

sudo ifconfig bridge0 addm vr0 up

#fix sysctl
sudo sysctl net.link.tap.user_open=1

#restart devfs to work with tap
sudo /etc/rc.d/devfs restart
exit 0

Note that if modules are already loaded, although you'll get an error message, it won't break anything.
For example, once the aio module is loaded, even after qemu stops, trying to unload it gives an error message of

kldunload: can't unload file: Operation not supported

So, if I run qemu more than one time before rebooting, when I run my qemuprep.sh script, I'll get the error message that aio is already loaded.  It's harmless.  If it bothers you, then you can do

kldstat | grep -q aio
  if [ "$?" -gt 0 ]; then
sudo kldload if_tap if_bridge kqemu aio

#if it's already loaded, then just load the others

  else

sudo kldload if_tap if_bridge kqemu

instead of the sudo kldload if_tap if_bridge kqemu aio line.

I add the host's interface as a member.  (If the user hasn't figured it, the addm option adds a member to the interface.  See man(8) ifconfig.

Since my shutdown script unloads the if_tap module, devfs.conf will have to be reread.  Therefore, I restart devfs at the end of the script.  I could also add the line

qemu -net nic -net tap -hda ~/qemu/"${1}" -localtime

to the end of it, but since much of the time, I use it to boot CDs, I don't add that line.  If you only have one qemu .img file that you use, and you almost always boot to that image, you could add something similar, replacing "${1}" with the image that you always use.   (I could do other things too, such as first present a list of ~/qemu and select one, or select to boot from CD, and I might do that at some point.)

When I'm finished with qemu, I like to remove the modules and bridge.  After I have closed the qemu guest session, I run my qemudown.sh


#bring down the bridge
sudo ifconfig bridge0 destroy


#unload the unused modules.  

sudo kldunload kqemu
sleep 1
sudo kldunload if_tap
sleep 1
sudo kldunload if_bridge

exit 0

Note that I don't unload aio, since it won't unload.  I've also found that unloading the if_tap module makes the devfs.conf line irrelevant, so there's no need to remove those lines from devfs.conf.   

The reason I do a sleep 1 before unloading the tap and bridge modules separately might simply be an oddity in my system.  I've found that if I just do kldunload kqemu if_tap if_bridge, the tap and bridge modules may not unload.  Whether that's a normal thing or simply an oddity on this box, I don't know, nor have I investigated.  Doing a sleep 1 and then unloading the  modules seems to usually be successful.
I could add a bit to check that unloading if_tap was definitely successful and if not, try again, but at this point, it doesn't seem worth the trouble.  I usully run kldstat once more after this to make sure all is unloaded, and if if_tap STILL isn't unloaded, just kldunload it again.  However, that's only necessary on rare occasions. 

Remember, if tap isn't unloaded, then the next time you start qemu, your qemu-ifup script will give an error.  So, if you start qemu and get an error that tap couldn't be added as a member to bridge0 as it's already loaded, you can either comment out the addm tap0 line in /etc/qemu-ifup or just keep doing kldunload until you see that if_tap is unloaded.  However, it's not necessary, as long as you have the exit 0 line at the end of your qemu-ifup script, the message causes no problems.



The above two scripts are rather kludgy.  At some point, if I ever get around to it, I hope to make them a bit more sophisticated.  See dev1's script in this thread.   Lame as these scripts are, they do the job for me.  As arun mentions in his post below, they also save a reasonable amount of typing.

The above is a very simple tap networking setup.   Bakul's setup, for example, has all the qemu guests  on their own subnet, and the host does their dhcp, forwarding and nat.  His qemu-ifup script does the adding of the tap0 interface to the bridge.   He also runs multiple guests.

There are various other ways to do this.  As one can see in the thread started by this post, one can script most of this to load and unload as desired.

Last edited by scottro (2007-06-02 22:01:17)

<@andre> i would be so much more efficient if i wasn't so stupid

Re: Qemu with tap networking on FreeBSD Current

I too had issues with tap0 not being found. After some trail and error I added the last line to /etc/qemu-ifup:

#!/bin/sh

sudo ifconfig ${1} 0.0.0.0 up
sudo ifconfig bridge0 addm fxp0 addm ${1} up

That solved it. smile

Also, I use this script to launch my openbsd image. Saves a lot of typing. smile

#!/bin/sh

# create the tap interface
sudo kldload if_tap
sudo chown 1001 /dev/tap0
# enable user to use the tap interface
sudo sysctl net.link.tap.user_open=1
#sudo kldload kqemu
# create bridge between fxp0 and tap0
sudo kldload if_bridge
sudo ifconfig bridge0 create
# load the image
qemu -m 64 -net nic -net tap -hda openbsd.qcow

#sudo kldunload kqemu - Open has problems with kqemu
sudo kldunload if_tap
sudo kldunload if_bridge
sudo sysctl net.link.tap.user_open=0

I got problems with kqemu and Open. It just won't boot. Got any solutions ?

PS: I got the script idea from a doc I saw on vermaden's site. smile

Last edited by arun (2007-01-17 20:21:47)

Keep Smiling

Re: Qemu with tap networking on FreeBSD Current

Yes Bakul uses a similar script to yours.  Here's why I decided against putting it in.

Suppose I do it to open a Win2k qemu session. I do what I have to do, then close qemu.

Now, the next time I open qemu, if, for example, I know want to run an Archlinux session, when it starts doing the addm vr0 I'll get an error because the bridge has already been done.

As for the problems with kqemu and OpenBSD, I don't know.  I haven't tried OpenBSD on it yet.

Darn, I vaguely remember hearing something about something not running well with kqemu--how's that for useless information?

Bakul gave me a few ideas that I will probably add to the howto, I just want to test them first.

<@andre> i would be so much more efficient if i wasn't so stupid

Re: Qemu with tap networking on FreeBSD Current

scottro wrote:

Suppose I do it to open a Win2k qemu session. I do what I have to do, then close qemu.

Now, the next time I open qemu, if, for example, I know want to run an Archlinux session, when it starts doing the addm vr0 I'll get an error because the bridge has already been done.

That's why I have 'sudo kldunload if_bridge' in the script. I don't know if this is the right way to take down the bridge. But it works. wink

Keep Smiling

Re: Qemu with tap networking on FreeBSD Current

Yes, that should certainly work. I would think you'd destroy the bridge first.

I have a few odd things with the bridge--sometimes, the guest stops being able to reach the host (though it can reach EVERYTHING else).  Now, the weird part is that I fixed this by giving the bridge the same IP as the host but with a 32 bit, rather than 24 bit netmask. I really don't know enough about bridging.  Sigh.

<@andre> i would be so much more efficient if i wasn't so stupid

Re: Qemu with tap networking on FreeBSD Current

Just thought I'd add that this howto has been added to the pkg_message for qemu, since slirp networking is having problems with CURRENT.  smile    <modest bow>

Maybe someone who really understands bridging will read it and let me know should the bridge address be the host's or simply another address on the subnet.

<@andre> i would be so much more efficient if i wasn't so stupid

Re: Qemu with tap networking on FreeBSD Current

Very nice howto scottro smile

"religions, worst damnation of mankind"
[color=Blue]Linux is not UNIX! Face it! It is not an insult. It is fact: GNU is a recursive acronym for

Re: Qemu with tap networking on FreeBSD Current

Thank you dev1, that's great.

<@andre> i would be so much more efficient if i wasn't so stupid

Re: Qemu with tap networking on FreeBSD Current

Thank you for howto Scott ... and to Vermaden for his qemu HOWTO ... I got everything set up in 10 minutes, now I can waste my time and rot my brains playing computer games.

Oh ... and merry christmas, happy new year, etc ... I know ... better late than never

Trust me, I know what I'm doing.

Re: Qemu with tap networking on FreeBSD Current

And you too--don't smoke too much carpet.  smile

<@andre> i would be so much more efficient if i wasn't so stupid

Re: Qemu with tap networking on FreeBSD Current

Same to you Carpetsmoker wink

"religions, worst damnation of mankind"
[color=Blue]Linux is not UNIX! Face it! It is not an insult. It is fact: GNU is a recursive acronym for

Re: Qemu with tap networking on FreeBSD Current

The above two scripts are rather kludgy.  At some point, if I ever get around to it, I hope to make them a bit more sophisticated.  See dev1's script in this thread.   Lame as these scripts are, they do the job for me.  As arun mentions in his post below, they also save a reasonable amount of typing.

I am also agree with you that these two scripts are sophisticated. I think we need to find out easier scripts. I am working for it. If I get success in this case, I will update it.

Re: Qemu with tap networking on FreeBSD Current

So what was problem in script?
astigmatism

Last edited by tarapozan (2011-06-24 12:11:51)