= Installation and Usage of Open vSwitch = This page describes how to install and use OpenVSwitch (OVS). OVS is a software switch implementation designed to run on Linux. More information about the software can be found [http://openvswitch.org/ here]. == Contents == [#install I. Installation] [[BR]] [#run II. Running OVS] [[BR]] ---- = I Installation = #install OVS can be installed either from source or from binaries. Section 1.2 covers how to build from source, and section 1.3 covers install from binaries. == 1.1 Prerequisites == The node used here is a NetFPGA cube, the standard node for !SandBox9. The node was imaged with ubuntu.ndz, based on Ubuntu 10.10 (Maverick). The steps have been tested successfully with up to Ubuntu 12.04. == 1.2 Method 1: from source == The steps followed in this section are based on the contents of INSTALL.Linux, included with the OVS tarball and also found [http://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.Linux;hb=HEAD here](also attached at the end of this page for your convenience). It should be found under the OVS root directory. == 1.2.1 Getting the source and dependencies == 1. If your Ubuntu install is still fresh, run 'apt-get update'. If this fails, repeat after replacing all instances of 'apt:9999' with 'us.archive.ubuntu.com' in /etc/apt/sources.list . 1. Install dependencies: {{{ apt-get install pkg-config autoconf automake linux-libc-dev libtool }}} 1. Fetch the OVS tarball and untar. the latest source can be found at http://openvswitch.org/download/ . {{{ cd ~/ wget http://openvswitch.org/releases/openvswitch-1.6.1.tar.gz tar -xf openvswitch-1.1.1.tar.gz }}} 1. Build. On Debian (and its variants), Open vSwitch must be built as a kernel module. If everything is sound, installing OVS should be little more than following the steps in INSTALL.Linux. {{{ cd ~/openvswitch-1.1.1 ./boot.sh ./configure --with-l26=/lib/modules/`uname -r`/build make make install }}} 1. instantiate the kernel module: {{{ insmod datapath/linux-2.6/openvswitch_mod.ko }}} === 1.2.2 Some Sanity Checks. === If things don't go well, here are some things worth checking: * Check /usr/src/linux-headers-!`uname -r`/.config for the following kernel configs: * CONFIG_BRIDGE as module (=m) * NET_CLS_ACT, NET_CLS_U32, NET_SCH_INGRESS as modules or built-in (=m or =y, respectively) if policing * CONFIG_TUN if you need tunneling Fix them as necessary. * The bridge module should not be loaded (e.g. should not show up when you do `lsmod | grep bridge`); remove it if it is loaded. If it seems to be loaded at boot time, there may be an entry for it somewhere in /etc/modules. * /lib/modules/$(uname -r)/build should be a link to the Linux kernel header directory: {{{ root@node1-1:~# ls -ald /lib/modules/$(uname -r)/build lrwxrwxrwx 1 root root 40 2011-07-26 13:41 /lib/modules/2.6.35-30-generic/build -> /usr/src/linux-headers-2.6.35-30-generic }}} This directory should contain (mostly) unbroken links. If not, repeat step 3 of the prerequisites with another kernel version, e.g. by upgrading the kernel as follows: {{{ apt-get install linux-headers-2.6.35-30-generic linux-image-2.6.35-30-generic reboot apt-get install linux-source-2.6.35 }}} When linux-image is installed, grub is updated so that the newest kernel is loaded automatically upon next reboot. Re-installing linux-source after reboot should install the proper version for the new kernel. * Modules may not be loaded properly; look for the OVS module with `lsmod`: {{{ root@node1-1:~/openvswitch-1.1.1# lsmod | grep open openvswitch_mod 68183 1 }}} * In general, `dmesg` can be used to check for various anomalies when things e.g. insmod fail silently. == 1.3 Method 2: from binaries == Binaries are easier to set up, but lag in OVS version and support a narrower range of architectures and kernel versions. 1. Install the openvswitch packages. Do not use the Ubuntu repositories since the install the incorrect versions of the package; Download the packages that match your kernel version from [http://openvswitch.org/releases/binaries/1.2.2.10448/natty_amd64/ here] [[BR]] For an x86_64 system, the following are needed (The package '''openvswitch-brcompat_1.2.2.10448_amd64.deb''' should be added if bridge compatibility is needed): {{{ openvswitch-datapath-module-2.6.38-8-server_1.2.2.10448_amd64.deb openvswitch-common_1.2.2.10448_amd64.deb openvswitch-switch_1.2.2.10448_amd64.deb }}} Install them in that order with "dpkg -i". It will recommend a restart. 1. The module should be loaded automatically upon installation and system reboot. You should be able to query the module: {{{ root@external3:~# ovs-vsctl show d03e1847-34f4-4129-8821-63fff3403553 ovs_version: "1.2.2.10448" }}} lsmod should also show the running openvswitch_mod. == 1.4 Section I References == The following links were referenced but aren't relevant overall; this is just for citation. * http://www.cyberciti.biz/tips/build-linux-kernel-module-against-installed-kernel-source-tree.html * http://forums.gentoo.org/viewtopic-t-118180-highlight-bridgeutils.html?sid=4d602c9e364130609caff99aa2a40c69 * https://help.ubuntu.com/community/Kernel/Compile * http://ubuntuguide.net/ubuntu-11-04-upgrade-linux-kernel-to-2-6-39-0 ---- = II Running OVS. = #run OVS has three main components that must be initialized: * openvswitch_mod.ko, the OVS kernel module * ovsdb, the database containing configurations * ovs-vswitchd, the OVS switch daemon The daemon configures itself using the data provided by the database; `ovs-vsctl` is used to modify the contents of the database in order to configure the OVS switch at runtime. == 2.1 Initialization == 1. Load openVswitch kernel module {{{ cd datapath/linux/ insmod openvswitch_mod.ko }}} Note, OVS and Linux bridging may not be used at the same time. This step will fail if the bridge module (bridge.ko) is loaded. You may need to reboot the node in order to unload bridge.ko.[[BR]] If this is the first time OVS is being run, make am openvswitch directory in /usr/local/etc/ and run `ovsdb-tool` to create the database file: {{{ mkdir -p /usr/local/etc/openvswitch ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema }}} 1. Start ovs-db: {{{ ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \ --remote=db:Open_vSwitch,manager_options \ --pidfile --detach }}} 1. Initialize the database: {{{ utilities/ovs-vsctl --no-wait init }}} the `--no-wait` allows the database to be initialized before ovs-vswitchd is invoked. 1. Start ovs-vswitchd: {{{ vswitchd/ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock --pidfile --detach }}} The 'unix:...db.sock' specifies that the process attach to the socket opened by `ovsdb`. == 2.2 Configuring OVS == Once OVS is running, virtual switches may be created and configured. In general, a virtual switch is comprised of a bridge interface (usually named br''x''), and one or more interfaces associated with it. A single vswitchd can control multiple virtual switches with arbitrary number of ports each. === 2.2.1 Creating virtual switches === The following steps create a bridge interface (br0) and associate an interface to it: {{{ ovs-vsctl add-br br0 ovs-vsctl add-port br0 eth0 }}} The same steps can be used to add VLAN interfaces: {{{ ovs-vsctl add-port br0 eth0.222 }}} In this case, the ports added to the bridge interface are trunked by default. Using the option tag=VLAN ID makes the interfaces behave as access ports for the VLAN ID specified: {{{ ovs-vsctl add-port br0 eth0.111 tag=111 ovs-vsctl add-port br0 eth0.222 tag=222 }}} === 2.2.2 Network configuration === For an OVS switch to behave as a plain vanilla switch, both bridge interface and associated network interfaces must be up and configured with IP layer information. The bridge interface can be configured with tools such as `ifconfig` (or even DHCP) like any other *nic interface. Its configurations may even be stored in /etc/network/interfaces for persistence: {{{ #static bridge interface configs auto br0 iface br0 inet static address 192.168.1.10 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 #bridges can use dynamic configuration as well auto br1 iface br1 inet dhcp }}} == 2.3 OVS with KVM == OVS can be used with KVM. The instructions for setting this up can be found [http://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.KVM;hb=HEAD here], or in the root directory of the source as INSTALL.KVM. A use case is documented in this separate [http://orbit-lab.org/wiki/Internal/VMHostSetup wiki entry]. == 2.4 OVS with !OpenFlow == OVS switches may be run as !OpenFlow switches. The following steps describe how to run OVS in !OpenFlow mode. 1. If it has not been done already, fire up an !OpenFlow controller. The procedures for this step differ according to the controller in use, and are discussed in the pages for each respective controller. [[BR]] A sanity check for this step is to test your virtual switch with the OVS built-in controller, `ovs-controller`, which may be initialized on the same node running OVS: {{{ ovs-controller -v ptcp:6633 }}} When ovs-controller is used, the controller IP is, unsurprisingly, 127.0.0.1. 2. Point ovs-vswitchd to the !OpenFlow controller. {{{ ovs-vsctl set-controller br0 tcp:172.16.0.14:6633 }}} In this example, the OVS process is pointed to a BSN controller (kvm-big) on 172.16.0.14, listening on port 6633. With a properly initialized and configured database, `ovs-vswitchd` will spit out a bunch of messages as it attempts to connect to the controller. Its output should look something similar to this: {{{ root@node1-4:/opt/openvswitch-1.2.2# vswitchd/ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock --pidfile --detach Nov 07 17:37:02|00001|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: connecting... Nov 07 17:37:02|00002|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: connected Nov 07 17:37:02|00003|bridge|INFO|created port br0 on bridge br0 Nov 07 17:37:02|00004|bridge|INFO|created port eth0.101 on bridge br0 Nov 07 17:37:02|00005|bridge|INFO|created port eth0.102 on bridge br0 Nov 07 17:37:02|00006|ofproto|INFO|using datapath ID 0000002320b91d13 Nov 07 17:37:02|00007|ofproto|INFO|datapath ID changed to 000002972599b1ca Nov 07 17:37:02|00008|rconn|INFO|br0<->tcp:172.16.0.14:6633: connecting... }}} The !OpenvSwitch !OpenFlow switch should be functional as soon as it finds and connects to the controller. As you can see above, a DPID is chosen at random; if a random DPID does not suit your needs, a DPID may be specified manually using ovs-vsctl: {{{ ovs-vsctl set bridge other-config:datapath-id= }}} Where is a 16-digit hex value. For our network node, this becomes: {{{ ovs-vsctl set bridge br0 other-config:datapath-id=0000009900113300 }}} Once running, all typical things applicable to an !OpenFlow switch applies to the running OVS switch.