Install OpenVSwitch on CentOS 6.x x64
背景
前面两篇文章介绍了OpenvSwitch的原理，用途。
本文包含openvswitch的模块安装, 管理软件安装.
玩转起来。
安装过程
安装依赖包, 注意内核版本匹配 :
# uname -r
2.6.32-431.el6.x86_64
# yum install -y rpm-build redhat-rpm-config kernel-devel-2.6.32-431.el6.x86_64
下载openvswitch LTS版本 :
[root@176 ~]# mkdir /opt/soft_bak
[root@176 ~]# cd /opt/soft_bak/
# wget http://openvswitch.org/releases/openvswitch-1.9.3.tar.gz
解压, 并将源码包放到RHEL的rpm building SOURCES目录.
# tar -zxvf openvswitch-1.9.3.tar.gz
# mkdir -p /root/rpmbuild/SOURCES
# cp openvswitch-1.9.3.tar.gz /root/rpmbuild/SOURCES/
# cd openvswitch-1.9.3
# less INSTALL.RHEL 查看安装步骤.
生成软件rpm包,
# rpmbuild -bb rhel/openvswitch.spec
# cp rhel/openvswitch-kmod.files /root/rpmbuild/SOURCES/
生成内核模块重新执行重新执行包 :
[root@176 openvswitch-1.9.3]# uname -r
2.6.32-431.el6.x86_64
# rpmbuild -bb -D "kversion 2.6.32-431.el6.x86_64" -D "kflavors default" rhel/openvswitch-kmod-rhel6.spec
********************************************************************************
*********************** KERNEL ABI COMPATIBILITY WARNING ***********************
********************************************************************************
The kernel ABI reference files (provided by kabi-whitelists) were not found.
No compatibility check was performed. Please install the kABI reference files
and rebuild if you would like to verify compatibility with kernel ABI.
以上信息, 表示没有合适的kabi-whitelists包来检查兼容性, 可以忽略, 或者你安装一下这个包, 但是可能包太老, 新的内核没有加进来.
# yum install -y kabi-whitelists
然后重新执行, (如果不需要检测的话, 不需要重新执行这个) :
# rpmbuild -bb -D "kversion 2.6.32-431.el6.x86_64" -D "kflavors default" rhel/openvswitch-kmod-rhel6.spec
使用build的rpm包安装 :
[root@176 openvswitch-1.9.3]# cd /root/rpmbuild/RPMS/x86_64/
[root@176 x86_64]# ll
total 4080
-rw-r--r-- 1 root root 1199928 Aug 1 11:44 kmod-openvswitch-1.9.3-1.el6.x86_64.rpm
-rw-r--r-- 1 root root 2975203 Aug 1 11:39 openvswitch-1.9.3-1.x86_64.rpm
[root@176 x86_64]# rpm -ivh kmod-openvswitch-1.9.3-1.el6.x86_64.rpm
Preparing... ########################################### [100%]
1:kmod-openvswitch ########################################### [100%]
[root@176 x86_64]# rpm -ivh openvswitch-1.9.3-1.x86_64.rpm
Preparing... ########################################### [100%]
1:openvswitch ########################################### [100%]
安装完之后, 开启了一个服务.
[root@176 x86_64]# chkconfig --list|grep openv
openvswitch 0:off 1:off 2:on 3:on 4:on 5:on 6:off
安装的两个包的内容 :
[root@176 x86_64]# rpm -ql kmod-openvswitch
/etc/depmod.d/openvswitch.conf
/lib/modules/2.6.32-431.el6.x86_64
/lib/modules/2.6.32-431.el6.x86_64/extra
/lib/modules/2.6.32-431.el6.x86_64/extra/openvswitch
/lib/modules/2.6.32-431.el6.x86_64/extra/openvswitch/brcompat.ko
/lib/modules/2.6.32-431.el6.x86_64/extra/openvswitch/openvswitch.ko
[root@176 x86_64]# rpm -ql openvswitch
/etc/init.d/openvswitch
/etc/logrotate.d/openvswitch
/etc/sysconfig/network-scripts/ifdown-ovs
/etc/sysconfig/network-scripts/ifup-ovs
/usr/bin/ovs-appctl
/usr/bin/ovs-benchmark
/usr/bin/ovs-dpctl
/usr/bin/ovs-ofctl
/usr/bin/ovs-parse-backtrace
/usr/bin/ovs-parse-leaks
/usr/bin/ovs-pcap
/usr/bin/ovs-pki
/usr/bin/ovs-tcpundump
/usr/bin/ovs-vlan-test
/usr/bin/ovs-vsctl
/usr/bin/ovsdb-client
/usr/bin/ovsdb-tool
/usr/sbin/ovs-brcompatd
/usr/sbin/ovs-bugtool
/usr/sbin/ovs-vswitchd
/usr/sbin/ovsdb-server
/usr/share/doc/openvswitch-1.9.3/FAQ
/usr/share/doc/openvswitch-1.9.3/README.RHEL
/usr/share/man/man1/ovs-benchmark.1.gz
/usr/share/man/man1/ovs-pcap.1.gz
/usr/share/man/man1/ovs-tcpundump.1.gz
/usr/share/man/man1/ovsdb-client.1.gz
/usr/share/man/man1/ovsdb-server.1.gz
/usr/share/man/man1/ovsdb-tool.1.gz
/usr/share/man/man5/ovs-vswitchd.conf.db.5.gz
/usr/share/man/man8/ovs-appctl.8.gz
/usr/share/man/man8/ovs-brcompatd.8.gz
/usr/share/man/man8/ovs-bugtool.8.gz
/usr/share/man/man8/ovs-ctl.8.gz
/usr/share/man/man8/ovs-dpctl.8.gz
/usr/share/man/man8/ovs-ofctl.8.gz
/usr/share/man/man8/ovs-parse-backtrace.8.gz
/usr/share/man/man8/ovs-parse-leaks.8.gz
/usr/share/man/man8/ovs-pki.8.gz
/usr/share/man/man8/ovs-vlan-test.8.gz
/usr/share/man/man8/ovs-vsctl.8.gz
/usr/share/man/man8/ovs-vswitchd.8.gz
/usr/share/openvswitch/bugtool-plugins
/usr/share/openvswitch/bugtool-plugins/kernel-info
/usr/share/openvswitch/bugtool-plugins/kernel-info/openvswitch.xml
/usr/share/openvswitch/bugtool-plugins/network-status
/usr/share/openvswitch/bugtool-plugins/network-status/openvswitch.xml
/usr/share/openvswitch/bugtool-plugins/system-configuration
/usr/share/openvswitch/bugtool-plugins/system-configuration.xml
/usr/share/openvswitch/bugtool-plugins/system-configuration/openvswitch.xml
/usr/share/openvswitch/bugtool-plugins/system-logs
/usr/share/openvswitch/bugtool-plugins/system-logs/openvswitch.xml
/usr/share/openvswitch/python
/usr/share/openvswitch/python/argparse.py
/usr/share/openvswitch/python/ovs
/usr/share/openvswitch/python/ovs/__init__.py
/usr/share/openvswitch/python/ovs/daemon.py
/usr/share/openvswitch/python/ovs/db
/usr/share/openvswitch/python/ovs/db/__init__.py
/usr/share/openvswitch/python/ovs/db/data.py
/usr/share/openvswitch/python/ovs/db/error.py
/usr/share/openvswitch/python/ovs/db/idl.py
/usr/share/openvswitch/python/ovs/db/parser.py
/usr/share/openvswitch/python/ovs/db/schema.py
/usr/share/openvswitch/python/ovs/db/types.py
/usr/share/openvswitch/python/ovs/dirs.py
/usr/share/openvswitch/python/ovs/fatal_signal.py
/usr/share/openvswitch/python/ovs/json.py
/usr/share/openvswitch/python/ovs/jsonrpc.py
/usr/share/openvswitch/python/ovs/ovsuuid.py
/usr/share/openvswitch/python/ovs/poller.py
/usr/share/openvswitch/python/ovs/process.py
/usr/share/openvswitch/python/ovs/reconnect.py
/usr/share/openvswitch/python/ovs/socket_util.py
/usr/share/openvswitch/python/ovs/stream.py
/usr/share/openvswitch/python/ovs/timeval.py
/usr/share/openvswitch/python/ovs/unixctl
/usr/share/openvswitch/python/ovs/unixctl/__init__.py
/usr/share/openvswitch/python/ovs/unixctl/client.py
/usr/share/openvswitch/python/ovs/unixctl/server.py
/usr/share/openvswitch/python/ovs/util.py
/usr/share/openvswitch/python/ovs/version.py
/usr/share/openvswitch/python/ovs/vlog.py
/usr/share/openvswitch/python/ovstest
/usr/share/openvswitch/python/ovstest/__init__.py
/usr/share/openvswitch/python/ovstest/args.py
/usr/share/openvswitch/python/ovstest/rpcserver.py
/usr/share/openvswitch/python/ovstest/tcp.py
/usr/share/openvswitch/python/ovstest/tests.py
/usr/share/openvswitch/python/ovstest/udp.py
/usr/share/openvswitch/python/ovstest/util.py
/usr/share/openvswitch/python/ovstest/vswitch.py
/usr/share/openvswitch/python/uuid.py
/usr/share/openvswitch/scripts/ovs-bugtool-bond-show
/usr/share/openvswitch/scripts/ovs-bugtool-cfm-show
/usr/share/openvswitch/scripts/ovs-bugtool-coverage-show
/usr/share/openvswitch/scripts/ovs-bugtool-daemons-ver
/usr/share/openvswitch/scripts/ovs-bugtool-lacp-show
/usr/share/openvswitch/scripts/ovs-bugtool-memory-show
/usr/share/openvswitch/scripts/ovs-bugtool-ovsdb-dump
/usr/share/openvswitch/scripts/ovs-bugtool-tc-class-show
/usr/share/openvswitch/scripts/ovs-bugtool-vsctl-show
/usr/share/openvswitch/scripts/ovs-check-dead-ifs
/usr/share/openvswitch/scripts/ovs-ctl
/usr/share/openvswitch/scripts/ovs-lib
/usr/share/openvswitch/scripts/ovs-save
/usr/share/openvswitch/scripts/sysconfig.template
/usr/share/openvswitch/vswitch.ovsschema
/var/lib/openvswitch
/var/lib/openvswitch/pki
将bridge模块加入黑名单, 禁止自动加载bridge启动，bridge和ovs我们二选一来使用，所以这里禁掉bridge.
# lsmod |grep bri
## Blacklist the bridge module
vi /etc/modprobe.d/blacklist.conf
# Bridge, using openvswitch instead
blacklist bridge
重启机器
# reboot
可以看到, 已经加载了openvswitch模块.
[root@176 ~]# lsmod|grep openvs
openvswitch 96678 0
当前没有建立虚拟交换机
[root@176 ~]# ovs-vsctl show
8901fd9e-3b91-48ff-b0e8-8aa926fd2617
ovs_version: "1.9.3"
创建一个网桥
[root@176 ~]# ovs-vsctl add-br br0
查看刚创建的网桥
[root@176 ~]# ifconfig -a
br0 Link encap:Ethernet HWaddr 1A:E4:96:0B:24:43
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth0 Link encap:Ethernet HWaddr 00:23:7D:A3:F0:4E
inet addr:172.16.3.176 Bcast:172.16.3.255 Mask:255.255.255.0
inet6 addr: fe80::223:7dff:fea3:f04e/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7096 errors:0 dropped:0 overruns:0 frame:0
TX packets:1311 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:784135 (765.7 KiB) TX bytes:101647 (99.2 KiB)
eth1 Link encap:Ethernet HWaddr 00:23:7D:A3:F0:50
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
当前br0链路是DOWN的.
[root@176 ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:23:7d:a3:f0:4e brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:23:7d:a3:f0:50 brd ff:ff:ff:ff:ff:ff
5: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether 1a:e4:96:0b:24:43 brd ff:ff:ff:ff:ff:ff
启动网桥(或者使用ip link up来启动link)
[root@176 ~]# ifconfig br0 up
现在可以看到这个接口已经起来了, 状态是up的
[root@176 ~]# ifconfig
br0 Link encap:Ethernet HWaddr 1A:E4:96:0B:24:43
inet6 addr: fe80::18e4:96ff:fe0b:2443/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:398 (398.0 b)
[root@176 ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:23:7d:a3:f0:4e brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:23:7d:a3:f0:50 brd ff:ff:ff:ff:ff:ff
5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 1a:e4:96:0b:24:43 brd ff:ff:ff:ff:ff:ff
现在IP地址还配置在ETH0上面, 因为我们eth1没有配置IP, 所以如果直接把eth0接口加入网桥br0的话, 服务器就不通了.
需要新建br配置文件, 把IP配置到网桥的管理接口上, ETH0的IP消掉. 同时把eth0加入网桥. eth1不管他, 以后可以加入网桥也可以不加入网桥, 看怎么个用法.
[root@176 ~]# cd /etc/sysconfig/network-scripts/
[root@176 network-scripts]# cp ifcfg-eth0 ifcfg-br0
[root@176 network-scripts]# vi ifcfg-br0
DEVICE="br0"
BOOTPROTO="static"
BROADCAST="172.16.3.255"
DNS1="202.101.172.35"
GATEWAY="172.16.3.1"
IPADDR="172.16.3.176"
NETMASK="255.255.255.0"
NM_CONTROLLED="no"
ONBOOT="yes"
[root@176 network-scripts]# vi ifcfg-eth0
DEVICE="eth0"
BOOTPROTO="static"
HWADDR="00:23:7D:A3:F0:4E"
NM_CONTROLLED="no"
ONBOOT="yes"
BRIDGE="br0"
将eth0加入br0, 同时重启网络服务
# ovs-vsctl add-port br0 eth0; service network restart
现在可以看到, eth0加入了网桥br0, 还有一个接口叫br0 , 是这个网桥默认的内部接口, 不用管它.
[root@176 ~]# ovs-vsctl show
8901fd9e-3b91-48ff-b0e8-8aa926fd2617
Bridge "br0"
Port "eth0"
Interface "eth0"
Port "br0"
Interface "br0"
type: internal
ovs_version: "1.9.3"
虚拟机可以使用br0来创建虚拟接口了, 创建好虚拟接口后, 可以添加到ovs, 改vlan tag, 等操作.
参考
http://blog.163.com/digoal@126/blog/static/16387704020147111145122/
还可以继续新建网桥(每个网桥代表一个虚拟交换机)，如果你要把各个虚拟交换机连起来，可以创建很多peer设备，把peer设备的两端加入网桥即可。
[root@176 ~]# ovs-vsctl add-br br1
[root@176 ~]# ifconfig br1 up
[root@176 ~]# ovs-vsctl show
8901fd9e-3b91-48ff-b0e8-8aa926fd2617
Bridge "br1"
Port "br1"
Interface "br1"
type: internal
Bridge "br0"
Port "eth0"
Interface "eth0"
Port "br0"
Interface "br0"
type: internal
ovs_version: "1.9.3"
删除网桥
[root@176 ~]# ovs-vsctl del-br br1
[root@176 ~]# ovs-vsctl show
8901fd9e-3b91-48ff-b0e8-8aa926fd2617
Bridge "br0"
Port "eth0"
Interface "eth0"
Port "br0"
Interface "br0"
type: internal
ovs_version: "1.9.3"
参考
1. README
2. README.RHEL
3. http://blog.163.com/digoal@126/blog/static/16387704020147111145122/