ovs實踐1:基礎概念和gre隧道實踐操作

1.
測試環境
兩個虛機:nginx-1和nginx-3
centos7 系統內核3.10.0-514.el7.x86_64
ovs_version: "2.0.0"

2.
注意事項
關閉selinux
關閉NetworkManager(必須關閉,不然出莫名其妙的網絡故障)
關閉firewalld

3.
安裝openvswitch(可編譯安裝,這裏使用yum安裝,簡單測試)

yum install openvswitch openvswitch-devel openvswitch-test openvswitch-debuginfor
systemctl enable  openvswitch
systemctl start  openvswitch

4.
創建橋前的網絡情況

創建橋前

[root@nginx-1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
      valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
      valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:80:20:b2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.32.139/24 brd 192.168.32.255 scope global dynamic ens33
      valid_lft 1721sec preferred_lft 1721sec
    inet6 fe80::6d0c:96f7:cbae:c576/64 scope link
      valid_lft forever preferred_lft forever
[root@nginx-1 ~]#

[root@nginx-1 ~]# ip r
default via 192.168.32.2 dev ens33  proto static  metric 100
192.168.32.0/24 dev ens33  proto kernel  scope link  src 192.168.32.132  metric 100
[root@nginx-1 ~]#

5.
ovs基礎知識,參考文檔:https://opengers.github.io/openstack/openstack-base-use-openvswitch/
port類型說明

normal類型
操作系統中已有的網卡(物理網卡或者虛擬機中的虛擬網卡)連接到ovs橋上,
ovs會生成一個port處理這塊網卡的進出數據包.這個端口類型爲normal
掛載到OVS上的網卡設備不支持分配IP地址

internal
internal端口是ovs內部創建的虛擬網卡接口
每創建一個Port,OVS會自動創建一個同名接口(Interface)掛載到新創建的Port上
interface可用來配置ip

patch
多個ovs網橋可用patch port連接,類似於veth pair.
從一個Patch Port收到的數據包會被轉發到另一個Patch Port
patch port總是成對出現,分別連接在兩個網橋上.使用Patch連接的兩個網橋跟一個網橋沒什麼區別

tunnel
OVS中支持添加隧道(Tunnel)端口,常見隧道技術有兩種gre或vxlan.
網絡之上構建一層虛擬網絡,上層應用只與虛擬網絡相關.

Interface
nterface是連接到Port的網絡接口設備,是OVS與外部交換數據包的組件,在通常情況下,
Port和Interface是一對一的關係,只有在配置Port爲 bond模式後,Port和Interface是一對多的關係.

6.
實際操作

虛機1創建橋

[root@nginx-1 ~]# ovs-vsctl add-br br0
[root@nginx-1 ~]#

[root@nginx-1 ~]# ovs-vsctl show
67ccf09b-d0d9-4ccb-9f2f-1ac7918e5c46
    Bridge "br0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.0.0"
[root@nginx-1 ~]#

虛機2創建橋

[root@nginx-3 ~]# ovs-vsctl add-br br0
[root@nginx-3 ~]#

[root@nginx-3 ~]# ovs-vsctl show
25cdb171-59ee-4eb4-a8af-cd93dc460926
    Bridge "br0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.0.0"

橋默認有一個br0端口和br0接口,type類型爲internel
internal端口是ovs內部創建的虛擬網卡接口
每創建一個橋自動創建個類型爲internal的和橋一樣名字的port和interface.

7.
測試思路

虛機1和虛機2的br0各定義同網段ip,測試互通.
虛機1配置br0爲192.168.100.2/24
虛機2配置br0爲192.168.100.3/24

臨時配置命令,見下:

[root@nginx-1 ~]# ifconfig br0 192.168.100.2/24 up

[root@nginx-3 ~]# ifconfig br0 192.168.100.3/24 up

在沒有建立隧道前,網絡是不通的.
虛機1

[root@nginx-1 ~]# ip a |grep br0
5: br0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    inet 192.168.100.2/24 brd 192.168.100.255 scope global br0
[root@nginx-1 ~]# ip r
default via 192.168.32.2 dev ens33
169.254.0.0/16 dev ens33  scope link  metric 1002
192.168.32.0/24 dev ens33  proto kernel  scope link  src 192.168.32.132
192.168.100.0/24 dev br0  proto kernel  scope link  src 192.168.100.2
[root@nginx-1 ~]# ping 192.168.100.2
PING 192.168.100.2 (192.168.100.2) 56(84) bytes of data.
64 bytes from 192.168.100.2: icmp_seq=1 ttl=64 time=0.045 ms
64 bytes from 192.168.100.2: icmp_seq=2 ttl=64 time=0.167 ms
^C
--- 192.168.100.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1042ms
rtt min/avg/max/mdev = 0.045/0.106/0.167/0.061 ms
[root@nginx-1 ~]# ping 192.168.100.3
PING 192.168.100.3 (192.168.100.3) 56(84) bytes of data.
From 192.168.100.2 icmp_seq=1 Destination Host Unreachable
From 192.168.100.2 icmp_seq=2 Destination Host Unreachable
From 192.168.100.2 icmp_seq=3 Destination Host Unreachable
^C
--- 192.168.100.3 ping statistics ---
6 packets transmitted, 0 received, +3 errors, 100% packet loss, time 5138ms
pipe 4
[root@nginx-1 ~]#

虛機2

[root@nginx-3 ~]# ip a|grep br0
5: br0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    inet 192.168.100.3/24 brd 192.168.100.255 scope global br0
[root@nginx-3 ~]# ip r
default via 192.168.32.2 dev ens33
169.254.0.0/16 dev ens33  scope link  metric 1002
169.254.0.0/16 dev br0  scope link  metric 1005
192.168.32.0/24 dev ens33  proto kernel  scope link  src 192.168.32.134
192.168.100.0/24 dev br0  proto kernel  scope link  src 192.168.100.3
[root@nginx-3 ~]# ping 192.168.100.3
PING 192.168.100.3 (192.168.100.3) 56(84) bytes of data.
64 bytes from 192.168.100.3: icmp_seq=1 ttl=64 time=0.066 ms
64 bytes from 192.168.100.3: icmp_seq=2 ttl=64 time=0.095 ms
^C
--- 192.168.100.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1042ms
rtt min/avg/max/mdev = 0.066/0.080/0.095/0.017 ms
[root@nginx-3 ~]# ping 192.168.100.2
PING 192.168.100.2 (192.168.100.2) 56(84) bytes of data.
From 192.168.100.3 icmp_seq=1 Destination Host Unreachable
From 192.168.100.3 icmp_seq=2 Destination Host Unreachable
From 192.168.100.3 icmp_seq=3 Destination Host Unreachable
^C
--- 192.168.100.2 ping statistics ---
5 packets transmitted, 0 received, +3 errors, 100% packet loss, time 4092ms
pipe 4
[root@nginx-3 ~]#

8.
上面只是臨時配置br0的地址,重啓就消失.
可寫入網卡配置文件,永久生效.參考見下:

虛機1

[root@nginx-1 ~]# cd /etc/sysconfig/network-scripts/
[root@nginx-1 network-scripts]# pwd
/etc/sysconfig/network-scripts
[root@nginx-1 network-scripts]# cat ifcfg-br0
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=none
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=br0
DEVICE=br0
ONBOOT=yes
IPADDR=192.168.100.2
NETMASK=255.255.255.0
[root@nginx-1 network-scripts]#

虛機2

[root@nginx-3 ~]# cd /etc/sysconfig/network-scripts/
[root@nginx-3 network-scripts]# cat ifcfg-br0
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=none
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=br0
DEVICE=br0
ONBOOT=yes
IPADDR=192.168.100.3
NETMASK=255.255.255.0
[root@nginx-3 network-scripts]#

9.
創建gre隧道

虛機1
網橋br0添加gre1端口和gre1接口,設置端口類型爲gre,配置遠端Ip爲192.168.32.140,遠端ip爲另一方的ip.

[root@nginx-1 ~]# ovs-vsctl add-port br0 gre1 -- set interface gre1 type=gre option:remote_ip=192.168.32.140

虛機2
網橋br0添加gre1端口和gre1接口,設置端口類型爲gre,配置遠端Ip爲192.168.32.139,遠端ip爲另一方的ip.

[root@nginx-3 ~]# ovs-vsctl add-port br0 gre1 -- set interface gre1 type=gre option:remote_ip=192.168.32.139

ovs信息

虛機1

[root@nginx-1 ~]# ovs-vsctl show
67ccf09b-d0d9-4ccb-9f2f-1ac7918e5c46
    Bridge "br0"
        Port "gre1"
            Interface "gre1"
                type: gre
                options: {remote_ip="192.168.32.140"}
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.0.0"
[root@nginx-1 ~]#

虛機2

[root@nginx-3 ~]# ovs-vsctl show
25cdb171-59ee-4eb4-a8af-cd93dc460926
    Bridge "br0"
        Port "gre1"
            Interface "gre1"
                type: gre
                options: {remote_ip="192.168.32.139"}
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.0.0"
[root@nginx-3 ~]#

隧道建立後,網絡實現互通.

[root@nginx-1 ~]# ip a |grep br0
5: br0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    inet 192.168.100.2/24 brd 192.168.100.255 scope global br0
[root@nginx-1 ~]# ip r
default via 192.168.32.2 dev ens33
169.254.0.0/16 dev ens33  scope link  metric 1002
169.254.0.0/16 dev br0  scope link  metric 1005
192.168.32.0/24 dev ens33  proto kernel  scope link  src 192.168.32.139
192.168.100.0/24 dev br0  proto kernel  scope link  src 192.168.100.2
[root@nginx-1 ~]# ping 192.168.100.3
PING 192.168.100.3 (192.168.100.3) 56(84) bytes of data.
64 bytes from 192.168.100.3: icmp_seq=1 ttl=64 time=3.61 ms
64 bytes from 192.168.100.3: icmp_seq=2 ttl=64 time=1.61 ms
64 bytes from 192.168.100.3: icmp_seq=3 ttl=64 time=0.936 ms
^C
--- 192.168.100.3 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 0.936/2.056/3.613/1.135 ms
[root@nginx-1 ~]#

[root@nginx-3 ~]# ip a |grep br0
5: br0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    inet 192.168.100.3/24 brd 192.168.100.255 scope global br0
[root@nginx-3 ~]# ip r
default via 192.168.32.2 dev ens33
169.254.0.0/16 dev ens33  scope link  metric 1002
169.254.0.0/16 dev br0  scope link  metric 1005
192.168.32.0/24 dev ens33  proto kernel  scope link  src 192.168.32.140
192.168.100.0/24 dev br0  proto kernel  scope link  src 192.168.100.3
[root@nginx-3 ~]# ping 192.168.100.2
PING 192.168.100.2 (192.168.100.2) 56(84) bytes of data.
64 bytes from 192.168.100.2: icmp_seq=1 ttl=64 time=3.59 ms
64 bytes from 192.168.100.2: icmp_seq=2 ttl=64 time=3.38 ms
64 bytes from 192.168.100.2: icmp_seq=3 ttl=64 time=0.469 ms
^C
--- 192.168.100.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.469/2.483/3.596/1.427 ms
[root@nginx-3 ~]#

10.
創建gre通道後系統重啓,br0無法獲取ip.

暫時沒有找到徹底解決這個問題的方法.懷疑是gre本身的問題導致.
臨時測試,可以通過把執行命令寫入rc.local,解決掉這個問題,命令見下:

[root@nginx-3 ~]#
[root@nginx-1 network-scripts]# cat /etc/rc.d/rc.local |grep -v ^#
touch /var/lock/subsys/local
ifdown br0 &&ifup br0                                  ##加這條命令
[root@nginx-1 network-scripts]#

注意必須保證rc.local具有x權限,可執行命令chmod +x /etc/rc.d/rc.local

虛機2做一樣的操作.

11.
互相ping,互相抓包.
必須ping才能抓到包.注意了.
如果沒有tcpdump命令,安裝即可,參考命令:

yum -y install tcpdump

虛機1br0端口抓包icmp

[root@nginx-1 ~]# tcpdump -i br0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:38:48.871368 IP 192.168.100.3 > ovs1: ICMP echo request, id 2235, seq 10, length 64
14:38:48.871442 IP ovs1 > 192.168.100.3: ICMP echo reply, id 2235, seq 10, length 64
14:38:49.873007 IP 192.168.100.3 > ovs1: ICMP echo request, id 2235, seq 11, length 64
14:38:49.873059 IP ovs1 > 192.168.100.3: ICMP echo reply, id 2235, seq 11, length 64
14:38:50.874202 IP 192.168.100.3 > ovs1: ICMP echo request, id 2235, seq 12, length 64
14:38:50.874286 IP ovs1 > 192.168.100.3: ICMP echo reply, id 2235, seq 12, length 64
14:38:51.876210 IP 192.168.100.3 > ovs1: ICMP echo request, id 2235, seq 13, length 64
14:38:51.876272 IP ovs1 > 192.168.100.3: ICMP echo reply, id 2235, seq 13, length 64
14:38:52.878047 IP 192.168.100.3 > ovs1: ICMP echo request, id 2235, seq 14, length 64
14:38:52.878112 IP ovs1 > 192.168.100.3: ICMP echo reply, id 2235, seq 14, length 64
14:38:53.880465 IP 192.168.100.3 > ovs1: ICMP echo request, id 2235, seq 15, length 64
14:38:53.880496 IP ovs1 > 192.168.100.3: ICMP echo reply, id 2235, seq 15, length 64
14:38:54.881610 IP 192.168.100.3 > ovs1: ICMP echo request, id 2235, seq 16, length 64
14:38:54.881681 IP ovs1 > 192.168.100.3: ICMP echo reply, id 2235, seq 16, length 64
^C
14 packets captured
14 packets received by filter
0 packets dropped by kernel

虛機2br0端口抓包icmp

[root@nginx-3 ~]# tcpdump -i br0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:40:13.642693 IP 192.168.100.2 > ovs2: ICMP echo request, id 2253, seq 1, length 64
14:40:13.642725 IP ovs2 > 192.168.100.2: ICMP echo reply, id 2253, seq 1, length 64
14:40:14.642769 IP 192.168.100.2 > ovs2: ICMP echo request, id 2253, seq 2, length 64
14:40:14.642810 IP ovs2 > 192.168.100.2: ICMP echo reply, id 2253, seq 2, length 64
14:40:15.643391 IP 192.168.100.2 > ovs2: ICMP echo request, id 2253, seq 3, length 64
14:40:15.643427 IP ovs2 > 192.168.100.2: ICMP echo reply, id 2253, seq 3, length 64
14:40:16.645941 IP 192.168.100.2 > ovs2: ICMP echo request, id 2253, seq 4, length 64
14:40:16.645992 IP ovs2 > 192.168.100.2: ICMP echo reply, id 2253, seq 4, length 64
14:40:17.647337 IP 192.168.100.2 > ovs2: ICMP echo request, id 2253, seq 5, length 64
14:40:17.647410 IP ovs2 > 192.168.100.2: ICMP echo reply, id 2253, seq 5, length 64
14:40:18.650543 IP 192.168.100.2 > ovs2: ICMP echo request, id 2253, seq 6, length 64
14:40:18.650606 IP ovs2 > 192.168.100.2: ICMP echo reply, id 2253, seq 6, length 64
^C
12 packets captured
12 packets received by filter
0 packets dropped by kernel

虛機1ens33端口抓gre包

[root@nginx-1 ~]#  tcpdump -i ens33 'proto gre' -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
14:46:39.388735 IP 192.168.32.140 > 192.168.32.139: GREv0, length 102: IP 192.168.100.3 > 192.168.100.2: ICMP echo request, id 2271, seq 22, length 64
14:46:39.388786 IP 192.168.32.139 > 192.168.32.140: GREv0, length 102: IP 192.168.100.2 > 192.168.100.3: ICMP echo reply, id 2271, seq 22, length 64
14:46:40.389953 IP 192.168.32.140 > 192.168.32.139: GREv0, length 102: IP 192.168.100.3 > 192.168.100.2: ICMP echo request, id 2271, seq 23, length 64
14:46:40.390074 IP 192.168.32.139 > 192.168.32.140: GREv0, length 102: IP 192.168.100.2 > 192.168.100.3: ICMP echo reply, id 2271, seq 23, length 64
14:46:41.390733 IP 192.168.32.140 > 192.168.32.139: GREv0, length 102: IP 192.168.100.3 > 192.168.100.2: ICMP echo request, id 2271, seq 24, length 64
14:46:41.390852 IP 192.168.32.139 > 192.168.32.140: GREv0, length 102: IP 192.168.100.2 > 192.168.100.3: ICMP echo reply, id 2271, seq 24, length 64
14:46:42.393785 IP 192.168.32.140 > 192.168.32.139: GREv0, length 102: IP 192.168.100.3 > 192.168.100.2: ICMP echo request, id 2271, seq 25, length 64
14:46:42.393947 IP 192.168.32.139 > 192.168.32.140: GREv0, length 102: IP 192.168.100.2 > 192.168.100.3: ICMP echo reply, id 2271, seq 25, length 64
14:46:43.396383 IP 192.168.32.140 > 192.168.32.139: GREv0, length 102: IP 192.168.100.3 > 192.168.100.2: ICMP echo request, id 2271, seq 26, length 64
14:46:43.396545 IP 192.168.32.139 > 192.168.32.140: GREv0, length 102: IP 192.168.100.2 > 192.168.100.3: ICMP echo reply, id 2271, seq 26, length 64
^C
10 packets captured
10 packets received by filter
0 packets dropped by kernel

虛機2ens33端口抓gre包

[root@nginx-3 ~]#  tcpdump -i ens33 'proto gre' -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
14:47:57.785331 IP 192.168.32.139 > 192.168.32.140: GREv0, length 102: IP 192.168.100.2 > 192.168.100.3: ICMP echo request, id 2298, seq 6, length 64
14:47:57.785451 IP 192.168.32.140 > 192.168.32.139: GREv0, length 102: IP 192.168.100.3 > 192.168.100.2: ICMP echo reply, id 2298, seq 6, length 64
14:47:58.789091 IP 192.168.32.139 > 192.168.32.140: GREv0, length 102: IP 192.168.100.2 > 192.168.100.3: ICMP echo request, id 2298, seq 7, length 64
14:47:58.789169 IP 192.168.32.140 > 192.168.32.139: GREv0, length 102: IP 192.168.100.3 > 192.168.100.2: ICMP echo reply, id 2298, seq 7, length 64
14:47:59.787219 IP 192.168.32.139 > 192.168.32.140: GREv0, length 46: ARP, Request who-has 192.168.100.3 tell 192.168.100.2, length 28
14:47:59.787618 IP 192.168.32.140 > 192.168.32.139: GREv0, length 46: ARP, Reply 192.168.100.3 is-at 12:8f:8c:8f:a8:4e, length 28
14:47:59.789472 IP 192.168.32.139 > 192.168.32.140: GREv0, length 102: IP 192.168.100.2 > 192.168.100.3: ICMP echo request, id 2298, seq 8, length 64
14:47:59.789573 IP 192.168.32.140 > 192.168.32.139: GREv0, length 102: IP 192.168.100.3 > 192.168.100.2: ICMP echo reply, id 2298, seq 8, length 64
^C
8 packets captured
8 packets received by filter
0 packets dropped by kernel

由以上抓包可見,配置實現了gre隧道的連通.

ovs實踐1:基礎概念和gre隧道實踐操作

2024年DataOps趨勢預測：AI不會取代數據工程師

雲原生週刊：K8s 中的服務和網絡｜ 2024.4.29

通過Http鏈接地址爬取有贊微信商城商品信息及下載至EXCEL

多人同時導出 Excel 幹崩服務器！新來的阿里大佬給出的解決方案太優雅了！

[轉帖]cpupower

今天，昨天，近七天，近30天，近90天，js封裝

華爲云云原生FinOps解決方案，釋放雲原生最大價值

k8s實踐8:traefik基礎部署(外部訪問kuberntes業務應用)

k8s集羣部署v1.15實踐12:work節點部署kube-proxy

k8s集羣部署v1.15實踐9:部署高可用 kube-scheduler 集羣

k8s實踐4:容器應用配置文件管理利器configmap

k8s實踐18:helm部署安裝grafana配置測試

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結