Nova與Neutron節點接口架構圖
繼上一期的Nova節點網絡邏輯關係，本期繼續上一期話題，主要講Neutron節點上的組件與接口，繼續引用“邏輯圖”。
1.通過以上圖，從網絡節點（Neutron）入手，通過使用相關命令去查詢一些信息，操作如下：
[root@controller01 ~]# ip a | grep q
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
6: br-int: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN
11: tapf733605f-e2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
12: tapeb25bcee-1b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
13: tap3f46f81b-4f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
18: tap0a0688b4-f8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
27: tapfb9f4bc2-30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
39: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN
40: eth1.13@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
41: eth1.2@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP
42: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
43: tapf36b911d-fa: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
44: tap4ba24208-80: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
從1的命令上可以看到，在這有三個Linux bridge，還有大部分是tap接口，還有一些被隱藏的qr，而其中eth1.2@eth1和eth1.13是使用虛擬網卡技術切片的虛擬網卡，br-ex是出外網絡的網橋，繼續用命令查看
2.繼續查看ovs的接口信息，操作如下：
[root@controller01~]# ovs-vsctl show
2b375334-9ce8-46f2-b5ee-ac60781da7f7
Bridge br-ex
Port "eth1.2"
Interface "eth1.2"
Port "tap4ba24208-80"
Interface "tap4ba24208-80"
Port br-ex
Interface br-ex
type: internal
Bridge br-int
fail_mode: secure
Port "tap0a0688b4-f8"
tag: 1
Interface "tap0a0688b4-f8"
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "tapeb25bcee-1b"
tag: 3
Interface "tapeb25bcee-1b"
Port "tap3f46f81b-4f"
tag: 3
Interface "tap3f46f81b-4f"
Port "tapf733605f-e2"
tag: 4095
Interface "tapf733605f-e2"
Port "tapf36b911d-fa"
tag: 4
Interface "tapf36b911d-fa"
Port "tapfb9f4bc2-30"
tag: 4
Interface "tapfb9f4bc2-30"
Port br-int
Interface br-int
type: internal
Bridge br-tun
fail_mode: secure
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port "vxlan-0af80903"
Interface "vxlan-0af80903"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="10.248.9.2", out_key=flow, remote_ip="10.248.9.3"}
Port "vxlan-0af80905"
Interface "vxlan-0af80905"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="10.248.9.2", out_key=flow, remote_ip="10.248.9.5"}
Port "vxlan-0af80901"
Interface "vxlan-0af80901"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="10.248.9.2", out_key=flow, remote_ip="10.248.9.1"}
Port br-tun
Interface br-tun
type: internal
ovs_version: "2.1.3"
從2上可以看到有很多接口，而此次只針對一個路由器和一個HDCP，還有針對以上三個網橋進行查看。
3.查詢以上兩個網橋配置信息，操作如下：
[root@controller01 ~]# ovs-vsctl list-ports br-int
patch-tun
tap0a0688b4-f8
tap3f46f81b-4f
tapeb25bcee-1b
tapf36b911d-fa
tapf733605f-e2
tapfb9f4bc2-30
[root@controller01 ~]# ovs-vsctl list-ports br-tun
patch-int
vxlan-0af80901
vxlan-0af80903
vxlan-0af80905
[root@controller01 ~]# ovs-vsctl list-ports br-ex
eth1.2
從3上命令可以看到：
A.br-int有很多接口，其中patch-tun是網橋接口，是和br-tun對接的，而tap接口都是DHCP的接口
B.br-tun有三個Vxlan隧道接口，點對點，這是計算節點和網絡節點隧道通訊，而patch-int是網橋接口，是和br-int對接的。
C.br-ex只有一個虛擬的網口eth1.2加入，此網口是通向外部網絡IP地址段：124.56.30.0/27，也就是所以虛擬主機與外部網絡通訊的基礎。
4.根據邏輯圖，繼續查詢qrouter和dhcp，這兩個是網絡命名空間（network namespace）的元素，有空的同學可以瞭解一個，在此只舉一個例子，讓同學們瞭解它們的接口結構，操作如下：
[root@controller01 ~]# ip netns
qdhcp-98daaa5b-3481-44d5-b697-76c8705a2409 -----DHCP的例子
qdhcp-237be48f-e8c0-4f35-93f6-8193005cdb21
qdhcp-50cabb4c-7b6c-44eb-a26b-d240132798c8
qdhcp-141e2f07-4ef0-4f1e-b0a9-1b7113072d8c
qrouter-4d992933-9f4e-4fe6-9507-a75649ef37db -----軟路由的例子
從4以上命令可以查看到上面有四個DHCP和一個軟router，接下繼續分析這兩個網絡命名空間的詳細配置信息。
5.查詢在IP Addr看不到隱藏的端口：qr,qg,ns都可以檢查到了，使用以下指令查詢：
[root@controller01~]# ip netns exec qdhcp-98daaa5b-3481-44d5-b697-76c8705a2409 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ns-f36b911d-fa: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP qlen 1000 ---在IP Add 可以查看到的tap ID 一樣
link/ether fa:16:3e:f4:08:38 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.2/24 brd 172.16.0.255 scope global ns-f36b911d-fa ---這DHCP自有IP
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fef4:838/64 scope link
valid_lft forever preferred_lft forever
[root@controller01 ~]# ip netns exec qrouter-4d992933-9f4e-4fe6-9507-a75649ef37db ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: qr-3f46f81b-4f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP qlen 1000
link/ether fa:16:3e:e6:48:fb brd ff:ff:ff:ff:ff:ff
inet 10.0.0.1/24 brd 10.0.0.255 scope global qr-3f46f81b-4f
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fee6:48fb/64 scope link
valid_lft forever preferred_lft forever
4: qr-fb9f4bc2-30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP qlen 1000 ---到內網的網關
link/ether fa:16:3e:1a:84:f0 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.1/24 brd 172.16.0.255 scope global qr-fb9f4bc2-30
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe1a:84f0/64 scope link
valid_lft forever preferred_lft forever
7: qg-4ba24208-80: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP qlen 1000 ---到外網的網關
link/ether fa:16:3e:d4:19:15 brd ff:ff:ff:ff:ff:ff
inet 124.56.30.119/27 brd 123.58.34.127 scope global qg-4ba24208-80
valid_lft forever preferred_lft forever
inet 124.56.30.118/32 brd 123.58.34.118 scope global qg-4ba24208-80
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fed4:1915/64 scope link
valid_lft forever preferred_lft forever
[root@controller01 ~]# ip netns exec qdhcp-98daaa5b-3481-44d5-b697-76c8705a2409 route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.0.1 0.0.0.0 UG 0 0 0 ns-f36b911d-fa ---在IP Add 可以查看到的tap ID 一樣
172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 ns-f36b911d-fa
[root@controller01 ~]# ip netns exec qrouter-4d992933-9f4e-4fe6-9507-a75649ef37db route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 124.56.30.97 0.0.0.0 UG 0 0 0 qg-4ba24208-80 ---到外網的網關
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-3f46f81b-4f
124.56.30.96 0.0.0.0 255.255.255.224 U 0 0 0 qg-4ba24208-80
172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-fb9f4bc2-30 ---到內網的網關
從5加上以前的信息，現在已經完全清楚Nova與Neutron之間邏輯關係，以及數據流向，qg-4ba24208-80 是向外到外網的路由端口，qr-fb9f4bc2-30 是向內網到虛擬雲主機的路由端口，這兩個端口都是網關端口。 ns-f36b911d-fa實際就是tapf36b911d-fa與br-int連接，所以大家可能假設數據流從instance0開始到外網是怎麼流的，嘗試自己走一遍，基本就知道Openstack的網絡結構。
總結：看完Nova與Neutron的網絡邏輯關係後，相信大家對Openstack的網絡結構有一個初步瞭解，對一些問題排查也有一定作用了，當然我個人覺得整個網絡結構裏面Linux Bridge知識很重要，還有隧道技術，網絡命名空間等技術如果大家去了解，將會幫且我們更好了解Neutron，謝謝！