In the previous post, I described the setup for installing FreeIPA on a VM parallel to the undercloud VM setup by Tripleo Quickstart. The network on the undercloud VM has been setup up by Ironic and Neutron to listen on a network defined for the overcloud. I want to reproduce this on a second machine that is not enrolled in the undercloud. How can I reproduce the steps?
UPDATE:
This is far more complex than necessary. All I needed to do was:
sudo ip addr add 192.0.2.29/24 dev eth1 sudo ip link set eth1 up |
To get connectivity, and persist that info in /etc/sysconfig/network-scripts/ifcfg-eth1
But the OVS “cloning” here is still interesting enough to warrant its own post.
END UPDATE
Using Tripleo Quickstart, I see that the interface I need is created with:
sudo bash -c 'cat < /etc/sysconfig/network-scripts/ifcfg-vlan10 DEVICE=vlan10 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSIntPort BOOTPROTO=static IPADDR=10.0.0.1 NETMASK=255.255.255.0 OVS_BRIDGE=br-ctlplane OVS_OPTIONS="tag=10" EOF' sudo ifup ifcfg-vlan10 |
But My VM does not have an OVS_BRIDGE br-ctlplane defined. How do I create that?
Using the ovs commands, I can look at the bridge definition:
$ sudo ovs-vsctl show 84640eab-ba50-452b-bfe9-615d7b254972 Bridge br-ctlplane Port "vlan10" tag: 10 Interface "vlan10" type: internal Port br-ctlplane Interface br-ctlplane type: internal Port phy-br-ctlplane Interface phy-br-ctlplane type: patch options: {peer=int-br-ctlplane} Port "eth1" Interface "eth1" Bridge br-int fail_mode: secure Port int-br-ctlplane Interface int-br-ctlplane type: patch options: {peer=phy-br-ctlplane} Port br-int Interface br-int type: internal Port "tapacff1724-9f" tag: 1 Interface "tapacff1724-9f" type: internal ovs_version: "2.5.0" |
And that does not exist on the new VM. I’ve been able to deduce that the creation of this bridge happened as a side effect of running
openstack undercloud install |
Since I don’t want an undercloud on my other node, I need to reproduce the OVS commands to build the bridge.
I’m in luck. These commands are all captured in /etc/openvswitch/conf.db I can pull them out with:
grep '^\{' /etc/openvswitch/conf.db | jq '. | ._comment ' | sed -e 's!^\"!!g' -e's!ovs-vsctl:!!' -e 's!\"$!!' | grep -v null > build-ovs.sh |
That gets me:
ovs-vsctl --no-wait -- init -- set Open_vSwitch . db-version=7.12.1 ovs-vsctl --no-wait set Open_vSwitch . ovs-version=2.5.0 \"external-ids:system-id=\\\"a9460ec6-db71-42fb-aec7-a5356bcda153\\\"\" \"system-type=\\\"CentOS\\\"\" \"system-version=\\\"7.2.1511-Core\\\"\" ovs-vsctl -t 10 -- --may-exist add-br br-ctlplane -- set bridge br-ctlplane other-config:hwaddr=00:59:cf:9c:84:3a -- br-set-external-id br-ctlplane bridge-id br-ctlplane ovs-vsctl -t 10 -- --if-exists del-port br-ctlplane eth1 -- add-port br-ctlplane eth1 ovs-vsctl -t 10 -- --if-exists del-port br-ctlplane eth1 -- add-port br-ctlplane eth1 /bin/ovs-vsctl --timeout=10 --oneline --format=json -- --may-exist add-br br-int -- set Bridge br-int datapath_type=system /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set-fail-mode br-int secure /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Bridge br-int protocols=OpenFlow10 /bin/ovs-vsctl --timeout=10 --oneline --format=json -- --may-exist add-br br-ctlplane -- set Bridge br-ctlplane datapath_type=system /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Bridge br-ctlplane protocols=OpenFlow10 /bin/ovs-vsctl --timeout=10 --oneline --format=json -- --may-exist add-port br-int int-br-ctlplane -- set Interface int-br-ctlplane type=patch options:peer=nonexistent-peer /bin/ovs-vsctl --timeout=10 --oneline --format=json -- --may-exist add-port br-ctlplane phy-br-ctlplane -- set Interface phy-br-ctlplane type=patch options:peer=nonexistent-peer /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Interface int-br-ctlplane options:peer=phy-br-ctlplane /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Interface phy-br-ctlplane options:peer=int-br-ctlplane /bin/ovs-vsctl --timeout=10 --oneline --format=json -- add-port br-int tapacff1724-9f -- set Interface tapacff1724-9f type=internal external_ids:iface-id=acff1724-9fb2-4771-a7db-8bd93e7f3833 external_ids:iface-status=active external_ids:attached-mac=fa:16:3e:f6:6d:86 /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Port tapacff1724-9f other_config:physical_network=ctlplane other_config:net_uuid=6dd40444-6cc9-4cfa-bfbd-15b614f6e9e1 other_config:network_type=flat /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Port tapacff1724-9f other_config:tag=1 other_config:physical_network=ctlplane other_config:net_uuid=6dd40444-6cc9-4cfa-bfbd-15b614f6e9e1 other_config:network_type=flat /bin/ovs-vsctl --timeout=10 --oneline --format=json -- set Port tapacff1724-9f tag=1 ovs-vsctl -t 10 -- --may-exist add-port br-ctlplane vlan10 tag=10 -- set Interface vlan10 type=internal |
Now I don’t want to blindly re-execute this, as there are some embedded values particular to the first machine. The MAC 00:59:cf:9c:84:3a for eth1 is reused by the bridge. The first two lines look like system specific setup. Let’s see if the new VM has anything along these lines.
Things to note:
- /etc/openvswitch/ is empty
- systemctl status openvswitch.service show the service is not running
Let’s try starting it:
sudo systemctl start openvswitch.service
grep '^\{' /etc/openvswitch/conf.db | jq '. | ._comment ' | sed -e 's!^\"!!g' -e's!ovs-vsctl:!!' -e 's!\"$!!' | grep -v null ovs-vsctl --no-wait -- init -- set Open_vSwitch . db-version=7.12.1 ovs-vsctl --no-wait set Open_vSwitch . ovs-version=2.5.0 \"external-ids:system-id=\\\"8f68fbfb-9278-4772-87f1-500bc80bb917\\\"\" \"system-type=\\\"CentOS\\\"\" \"system-version=\\\"7.2.1511-Core\\\"\" |
So we can drop those two lines.
Extract the MAC for interface eth1:
ip addr show eth1 3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 00:59:cf:9c:84:3e brd ff:ff:ff:ff:ff:ff |
OK, that is about all we can do. Execute it.
sudo ./build-ovs.sh |
No complaints. What did we get?
$ sudo ovs-vsctl show 1d0cb182-c7c7-4256-8c42-ab013628c2d1 Bridge br-int fail_mode: secure Port "tapacff1724-9f" tag: 1 Interface "tapacff1724-9f" type: internal Port br-int Interface br-int type: internal Port int-br-ctlplane Interface int-br-ctlplane type: patch options: {peer=phy-br-ctlplane} Bridge br-ctlplane Port phy-br-ctlplane Interface phy-br-ctlplane type: patch options: {peer=int-br-ctlplane} Port "vlan10" tag: 10 Interface "vlan10" type: internal Port br-ctlplane Interface br-ctlplane type: internal Port "eth1" Interface "eth1" ovs_version: "2.5.0" |
Looks right.
One thing I notice that is different is that on undercloud, I the bridge has an IP Address:
7: br-ctlplane: mtu 1500 qdisc noqueue state UNKNOWN
link/ether 00:59:cf:9c:84:3a brd ff:ff:ff:ff:ff:ff
inet 192.0.2.1/24 brd 192.0.2.255 scope global br-ctlplane
valid_lft forever preferred_lft forever
inet6 fe80::259:cfff:fe9c:843a/64 scope link
valid_lft forever preferred_lft forever
Let’s add one to the bridge on our new machine:
$ cat /etc/sysconfig/network-scripts/ifcfg-br-ctlplane # This file is autogenerated by os-net-config DEVICE=br-ctlplane ONBOOT=yes HOTPLUG=no NM_CONTROLLED=no PEERDNS=no DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR=192.0.2.1 NETMASK=255.255.255.0 OVS_EXTRA="set bridge br-ctlplane other-config:hwaddr=00:59:cf:9c:84:3a -- br-set-external-id br-ctlplane bridge-id br-ctlplane" |
Again, minor edits, to use proper MAC and a different IP address. Bring it up with:
sudo ifup br-ctlplane |
And we can see it:
$ ip addr show br-ctlplane 7: br-ctlplane: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN link/ether 00:59:cf:9c:84:3e brd ff:ff:ff:ff:ff:ff inet 192.0.2.3/24 brd 192.0.2.255 scope global br-ctlplane valid_lft forever preferred_lft forever inet6 fe80::259:cfff:fe9c:843e/64 scope link valid_lft forever preferred_lft forever |
Last step: we need to bring up the eth1 interface. Again, give it a config file, this time in /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1 ONBOOT=yes HOTPLUG=no NM_CONTROLLED=no PEERDNS=no DEVICETYPE=ovs TYPE=OVSPort OVS_BRIDGE=br-ctlplane BOOTPROTO=none |
And bring it up with :
sudo ifup eth1
Make sure it is up:
$ ip addr show eth1 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP qlen 1000 link/ether 00:59:cf:9c:84:3e brd ff:ff:ff:ff:ff:ff inet6 fe80::259:cfff:fe9c:843e/64 scope link valid_lft forever preferred_lft forever |
And usable:
$ ping 192.0.2.1 PING 192.0.2.1 (192.0.2.1) 56(84) bytes of data. 64 bytes from 192.0.2.1: icmp_seq=1 ttl=64 time=1.41 ms 64 bytes from 192.0.2.1: icmp_seq=2 ttl=64 time=0.627 ms |
I’d really like to laud the Open vSwitch developers for their approach to the database. Having the commands available in the database is a fantastic tool. That is pattern I would love to see emulated elsewhere.
For what you’re trying to do, I’m not sure any of the above is necessary. I wrote up a short “how to connect a vm to the ctlplane network” document here: http://blog.oddbit.com/2016/05/19/connecting-another-vm-to-your-tripleo-qu
If I’ve misunderstood what you’re trying to do, let me know.
It was the fact that the interface was down that tripped me up, which I caught during this more complex undertaking. You are right, and I’ve updated the article.