Uploaded image for project: 'CloudStack'
  1. CloudStack
  2. CLOUDSTACK-10127

4.9 / 4.10 KVM + openvswitch + vpc + static nat / secondary ip on eth2?



    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 4.7.0, 4.8.0, 4.9.0,
    • VPC
    • Security Level: Public (Anyone can view this level - this is the default.)
    • None
    • CentOS 7.4.1708 + KVM + OpenvSwitch 2.3-2.8
    • Important


      We have the following Problem.

      1. KVM
      2. Bridges
      bond with two interfaces and trunk (0,129,180,100-1500) to cloudbr0
      Cloudbr0 (0 - guest network)
      Fakebridge pub129 (public network)
      Fakebridge sto180 (secondary storage network)
      Fakebridge mgmt0 (management)

      If I have a vpc all things work until I add a secondary ip and add a static nat.

      The following will happen, first address will be on the the correct interface but static nat will be on the false network.
      Its on the eth2…

      {{ root@r-29-VM:~# ip a
      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet scope host lo
      2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
      link/ether 0e:00:a9:fe:03:81 brd ff:ff:ff:ff:ff:ff
      inet brd scope global eth0
      3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
      link/ether 1e:00:2c:00:00:68 brd ff:ff:ff:ff:ff:ff
      inet brd scope global eth1
      4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
      link/ether 02:00:57:07:00:0c brd ff:ff:ff:ff:ff:ff
      inet brd scope global eth2
      inet brd scope global eth2}}

      Normally I think the secondary ip should be on signed to eth1 not eth2!
      It sets my ip on the guest network vlan range on my cloudbr0 but it should be pub129. vnet6 has 1353 guest tag and not the public tag.

      [root@kvm01 ~]# ovs-vsctl list-br

      [root@kvm01 ~]# virsh domiflist r-29-VM
      Interface Type Source Model MAC
      vnet4 bridge cloud0 virtio 0e:00:a9:fe:03:81
      vnet5 bridge pub129 virtio 1e:00:2c:00:00:68
      vnet6 bridge cloudbr0 virtio 02:00:57:07:00:0c

      Bridge "cloud0"
      Port "vnet4"
      Interface "vnet4"

      Port "vnet5"
      tag: 129
      Interface "vnet5"
      Port "vnet6"
      tag: 1353
      Interface "vnet6"

      root@r-29-VM:~# cat /etc/cloudstack/ips.json {
      "eth0": [

      { "add": true, "broadcast": "", "cidr": "", "device": "eth0", "gateway": "None", "netmask": "", "network": "", "nic_dev_id": "0", "nw_type": "control", "one_to_one_nat": false, "public_ip": "", "size": "16", "source_nat": false }

      "eth1": [

      { "add": true, "broadcast": "", "cidr": "", "device": "eth1", "first_i_p": true, "gateway": "", "netmask": "", "network": "", "new_nic": false, "nic_dev_id": 1, "nw_type": "public", "one_to_one_nat": false, "public_ip": "", "size": "26", "source_nat": true, "vif_mac_address": "1e:00:2c:00:00:68" }

      "eth2": [

      { "add": true, "broadcast": "", "cidr": "", "device": "eth2", "first_i_p": true, "gateway": "", "netmask": "", "network": "", "new_nic": false, "nic_dev_id": 2, "nw_type": "public", "one_to_one_nat": true, "public_ip": "", "size": "26", "source_nat": true, "vif_mac_address": "1e:00:2c:00:00:68" }


      { "add": false, "broadcast": "", "cidr": "", "device": "eth2", "first_i_p": true, "gateway": "", "netmask": "", "network": "", "new_nic": false, "nic_dev_id": 2, "nw_type": "public", "one_to_one_nat": true, "public_ip": "", "size": "26", "source_nat": true, "vif_mac_address": "1e:00:2c:00:00:68" }


      { "add": true, "broadcast": "", "cidr": "", "device": "eth2", "gateway": "", "netmask": "", "network": "", "nic_dev_id": "2", "nw_type": "guest", "one_to_one_nat": false, "public_ip": "", "size": "24", "source_nat": false }

      "id": "ips"

      Frank Maximus from Nuage analysed the problem.

      That seems to be a bug in the lookup of the device number, in case of openvswitch.
      The config clearly sets device to eth2, while it should be eth1.

      More specifically:
      in LibvirtComputingResource.prepareNetworkElementCommand()
      The broadcastUriToNicNum map is filled depending on the VR nics.
      In openvswitch the guest bridge is used as is, so it overwrites the mapping of public.
      This was not an issue until 4.6 as then VR was using the macaddress to do lookup, while now it is using the device number.

      Kind Regards,

      I hope anyone can fix that fastly.


        Issue Links



              fmaximus Frank Maximus
              sven.vogel Sven Vogel
              1 Vote for this issue
              4 Start watching this issue