Uploaded image for project: 'CloudStack'
  1. CloudStack
  2. CLOUDSTACK-4750

bond.VLAN mapping in iptables FORWARD chain not created consistently

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 4.2.0
    • 4.2.1, 4.3.0
    • None
    • Security Level: Public (Anyone can view this level - this is the default.)
    • None
    • CloudStack 4.2, Advanced Zone with Security Groups, XenServer 6.2

    Description

      Create an Advanced Zone with Security Groups.
      Setup multiple subnets using multiple VLANs (e.g. 1230,1231,1232,1233,1234,1235) on a physical network labelled GUEST.
      Run up VM's in each network

      Issue:
      bond.VLAN interface does not consistently get added to the FORWARD chain in iptables preventing connectivity to/from a VM

      e.g. if I run up a machine on VLAN 1233

      Looking through the management-server.log files I see it setting it up:

      [root@csm1 management]# zcat management-server.log.2013-09-26.gz | grep 1233 -A 5 -B 5
      ...
      2013-09-26 18:52:27,850 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-20:null) Creating VIF for i-2-22-VM on nic [Nic:Guest-192.168.3.69-vlan://1233]
      2013-09-26 18:52:27,852 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-20:null) Looking for network named GUEST
      2013-09-26 18:52:27,882 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-20:null) Found a network called GUEST on host=10.1.2.3; Network=ceb6ea91-de34-cf95-5326-f865be6851a2; pif=5884f784-f9ce-58a6-517f-30caa04e67be
      2013-09-26 18:52:27,883 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-20:null) Creating VLAN 1233 on host 10.1.2.3 on device bond2
      2013-09-26 18:52:28,482 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Seq 9-390463667: Response Received:
      2013-09-26 18:52:28,482 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 9-390463667: Received: { Ans: , MgmtId: 345052351047, via: 9, Ver: v1, Flags: 10,

      { GetStorageStatsAnswer } }
      2013-09-26 18:52:28,488 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-427:null) Seq 10-1220149422: Executing request
      2013-09-26 18:52:28,637 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-20:null) VLAN is created for 1233. The uuid is 85d5ad86-40e6-8e6c-e1a6-254ea64df8cd
      2013-09-26 18:52:28,646 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-20:null) Created a vif b57fdf9e-7d90-7689-0eee-9ad550951189 on 0
      2013-09-26 18:52:29,262 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-427:null) Seq 10-1220149422: Response Received:
      2013-09-26 18:52:29,263 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 10-1220149422: Received: { Ans: , MgmtId: 345052351047, via: 10, Ver: v1, Flags: 10, { GetStorageStatsAnswer }

      }
      …

      I inspect the host machine however and see

      Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
      pkts bytes target prot opt in out source destination
      94 7460 BRIDGE-FIREWALL all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond2.1234 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond2.1230 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth2 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond0 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth3 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth6 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth4 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth7 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth10 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond2 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth11 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth9 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth5 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth8 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth1 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth0 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond1 --physdev-is-bridged
      48 2880 DROP all – * * 0.0.0.0/0 0.0.0.0/0

      there should be a rule for bond2.1233.

      If I perform a 'force re-connect' the chain gets correctly updated:

      Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
      pkts bytes target prot opt in out source destination
      94 7460 BRIDGE-FIREWALL all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond2.1233 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond2.1234 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond2.1230 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth2 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond0 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth3 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth6 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth4 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth7 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth10 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond2 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth11 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth9 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth5 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth8 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth1 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out eth0 --physdev-is-bridged
      0 0 ACCEPT all – * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out bond1 --physdev-is-bridged
      48 2880 DROP all – * * 0.0.0.0/0 0.0.0.0/0

      After which I can successfully connect to/from the VM

      In the XenServer SMLog file after running the force re-connect I see

      [root@hypervisor4 log]# grep -i bond2.1233 SMlog -A 5 -B 5
      Sep 27 10:31:27 hypervisor4 SM: [28323] pread SUCCESS
      Sep 27 10:31:27 hypervisor4 SM: [28323] ['/bin/bash', '-c', "iptables -n -L FORWARD | grep 'eth3 '"]
      Sep 27 10:31:27 hypervisor4 SM: [28323] pread SUCCESS
      Sep 27 10:31:27 hypervisor4 SM: [28323] ['/bin/bash', '-c', "iptables -n -L FORWARD | grep 'eth2 '"]
      Sep 27 10:31:27 hypervisor4 SM: [28323] pread SUCCESS
      Sep 27 10:31:27 hypervisor4 SM: [28323] ['/bin/bash', '-c', "iptables -n -L FORWARD | grep 'bond2.1233 '"]
      Sep 27 10:31:27 hypervisor4 SM: [28323] FAILED in util.pread: (rc 1) stdout: '', stderr: ''
      Sep 27 10:31:27 hypervisor4 SM: [28323] ['iptables', '-I', 'FORWARD', '2', '-m', 'physdev', '--physdev-is-bridged', '--physdev-out', 'bond2.1233', '-j', 'ACCEPT']
      Sep 27 10:31:27 hypervisor4 SM: [28323] pread SUCCESS
      Sep 27 10:31:27 hypervisor4 SM: [28323] ['/bin/bash', '-c', "iptables -n -L FORWARD | grep 'eth4 '"]
      Sep 27 10:31:27 hypervisor4 SM: [28323] pread SUCCESS
      Sep 27 10:31:27 hypervisor4 SM: [28323] ['/bin/bash', '-c', "iptables -n -L FORWARD | grep 'eth2 '"]
      Sep 27 10:31:27 hypervisor4 SM: [28323] pread SUCCESS

      There were no other entries in the SMLog file for that vlan, although as you can see from the dates above, the vm was created yesterday and the vif/vlan were pushed to the host at that time.

      Attachments

        Activity

          People

            anthonyxu Anthony Xu
            gerard.lynch Gerard Lynch
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: