Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Implemented
    • Affects Version/s: pre-4.0.0, 4.0.0
    • Fix Version/s: Future
    • Component/s: KVM
    • Security Level: Public (Anyone can view this level - this is the default.)
    • Labels:

      Description

      OVS support currently works for XenServer but not in KVM. This is to improve the KVM agent to support OVS which has been part of the Linux kernel mainline for sometime now.

      Release Planning:
      Dev list discussion: http://markmail.org/thread/x2zncdgac52essxf
      Functional Spec: unknown
      Feature Branch: unknown

      1. agent.log
        8 kB
        angeline shen
      2. agentlog513.doc
        840 kB
        angeline shen
      3. Apache_CloudStack-4 1 0-Installation_Guide-en-US.pdf
        2.18 MB
        angeline shen
      4. cloudstack-agent-err.docx
        18 kB
        angeline shen
      5. cloudstack-agent-out.docx
        15 kB
        angeline shen
      6. management-server.log.2013-05-28.gz
        347 kB
        angeline shen
      7. management-server.log.gz
        284 kB
        angeline shen
      8. management-server.log.gz
        435 kB
        angeline shen

        Activity

        Hide
        Rohit Yadav added a comment -

        Any update on this one? I would like to work on it maybe?

        Show
        Rohit Yadav added a comment - Any update on this one? I would like to work on it maybe?
        Hide
        Rajesh Battala added a comment -

        I won't be able to make it for 4.1.0.
        Post 4.1.0 I can work on this feature.

        Show
        Rajesh Battala added a comment - I won't be able to make it for 4.1.0. Post 4.1.0 I can work on this feature.
        Hide
        Chip Childers added a comment -

        Prasanna - Did you reassign this to yourself because you believe you can complete it for 4.1.0? Or should we shift the fix version to "Future" so that we can bring it into 4.2.0 at a later time?

        Rohit - I notice that you asked if you could help... consider working with Prasanna on this?

        Show
        Chip Childers added a comment - Prasanna - Did you reassign this to yourself because you believe you can complete it for 4.1.0? Or should we shift the fix version to "Future" so that we can bring it into 4.2.0 at a later time? Rohit - I notice that you asked if you could help... consider working with Prasanna on this?
        Hide
        Prasanna Santhanam added a comment -

        Looking to take a stab at it for 4.1.0, but I've reset the version to Future for now.

        Show
        Prasanna Santhanam added a comment - Looking to take a stab at it for 4.1.0, but I've reset the version to Future for now.
        Hide
        Chip Childers added a comment -

        Let's keep it tagged for 4.1.0 for now then, that way we have testing plans in place if you are able to complete it in time. We can always pull it once you get a better feel for how much time you need to be feature complete. If you are certain that you can't do it by the end of the month, then go ahead and re-set to Future.

        Show
        Chip Childers added a comment - Let's keep it tagged for 4.1.0 for now then, that way we have testing plans in place if you are able to complete it in time. We can always pull it once you get a better feel for how much time you need to be feature complete. If you are certain that you can't do it by the end of the month, then go ahead and re-set to Future.
        Hide
        Prasanna Santhanam added a comment -

        Works for me. I'll circle back on list soon as I have a spec drafted

        Show
        Prasanna Santhanam added a comment - Works for me. I'll circle back on list soon as I have a spec drafted
        Hide
        Rohit Yadav added a comment -

        Go Prasanna, hack it up!

        Show
        Rohit Yadav added a comment - Go Prasanna, hack it up!
        Hide
        Chip Childers added a comment -

        Prasanna,

        Thanks for taking this on. For completeness, we need to have a PROPOSAL on the dev list and a functional spec on the wiki. Please update the description field with links to those once started. Also, please name the feature branch that you'll be using in the asf repo for a future MERGE request.

        Thanks!

        Show
        Chip Childers added a comment - Prasanna, Thanks for taking this on. For completeness, we need to have a PROPOSAL on the dev list and a functional spec on the wiki. Please update the description field with links to those once started. Also, please name the feature branch that you'll be using in the asf repo for a future MERGE request. Thanks!
        Hide
        Prasanna Santhanam added a comment -

        Chip, Since I've taken this up only recently after Rajesh indicated he was working on other features I might be a little late on the spec. I plan to start investigation next week on this and will post out to the community about it.

        Show
        Prasanna Santhanam added a comment - Chip, Since I've taken this up only recently after Rajesh indicated he was working on other features I might be a little late on the spec. I plan to start investigation next week on this and will post out to the community about it.
        Hide
        Prasanna Santhanam added a comment -

        As a part of improving Nicira support with KVM, this becomes a subtask. assigning to Hugo

        Show
        Prasanna Santhanam added a comment - As a part of improving Nicira support with KVM, this becomes a subtask. assigning to Hugo
        Hide
        Chip Childers added a comment -

        I notice that the docs are done... but is the feature complete? If so, can the commit-sh from master please be provided?

        Show
        Chip Childers added a comment - I notice that the docs are done... but is the feature complete? If so, can the commit-sh from master please be provided?
        Hide
        Hugo Trippaers added a comment -

        Commits:
        a8fcc4f3d3252291943f016f6f1674d80f96eef6
        9f00302ad3d8229f2e6bd786b3c7749d8c264712
        4267a3fc7ce15e85273eb88ce2c5a8cd2b53f72e
        87fe64695305d6a2c505f757ab7607b765c313b7
        e0ecf7b834d88d8b50064eba0e149e67bdf2de1e
        68523e641fa323e1303bfc7499a524b0475fa55f

        Show
        Hugo Trippaers added a comment - Commits: a8fcc4f3d3252291943f016f6f1674d80f96eef6 9f00302ad3d8229f2e6bd786b3c7749d8c264712 4267a3fc7ce15e85273eb88ce2c5a8cd2b53f72e 87fe64695305d6a2c505f757ab7607b765c313b7 e0ecf7b834d88d8b50064eba0e149e67bdf2de1e 68523e641fa323e1303bfc7499a524b0475fa55f
        Hide
        Hugo Trippaers added a comment -

        Done and dusted (i hope)

        Show
        Hugo Trippaers added a comment - Done and dusted (i hope)
        Hide
        angeline shen added a comment -

        Hi Hugo:

        1. Per https://issues.apache.org/jira/browse/CLOUDSTACK-101
        https://issues.apache.org/jira/browse/CLOUDSTACK-824
        Is the requirement for OS CentOS 6.3 KVM or OS Ubuntu 12.10 KVM?

        I was informed that CentOS 6.3 does not include Openvswitch . Unsure if this means non support for Openvswitch.

        However, I found http://nullworks.wordpress.com/2012/09/19/kvm-and-openvswitch-on-centos-6-3-minimal/
        Which provide procedures for installing KVM and Openvswitch 1.7.1 on CentOS 6.3

        I was informed that Ubuntu 12.10 include Openvswitch.

        2. Per http://nullworks.wordpress.com/2012/09/19/kvm-and-openvswitch-on-centos-6-3-minimal/
        where Openvswitch 1.7.1 works with http://libvirt.org/sources/libvirt-0.10.1.tar.gz

        I tested ACS 4.1 cloudsstack agent (which uses libvirt-0.9.10-21.el6.x86_64) with
        Openvswitch 1.7.1 (which uses libvirt-0.10.1)

        after adding host to Cloudstack management, SSVM failed to start with cloudstack agent errors in
        Agentlog513.doc:

        2013-03-15 00:00:05,273 WARN [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) Failed to start domain: v-2-VM: unsupported configuration: this qemu binary requires libvirt to be compiled with yajl

        2013-03-15 00:00:05,274 WARN [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) Exception

        org.libvirt.LibvirtException: unsupported configuration: this qemu binary requires libvirt to be compiled with yajl

        at org.libvirt.ErrorHandler.processError(Unknown Source)

        at org.libvirt.Connect.processError(Unknown Source)

        at org.libvirt.Domain.processError(Unknown Source)

        at org.libvirt.Domain.create(Unknown Source)

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startDomain(LibvirtComputingResource.java:1035)

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3084)

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1160)

        at com.cloud.agent.Agent.processRequest(Agent.java:525)

        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)

        Cloudstack-agent.out:

        2013-03-15 00:08:05,114

        {GMT} WARN [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5 Failed to start domain: v-2-VM: unsupported configuration: this qemu binary requires libvirt to be compiled with yajl

        2013-03-15 00:08:05,115{GMT}

        WARN [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5 Exception

        org.libvirt.LibvirtException: unsupported configuration: this qemu binary requires libvirt to be compiled with yajl

        at org.libvirt.ErrorHandler.processError(Unknown Source)

        at org.libvirt.Connect.processError(Unknown Source)

        at org.libvirt.Domain.processError(Unknown Source)

        at org.libvirt.Domain.create(Unknown Source)

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startDomain(LibvirtComputingResource.java:1035)

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3084)

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1160)

        at com.cloud.agent.Agent.processRequest(Agent.java:525)

        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)

        at com.cloud.utils.nio.Task.run(Task.java:83)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

        at java.lang.Thread.run(Thread.java:679)

        2013-03-15 00:08:05,115

        {GMT}

        WARN [cloud.agent.Agent] (agentRequest-Handler-5 Caught:

        java.lang.NullPointerException

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupVMNetworks(LibvirtComputingResource.java:4211)

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.handleVmStartFailure(LibvirtComputingResource.java:2980)

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3104)

        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1160)

        at com.cloud.agent.Agent.processRequest(Agent.java:525)

        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)

        at com.cloud.utils.nio.Task.run(Task.java:83)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

        at java.lang.Thread.run(Thread.java:679)

        3. So CentOS 6.3 KVM does not support openvswitch 1.7.1
        attempt to use ASF 4.1 cloudstack-agent using libvirt-0.9.10-21 with openvswitch 1.7.1 using libvirt-0.10 failed

        Attached agentlog503.doc cloudstack-agenterror.doc cloudstack-agent-out.doc

        Show
        angeline shen added a comment - Hi Hugo: 1. Per https://issues.apache.org/jira/browse/CLOUDSTACK-101 https://issues.apache.org/jira/browse/CLOUDSTACK-824 Is the requirement for OS CentOS 6.3 KVM or OS Ubuntu 12.10 KVM? I was informed that CentOS 6.3 does not include Openvswitch . Unsure if this means non support for Openvswitch. However, I found http://nullworks.wordpress.com/2012/09/19/kvm-and-openvswitch-on-centos-6-3-minimal/ Which provide procedures for installing KVM and Openvswitch 1.7.1 on CentOS 6.3 I was informed that Ubuntu 12.10 include Openvswitch. 2. Per http://nullworks.wordpress.com/2012/09/19/kvm-and-openvswitch-on-centos-6-3-minimal/ where Openvswitch 1.7.1 works with http://libvirt.org/sources/libvirt-0.10.1.tar.gz I tested ACS 4.1 cloudsstack agent (which uses libvirt-0.9.10-21.el6.x86_64) with Openvswitch 1.7.1 (which uses libvirt-0.10.1) after adding host to Cloudstack management, SSVM failed to start with cloudstack agent errors in Agentlog513.doc: 2013-03-15 00:00:05,273 WARN [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) Failed to start domain: v-2-VM: unsupported configuration: this qemu binary requires libvirt to be compiled with yajl 2013-03-15 00:00:05,274 WARN [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) Exception org.libvirt.LibvirtException: unsupported configuration: this qemu binary requires libvirt to be compiled with yajl at org.libvirt.ErrorHandler.processError(Unknown Source) at org.libvirt.Connect.processError(Unknown Source) at org.libvirt.Domain.processError(Unknown Source) at org.libvirt.Domain.create(Unknown Source) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startDomain(LibvirtComputingResource.java:1035) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3084) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1160) at com.cloud.agent.Agent.processRequest(Agent.java:525) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) Cloudstack-agent.out: 2013-03-15 00:08:05,114 {GMT} WARN [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5 Failed to start domain: v-2-VM: unsupported configuration: this qemu binary requires libvirt to be compiled with yajl 2013-03-15 00:08:05,115{GMT} WARN [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5 Exception org.libvirt.LibvirtException: unsupported configuration: this qemu binary requires libvirt to be compiled with yajl at org.libvirt.ErrorHandler.processError(Unknown Source) at org.libvirt.Connect.processError(Unknown Source) at org.libvirt.Domain.processError(Unknown Source) at org.libvirt.Domain.create(Unknown Source) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startDomain(LibvirtComputingResource.java:1035) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3084) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1160) at com.cloud.agent.Agent.processRequest(Agent.java:525) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) at com.cloud.utils.nio.Task.run(Task.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2013-03-15 00:08:05,115 {GMT} WARN [cloud.agent.Agent] (agentRequest-Handler-5 Caught: java.lang.NullPointerException at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupVMNetworks(LibvirtComputingResource.java:4211) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.handleVmStartFailure(LibvirtComputingResource.java:2980) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3104) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1160) at com.cloud.agent.Agent.processRequest(Agent.java:525) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) at com.cloud.utils.nio.Task.run(Task.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 3. So CentOS 6.3 KVM does not support openvswitch 1.7.1 attempt to use ASF 4.1 cloudstack-agent using libvirt-0.9.10-21 with openvswitch 1.7.1 using libvirt-0.10 failed Attached agentlog503.doc cloudstack-agenterror.doc cloudstack-agent-out.doc
        Hide
        angeline shen added a comment -

        Also tried testing with openswitch 1.7.1 running with libvirt-0.9.10-21 .

        After serivce openswitch start, the host network connection was completely lost.
        The host can no longer even ping www.google.com or anywhere.

        So this combination does not work either.

        Show
        angeline shen added a comment - Also tried testing with openswitch 1.7.1 running with libvirt-0.9.10-21 . After serivce openswitch start, the host network connection was completely lost. The host can no longer even ping www.google.com or anywhere. So this combination does not work either.
        Hide
        angeline shen added a comment -

        Hi Hugo:

        Thank you for providing Apache_Cloudstack-4.1.0-Installation_Guide.

        In the process of testing Ubuntu 12.04 ASF 4.1 cloudstack with openvswitch,
        I am currently blocked by https://issues.apache.org/jira/browse/CLOUDSTACK-1736.

        We need to have Ubuntu 12.04 ASF 4.1 cloudstack running as mentioned in Installation Guide.

        Show
        angeline shen added a comment - Hi Hugo: Thank you for providing Apache_Cloudstack-4.1.0-Installation_Guide. In the process of testing Ubuntu 12.04 ASF 4.1 cloudstack with openvswitch, I am currently blocked by https://issues.apache.org/jira/browse/CLOUDSTACK-1736 . We need to have Ubuntu 12.04 ASF 4.1 cloudstack running as mentioned in Installation Guide.
        Hide
        Sudha Ponnaganti added a comment - - edited

        Looks like ubuntu is blocked. Is this real blocker or env issue or documentation issue??

        Show
        Sudha Ponnaganti added a comment - - edited Looks like ubuntu is blocked. Is this real blocker or env issue or documentation issue??
        Hide
        angeline shen added a comment -

        Successful in configuring setup for Ubuntu 12.04.02
        MS Ubuntu 12.04.2 ASF 4.1 latest build 4/8/13
        host Ubuntu 12.04.2 ASF 4.1 latest build 4/8/13 libvirtd 0.10.2 openvswitch

        Currently conducting sanity testing

        Issue: https://issues.apache.org/jira/browse/CLOUDSTACK-1978

        Show
        angeline shen added a comment - Successful in configuring setup for Ubuntu 12.04.02 MS Ubuntu 12.04.2 ASF 4.1 latest build 4/8/13 host Ubuntu 12.04.2 ASF 4.1 latest build 4/8/13 libvirtd 0.10.2 openvswitch Currently conducting sanity testing Issue: https://issues.apache.org/jira/browse/CLOUDSTACK-1978
        Hide
        angeline shen added a comment -

        RHEL 6.3 (CentOS 6.3) openvswitch test configuration setup

        Test scanrio 1:
        After completing http://nullworks.wordpress.com/2012/09/19/kvm-and-openvswitch-on-centos-6-3-minimal/
        to setup RHEL 6.3 openvswitch, encountered following problems:

        1. In this scenario, after service openvswitch start, host is still up running but completely lost connection to network.

        [root@Rack3Host16 /]# service openvswitch start
        Inserting brcompat module [ OK ]
        Starting ovsdb-server [ OK ]
        Configuring Open vSwitch system IDs [ OK ]
        Starting ovs-vswitchd [ OK ]
        Starting ovs-brcompatd [ OK ]
        iptables already has a rule for gre, not explicitly enabling.
        [root@Rack3Host16 /]#

        2. content of various files:

        /etc/sysconfig/network-scripts/ifcfg-ovsbr1 :
        DEVICE=ovsbr1
        ONBOOT=yes
        DEVICETYPE=ovs
        TYPE=OVSBridge
        BOOTPROTO=static
        IPADDR=10.223.58.195
        GATEWAY=10.223.58.193
        NETMASK=255.255.255.0
        HOTPLUG=no

        ifconfig output:

        em2 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3D

        BROADCAST MULTICAST MTU:1500 Metric:1

        RX packets:0 errors:0 dropped:0 overruns:0 frame:0

        TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

        collisions:0 txqueuelen:1000

        RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

        Interrupt:17 Memory:dc000000-dc012800

        eth1 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C

        inet addr:10.223.58.195 Bcast:10.223.58.255 Mask:255.255.255.192

        inet6 addr: fc00:2::be30:5bff:fed4:163c/64 Scope:Global

        inet6 addr: fe80::be30:5bff:fed4:163c/64 Scope:Link

        UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1

        RX packets:12169 errors:0 dropped:0 overruns:0 frame:0

        TX packets:45 errors:0 dropped:0 overruns:0 carrier:0

        collisions:0 txqueuelen:1000

        RX bytes:10750397 (10.2 MiB) TX bytes:6661 (6.5 KiB)

        Interrupt:16 Memory:da000000-da012800

        lo Link encap:Local Loopback

        inet addr:127.0.0.1 Mask:255.0.0.0

        inet6 addr: ::1/128 Scope:Host

        UP LOOPBACK RUNNING MTU:16436 Metric:1

        RX packets:14 errors:0 dropped:0 overruns:0 frame:0

        TX packets:14 errors:0 dropped:0 overruns:0 carrier:0

        collisions:0 txqueuelen:0

        RX bytes:1064 (1.0 KiB) TX bytes:1064 (1.0 KiB)

        ovsbr1 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C

        BROADCAST MULTICAST MTU:1500 Metric:1

        RX packets:11826 errors:0 dropped:0 overruns:0 frame:0

        TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

        collisions:0 txqueuelen:0

        RX bytes:10653029 (10.1 MiB) TX bytes:0 (0.0 b)

        virbr0 Link encap:Ethernet HWaddr 4A:29:63:30:C6:48

        BROADCAST MULTICAST MTU:1500 Metric:1

        RX packets:0 errors:0 dropped:0 overruns:0 frame:0

        TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

        collisions:0 txqueuelen:0

        RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

        3. traceroute no longer works

        Show
        angeline shen added a comment - RHEL 6.3 (CentOS 6.3) openvswitch test configuration setup Test scanrio 1: After completing http://nullworks.wordpress.com/2012/09/19/kvm-and-openvswitch-on-centos-6-3-minimal/ to setup RHEL 6.3 openvswitch, encountered following problems: 1. In this scenario, after service openvswitch start, host is still up running but completely lost connection to network. [root@Rack3Host16 /] # service openvswitch start Inserting brcompat module [ OK ] Starting ovsdb-server [ OK ] Configuring Open vSwitch system IDs [ OK ] Starting ovs-vswitchd [ OK ] Starting ovs-brcompatd [ OK ] iptables already has a rule for gre, not explicitly enabling. [root@Rack3Host16 /] # 2. content of various files: /etc/sysconfig/network-scripts/ifcfg-ovsbr1 : DEVICE=ovsbr1 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR=10.223.58.195 GATEWAY=10.223.58.193 NETMASK=255.255.255.0 HOTPLUG=no ifconfig output: em2 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3D BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:17 Memory:dc000000-dc012800 eth1 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C inet addr:10.223.58.195 Bcast:10.223.58.255 Mask:255.255.255.192 inet6 addr: fc00:2::be30:5bff:fed4:163c/64 Scope:Global inet6 addr: fe80::be30:5bff:fed4:163c/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:12169 errors:0 dropped:0 overruns:0 frame:0 TX packets:45 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:10750397 (10.2 MiB) TX bytes:6661 (6.5 KiB) Interrupt:16 Memory:da000000-da012800 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:14 errors:0 dropped:0 overruns:0 frame:0 TX packets:14 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1064 (1.0 KiB) TX bytes:1064 (1.0 KiB) ovsbr1 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:11826 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:10653029 (10.1 MiB) TX bytes:0 (0.0 b) virbr0 Link encap:Ethernet HWaddr 4A:29:63:30:C6:48 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) 3. traceroute no longer works
        Hide
        Hugo Trippaers added a comment -

        Hey Angelina,

        Did you configure the physical network adapter as a ovs bridge port?

        if for example eth0 is your physical nic, you need to set
        DEVICETYPE=ovs
        TYPE=OVSPort
        OVS_BRIDGE=ovsbr1

        otherwise the physical interface will not be connected to the bridge and you will lose connectivity.

        You can check with "ovs-vsctl list-ports ovsbr1" , that command should list your physical interface.

        Feel free to ping me if you see me on IRC so i can help you directly.

        Show
        Hugo Trippaers added a comment - Hey Angelina, Did you configure the physical network adapter as a ovs bridge port? if for example eth0 is your physical nic, you need to set DEVICETYPE=ovs TYPE=OVSPort OVS_BRIDGE=ovsbr1 otherwise the physical interface will not be connected to the bridge and you will lose connectivity. You can check with "ovs-vsctl list-ports ovsbr1" , that command should list your physical interface. Feel free to ping me if you see me on IRC so i can help you directly.
        Hide
        angeline shen added a comment -

        Hugo:
        questions Pertain to http://nullworks.wordpress.com/2012/09/19/kvm-and-openvswitch-on-centos-6-3-minimal/ :

        1. Preparing the host vefore adding to cloudstack:
        follow line 1 through line 87 of above link which include :
        starting libvirtd, get and install openvswitch, build & install openvswitch, copy sysconfig files,

        line 89 - 98:

        #create bridge file

        vi /etc/sysconfig/network-scripts/ifcfg-ovsbr1 ......

        question: Does part of bridge filename after the '-' in ifcfg-ovsbr1 , that is 'ovsbr1' need to match with physical interface used in ovs-vsctl commands which are to be executed later after 'service openvswitch start' is executed?

        For example if eth1 is physical nic, content of /etc/sysconfig/network-scripts/ifcfg-eth1 is:

        DEVICE=eth1
        BOOTPROTO=none
        HWADDR=BC:30:5B:D4:16:3C
        IPV6INIT=no
        #MTU=1500
        NM_CONTROLLED=yes
        ONBOOT=yes
        TYPE=Ethernet
        UUID="0142ba15-de11-499c-a1ba-82837b30f890"
        IPADDR=10.223.58.195
        NETMASK=255.255.255.192
        GATEWAY=10.223.58.193
        DNS1=10.223.110.254
        USERCTL=yes
        IPV4_FAILURE_FATAL=yes
        DEFROUTE=yes
        NAME="System eth1"

        > vi /etc/sysconfig/network-scripts/ifcfg-br0 ??? here I used 'br0'
        DEVICE=br0
        ONBOOT=yes
        DEVICETYPE=ovs
        TYPE=OVSPort ??? in the document, TYPE=
        OVS_BRIDGE=br0
        BOOTPROTO=static
        IPADDR= <Host IPaddress>
        GATEWAY = <gateway IPAddress>
        NETMASK=255.255.255.0
        HOTPLUG=no

        > rmmod bridge

        > vi /etc/modprobe.d/blacklist.conf & append following at end of file
        blacklist bridge

        > service openvswitch start

        > iptables -A POSTROUTING -o eth0 -j MASQUERADE
        > iptables -A INPUT -i ovsbr1 -j ACCEPT
        > iptables -A FORWARD -i ovsbr1 -j ACCE
        > iptables -I INPUT -p tcp -m tcp --dport 22 -j ACCEPT
        > iptables -I INPUT -p tcp -m tcp --dport 1798 -j ACCEPT
        > iptables -I INPUT -p tcp -m tcp --dport 16509 -j ACCEPT
        > iptables -I INPUT -p tcp -m tcp --dport 5900:6100 -j ACCEPT
        > iptables -I INPUT -p tcp -m tcp --dport 49152:49216 -j ACCEPT
        > iptables-save > /etc/sysconfig/iptables
        > service iptables save

        > vi /etc/sysconfig/iptables & comment following line

        #-A FORWARD -j REJECT --reject-with icmp-host-prohibited

        > service iptables restart

        > ovs-vsctl add-br br0 ??? br0 to match /etc/sysconfig/network-scripts/ifcfg-br0 ?

        > ovs-vsctl add-port br0 eth1

        question: are above steps in correct order to prepare host before adding host to cloudstack?

        2. Add host to cloudstack

        THANKS

        Show
        angeline shen added a comment - Hugo: questions Pertain to http://nullworks.wordpress.com/2012/09/19/kvm-and-openvswitch-on-centos-6-3-minimal/ : 1. Preparing the host vefore adding to cloudstack: follow line 1 through line 87 of above link which include : starting libvirtd, get and install openvswitch, build & install openvswitch, copy sysconfig files, line 89 - 98: #create bridge file vi /etc/sysconfig/network-scripts/ifcfg-ovsbr1 ...... question: Does part of bridge filename after the '-' in ifcfg-ovsbr1 , that is 'ovsbr1' need to match with physical interface used in ovs-vsctl commands which are to be executed later after 'service openvswitch start' is executed? For example if eth1 is physical nic, content of /etc/sysconfig/network-scripts/ifcfg-eth1 is: DEVICE=eth1 BOOTPROTO=none HWADDR=BC:30:5B:D4:16:3C IPV6INIT=no #MTU=1500 NM_CONTROLLED=yes ONBOOT=yes TYPE=Ethernet UUID="0142ba15-de11-499c-a1ba-82837b30f890" IPADDR=10.223.58.195 NETMASK=255.255.255.192 GATEWAY=10.223.58.193 DNS1=10.223.110.254 USERCTL=yes IPV4_FAILURE_FATAL=yes DEFROUTE=yes NAME="System eth1" > vi /etc/sysconfig/network-scripts/ifcfg-br0 ??? here I used 'br0' DEVICE=br0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSPort ??? in the document, TYPE= OVS_BRIDGE=br0 BOOTPROTO=static IPADDR= <Host IPaddress> GATEWAY = <gateway IPAddress> NETMASK=255.255.255.0 HOTPLUG=no > rmmod bridge > vi /etc/modprobe.d/blacklist.conf & append following at end of file blacklist bridge > service openvswitch start > iptables -A POSTROUTING -o eth0 -j MASQUERADE > iptables -A INPUT -i ovsbr1 -j ACCEPT > iptables -A FORWARD -i ovsbr1 -j ACCE > iptables -I INPUT -p tcp -m tcp --dport 22 -j ACCEPT > iptables -I INPUT -p tcp -m tcp --dport 1798 -j ACCEPT > iptables -I INPUT -p tcp -m tcp --dport 16509 -j ACCEPT > iptables -I INPUT -p tcp -m tcp --dport 5900:6100 -j ACCEPT > iptables -I INPUT -p tcp -m tcp --dport 49152:49216 -j ACCEPT > iptables-save > /etc/sysconfig/iptables > service iptables save > vi /etc/sysconfig/iptables & comment following line #-A FORWARD -j REJECT --reject-with icmp-host-prohibited > service iptables restart > ovs-vsctl add-br br0 ??? br0 to match /etc/sysconfig/network-scripts/ifcfg-br0 ? > ovs-vsctl add-port br0 eth1 question: are above steps in correct order to prepare host before adding host to cloudstack? 2. Add host to cloudstack THANKS
        Hide
        angeline shen added a comment -

        Continuing from previous question. Let me clarify question about line 89-98 due to some typos:

        line 89 - 98:

        #create bridge file

        vi /etc/sysconfig/network-scripts/ifcfg-ovsbr1 ......

        question:
        1. Does part of bridge filename after the '-' in ifcfg-ovsbr1 , that is 'ovsbr1' need to match with physical interface used in ovs-vsctl commands which are to be executed later after 'service openvswitch start' is executed?

        2. Must bridge filename be 'ifcfg-ovsbr1' or can it be named ifcfg-<anything> as long as cloudstack UI
        'KVM traffic label' use the same <anything> as label?

        For example, if bridge filename is 'ifcfg-ovsbr1', then 'ovsbr1' is must be used in cloudstack UI
        public network 'KVM traffic label', guest network 'KVM traffic label', management network 'KVM traffic label' ?

        For example if eth1 is physical nic, content of /etc/sysconfig/network-scripts/ifcfg-eth1 is:

        DEVICE=eth1
        BOOTPROTO=none
        HWADDR=BC:30:5B:D4:16:3C
        IPV6INIT=no
        #MTU=1500
        NM_CONTROLLED=yes
        ONBOOT=yes
        TYPE=Ethernet
        UUID="0142ba15-de11-499c-a1ba-82837b30f890"
        IPADDR=10.223.58.195
        NETMASK=255.255.255.192
        GATEWAY=10.223.58.193
        DNS1=10.223.110.254
        USERCTL=yes
        IPV4_FAILURE_FATAL=yes
        DEFROUTE=yes
        NAME="System eth1"

        > vi /etc/sysconfig/network-scripts/ifcfg-br0 ??? here if I use 'br0' instead of 'ovsbr1'
        DEVICE=br0
        ONBOOT=yes
        DEVICETYPE=ovs
        TYPE=OVSPort ??? in document, TYPE=OVSBridge
        OVS_BRIDGE=br0
        BOOTPROTO=static
        IPADDR= 10.223.58.195 ??? same as ifcfg-eth1 above
        GATEWAY = 10.223.58.193 ??? same as ifcfg-eth1 above
        NETMASK=255.255.255.192 ??? same as ifcfg-eth1 above
        HOTPLUG=no

        Then 'ovsbr1' is to be used in cloudstack UI public network 'KVM traffic label',
        guest network 'KVM traffic label', management network 'KVM traffic label' ?

        Show
        angeline shen added a comment - Continuing from previous question. Let me clarify question about line 89-98 due to some typos: line 89 - 98: #create bridge file vi /etc/sysconfig/network-scripts/ifcfg-ovsbr1 ...... question: 1. Does part of bridge filename after the '-' in ifcfg-ovsbr1 , that is 'ovsbr1' need to match with physical interface used in ovs-vsctl commands which are to be executed later after 'service openvswitch start' is executed? 2. Must bridge filename be 'ifcfg-ovsbr1' or can it be named ifcfg-<anything> as long as cloudstack UI 'KVM traffic label' use the same <anything> as label? For example, if bridge filename is 'ifcfg-ovsbr1', then 'ovsbr1' is must be used in cloudstack UI public network 'KVM traffic label', guest network 'KVM traffic label', management network 'KVM traffic label' ? For example if eth1 is physical nic, content of /etc/sysconfig/network-scripts/ifcfg-eth1 is: DEVICE=eth1 BOOTPROTO=none HWADDR=BC:30:5B:D4:16:3C IPV6INIT=no #MTU=1500 NM_CONTROLLED=yes ONBOOT=yes TYPE=Ethernet UUID="0142ba15-de11-499c-a1ba-82837b30f890" IPADDR=10.223.58.195 NETMASK=255.255.255.192 GATEWAY=10.223.58.193 DNS1=10.223.110.254 USERCTL=yes IPV4_FAILURE_FATAL=yes DEFROUTE=yes NAME="System eth1" > vi /etc/sysconfig/network-scripts/ifcfg-br0 ??? here if I use 'br0' instead of 'ovsbr1' DEVICE=br0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSPort ??? in document, TYPE=OVSBridge OVS_BRIDGE=br0 BOOTPROTO=static IPADDR= 10.223.58.195 ??? same as ifcfg-eth1 above GATEWAY = 10.223.58.193 ??? same as ifcfg-eth1 above NETMASK=255.255.255.192 ??? same as ifcfg-eth1 above HOTPLUG=no Then 'ovsbr1' is to be used in cloudstack UI public network 'KVM traffic label', guest network 'KVM traffic label', management network 'KVM traffic label' ?
        Hide
        Hugo Trippaers added a comment -

        Hey Angeline,

        The name of the bridge is indeed the traffic label in KVM. So in your previous comment you have br0 as name for the bridge interface. If this bridge is for management traffic you need to configure the ip address on the bridge, so the typical configuration is as follows:

        1. The physical interface: eth1
          DEVICE=eth1
          BOOTPROTO=none
          HWADDR=BC:30:5B:D4:16:3C
          IPV6INIT=no
          #MTU=1500
          NM_CONTROLLED=no
          ONBOOT=yes
          TYPE=OVSPort
          DEVICETYPE=ovs
          OVS_BRIDGE=br0
        1. The bridge interface: br0
          DEVICE=br0
          ONBOOT=yes
          DEVICETYPE=ovs
          TYPE=OVSBridge
          BOOTPROTO=static
          IPADDR= 10.223.58.195
          GATEWAY = 10.223.58.193
          NETMASK=255.255.255.192
          HOTPLUG=no

        With this configurations you will have one bridge 'br0' with one connected port 'eth1'

        Show
        Hugo Trippaers added a comment - Hey Angeline, The name of the bridge is indeed the traffic label in KVM. So in your previous comment you have br0 as name for the bridge interface. If this bridge is for management traffic you need to configure the ip address on the bridge, so the typical configuration is as follows: The physical interface: eth1 DEVICE=eth1 BOOTPROTO=none HWADDR=BC:30:5B:D4:16:3C IPV6INIT=no #MTU=1500 NM_CONTROLLED=no ONBOOT=yes TYPE=OVSPort DEVICETYPE=ovs OVS_BRIDGE=br0 The bridge interface: br0 DEVICE=br0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR= 10.223.58.195 GATEWAY = 10.223.58.193 NETMASK=255.255.255.192 HOTPLUG=no With this configurations you will have one bridge 'br0' with one connected port 'eth1'
        Hide
        angeline shen added a comment -

        Hugo:

        Still FAILING configuration. I AM STILL GROPING IN THE DARK AS THERE IS NO DOCUMENTATION on
        how to configure openvswitch for CENTOS properly.

        There is only scattered documentation, I have lots of confusion on how to correctly configure host to make this work.
        Our developer thinks there is bug on openstack openvswitch side.
        Here is what I have done on the host configuration so far.
        Your previous comment of following FAILED:
        DEVICETYPE=ovs
        TYPE=OVSPort
        OVS_BRIDGE=ovsbr1

        My problem is I am getting little bits of scattered info of various configuration files one at a time, This will take forever to get this working. There is only scattered docs about this configuration.

        For this round of testing FAILURES, here are all the files on the host:

        I used 'cloudbr' instead of 'br0' . I am still confused about this device name how it is tied together with command:
        ovs-vsctl add-br cloudbr

        Host: 10.223.58.195

        1. /etc/sysconfig/ifcfg-eth1:

        DEVICE=eth1
        BOOTPROTO=none
        HWADDR=BC:30:5B:D4:16:3C
        IPV6INIT=no
        #MTU=1500
        NM_CONTROLLED=yes
        ONBOOT=yes
        TYPE=Ethernet
        UUID="0142ba15-de11-499c-a1ba-82837b30f890"
        IPADDR=10.223.58.195
        NETMASK=255.255.255.192
        GATEWAY=10.223.58.193
        DNS1=10.223.110.254
        USERCTL=yes
        IPV4_FAILURE_FATAL=yes
        DEFROUTE=yes
        NAME="System eth1"

        QUESTION: I don't have the line OVS_BRIDGE=cloudbr in /etc/sysconfig/ifcfg-eth1. Do I need this ?
        Does this mean I have to reinstall the host & reconfigure again at this time ?
        THIS IS ANOTHER EXAMPLE of HOW I AM GETTING only little bits of info one at a time.

        CAN YOU PLEASE GIVE ME CONCRETE EXAMPLE OF ALL THE FILES AND COMMANDS ON THE HOST ON HOW TO configure openswitch on CENTOS KVM?
        I HAVE BEEN REINSTALLING AND RECONFIGURING HOST, groping in the dark, each time discovering &
        experimenting on my own to move forward a bit. IT's taking too long.....

        2. /etc/sysconfig/ifcfg-cloudbr :

        DEVICE=cloudbr
        ONBOOT=yes
        DEVICETYPE=ovs
        #TYPE=OVSPort
        OVS_BRIDGE=cloudbr
        TYPE=OVSBridge
        BOOTPROTO=none
        IPADDR=10.223.58.195
        GATEWAY=10.23.58.193
        NETMASK=255.255.255.0
        HOTPLUG=no

        QUESTION: Originally, I had TYPE=OVSPort. After I executed following commands per
        Apache_CloudStack-410-installation_Guide-en-US.pdf section 8.1.8.3 page 75-76

        ovs-vsctl add-br cloudbr
        ovs-vsctl add-port cloudbr eth1

        The host CAN NO LONGER ping gateway, www.google.com.

        Then on the host, I commented out TYPE=OVSPort & reentered TYPE=OVSBridge. Please see above content.
        I did 'service network restart'. After several minutes, the HOST ABLE TO ping gateway, google, the WORLD..

        WHAT A DISCOVERY !!!!!

        BUT I AM STILL ON THE WRONG TRACK , I KNOW.

        3. brctl show

        [root@Rack3Host16 agent]# brctl show
        bridge name bridge id STP enabled interfaces
        cloud0 /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        0000.aa81045f5e4a no
        cloudbr /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        0000.2213344e8047 no

        QUESTION: OUR DEVELOPER edison.su@citrix.com says this is BUG IN OPENVSWITCH.

        4. /etc/cloudstack/agent/agent.properties:

        #Storage
        #Wed Apr 24 17:35:17 PDT 2013
        guest.network.device=cloudbr
        workers=5
        private.network.device=cloudbr
        port=8250
        resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource
        pod=1
        zone=1
        guid=be58e605-fec5-3ab0-8afc-6491a99baec3
        public.network.device=cloudbr
        cluster=1
        local.storage.uuid=75be87c8-ee41-491b-b261-86d78929e8e6
        domr.scripts.dir=scripts/network/domr/kvm
        LibvirtComputingResource.id=0
        host=10.223.195.114
        ~

        5. [root@Rack3Host16 agent]# ifconfig
        cloud0 Link encap:Ethernet HWaddr AA:81:04:5F:5E:4A
        inet addr:169.254.0.1 Bcast:169.254.255.255 Mask:255.255.0.0
        inet6 addr: fe80::a881:4ff:fe5f:5e4a/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:0 errors:0 dropped:0 overruns:0 frame:0
        TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:0 (0.0 b) TX bytes:4201 (4.1 KiB)

        cloudbr Link encap:Ethernet HWaddr 22:13:34:4E:80:47
        inet addr:10.223.58.195 Bcast:10.223.58.255 Mask:255.255.255.0
        inet6 addr: fe80::2013:34ff:fe4e:8047/64 Scope:Link
        UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
        RX packets:0 errors:0 dropped:0 overruns:0 frame:0
        TX packets:27 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:0 (0.0 b) TX bytes:4385 (4.2 KiB)

        eth1 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C
        inet addr:10.223.58.195 Bcast:10.223.58.255 Mask:255.255.255.192
        inet6 addr: fc00:2::be30:5bff:fed4:163c/64 Scope:Global
        inet6 addr: fe80::be30:5bff:fed4:163c/64 Scope:Link
        UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
        RX packets:2413713 errors:0 dropped:0 overruns:0 frame:0
        TX packets:116725 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:1000
        RX bytes:2287395217 (2.1 GiB) TX bytes:22431834 (21.3 MiB)
        Interrupt:16 Memory:da000000-da012800

        lo Link encap:Local Loopback
        inet addr:127.0.0.1 Mask:255.0.0.0
        inet6 addr: ::1/128 Scope:Host
        UP LOOPBACK RUNNING MTU:16436 Metric:1
        RX packets:78 errors:0 dropped:0 overruns:0 frame:0
        TX packets:78 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:10536 (10.2 KiB) TX bytes:10536 (10.2 KiB)

        6. [root@Rack3Host16 ~]# lsmod | grep bridge

        [root@Rack3Host16 ~]# lsmod | grep kvm
        kvm_intel 52570 0
        kvm 314739 1 kvm_intel

        [root@Rack3Host16 ~]# [root@Rack3Host16 ~]# lsmod | grep brcom
        brcompat 5873 0
        openvswitch 92544 1 brcompat

        [root@Rack3Host16 ~]# libvirtd --version
        libvirtd (libvirt) 0.10.1

        [root@Rack3Host16 ~]# ovs-vsctl list-br
        cloud0
        cloudbr

        [root@Rack3Host16 ~]# ovs-vsctl list-ports cloudbr

        7. UBUNTU 12.04 openvswitch configuration SUCCESSFUL:

        [ashen@localhost ~]$ ssh root@10.223.58.194
        root@10.223.58.194's password:
        Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-39-generic x86_64)

        System information as of Wed Apr 24 10:17:41 PDT 2013

        System load: 1.43 Users logged in: 1
        Usage of /: 1.7% of 213.44GB IP address for virbr0: 192.168.122.1
        Memory usage: 14% IP address for br0: 10.223.58.194
        Swap usage: 0% IP address for cloud0: 169.254.0.1
        Processes: 123

        root@Rack3Host15:~# brctl show
        bridge name bridge id STP enabled interfaces
        br0 0000.bc305bd41d41 no eth0
        vnet1
        vnet4
        vnet6
        breth0-1383 0000.bc305bd41d41 no eth0.1383
        vnet12
        vnet2
        vnet5
        vnet8
        breth0-2422 0000.bc305bd41d41 no eth0.2422
        vnet13
        vnet16
        vnet9
        breth0-2425 0000.bc305bd41d41 no eth0.2425
        vnet10
        vnet14
        vnet15
        cloud0 0000.02f4ebdb1a4d no vnet0
        vnet11
        vnet3
        vnet7
        cloudbr 0000.bc305bd41d42 no eth1
        virbr0 0000.229da19a354e no

        root@Rack3Host15:~# ifconfig
        br0 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41
        inet addr:10.223.58.194 Bcast:10.223.58.255 Mask:255.255.255.192
        inet6 addr: fc00:2::19a:fd66:9ec3:ea74/64 Scope:Global
        inet6 addr: fc00:2::bd85:4e19:5b66:6d5c/64 Scope:Global
        inet6 addr: fc00:2::9cf2:ae1f:85bd:e0a/64 Scope:Global
        inet6 addr: fc00:2::a177:4150:7f65:80ad/64 Scope:Global
        inet6 addr: fc00:2::be30:5bff:fed4:1d41/64 Scope:Global
        inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link
        inet6 addr: fc00:2::2c6e:5ef7:ab9a:1fdf/64 Scope:Global
        inet6 addr: fc00:2::8480:ad43:17eb:dcb5/64 Scope:Global
        inet6 addr: fc00:2::7c5e:a0d4:667d:d85/64 Scope:Global
        inet6 addr: fc00:2::2cb4:ca53:443f:e702/64 Scope:Global
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:73329203 errors:0 dropped:1420 overruns:0 frame:0
        TX packets:40658387 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:112440304684 (112.4 GB) TX bytes:22914745562 (22.9 GB)

        breth0-1383 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41
        inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:5809 errors:0 dropped:0 overruns:0 frame:0
        TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:495433 (495.4 KB) TX bytes:328 (328.0 B)

        breth0-2422 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41
        inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:1000 errors:0 dropped:0 overruns:0 frame:0
        TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:66833 (66.8 KB) TX bytes:468 (468.0 B)

        breth0-2425 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41
        inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:7301 errors:0 dropped:0 overruns:0 frame:0
        TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:475134 (475.1 KB) TX bytes:468 (468.0 B)

        cloud0 Link encap:Ethernet HWaddr 02:f4:eb:db:1a:4d
        inet addr:169.254.0.1 Bcast:169.254.255.255 Mask:255.255.0.0
        inet6 addr: fe80::f4:ebff:fedb:1a4d/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:365928 errors:0 dropped:0 overruns:0 frame:0
        TX packets:377930 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:66976216 (66.9 MB) TX bytes:75909408 (75.9 MB)

        eth0 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:111437360 errors:0 dropped:79 overruns:0 frame:0
        TX packets:48794963 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:1000
        RX bytes:115627172648 (115.6 GB) TX bytes:23678303462 (23.6 GB)
        Interrupt:16 Memory:da000000-da012800

        eth0.1383 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41
        inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:87508 errors:0 dropped:0 overruns:0 frame:0
        TX packets:114481 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:25974124 (25.9 MB) TX bytes:10361115 (10.3 MB)

        eth0.2422 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41
        inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:0 errors:0 dropped:0 overruns:0 frame:0
        TX packets:1012 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:0 (0.0 B) TX bytes:67769 (67.7 KB)

        eth0.2425 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41
        inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:0 errors:0 dropped:0 overruns:0 frame:0
        TX packets:7313 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:0 (0.0 B) TX bytes:476070 (476.0 KB)

        lo Link encap:Local Loopback
        inet addr:127.0.0.1 Mask:255.0.0.0
        inet6 addr: ::1/128 Scope:Host
        UP LOOPBACK RUNNING MTU:16436 Metric:1
        RX packets:302 errors:0 dropped:0 overruns:0 frame:0
        TX packets:302 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:27042 (27.0 KB) TX bytes:27042 (27.0 KB)

        virbr0 Link encap:Ethernet HWaddr 22:9d:a1:9a:35:4e
        inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:0 errors:0 dropped:0 overruns:0 frame:0
        TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

        vnet0 Link encap:Ethernet HWaddr fe:00:a9:fe:03:d9
        inet6 addr: fe80::fc00:a9ff:fefe:3d9/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:14 errors:0 dropped:0 overruns:0 frame:0
        TX packets:8035 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:1201 (1.2 KB) TX bytes:724054 (724.0 KB)

        vnet1 Link encap:Ethernet HWaddr fe:d3:dc:00:00:03
        inet6 addr: fe80::fcd3:dcff:fe00:3/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:70743 errors:0 dropped:0 overruns:0 frame:0
        TX packets:34281819 errors:0 dropped:0 overruns:758 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:7165910 (7.1 MB) TX bytes:27917346896 (27.9 GB)

        vnet2 Link encap:Ethernet HWaddr fe:aa:b6:00:00:10
        inet6 addr: fe80::fcaa:b6ff:fe00:10/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:5913 errors:0 dropped:0 overruns:0 frame:0
        TX packets:11493 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:2405133 (2.4 MB) TX bytes:3308880 (3.3 MB)

        vnet3 Link encap:Ethernet HWaddr fe:00:a9:fe:00:c2
        inet6 addr: fe80::fc00:a9ff:fefe:c2/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:15 errors:0 dropped:0 overruns:0 frame:0
        TX packets:8007 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:1251 (1.2 KB) TX bytes:722390 (722.3 KB)

        vnet4 Link encap:Ethernet HWaddr fe:83:72:00:00:05
        inet6 addr: fe80::fc83:72ff:fe00:5/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:344248 errors:0 dropped:0 overruns:0 frame:0
        TX packets:34523176 errors:0 dropped:0 overruns:682 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:45023067 (45.0 MB) TX bytes:27946596969 (27.9 GB)

        vnet5 Link encap:Ethernet HWaddr fe:91:ec:00:00:11
        inet6 addr: fe80::fc91:ecff:fe00:11/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:14 errors:0 dropped:0 overruns:0 frame:0
        TX packets:4238 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:1264 (1.2 KB) TX bytes:362853 (362.8 KB)

        vnet6 Link encap:Ethernet HWaddr fe:34:fe:00:00:0d
        inet6 addr: fe80::fc34:feff:fe00:d/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:18 errors:0 dropped:0 overruns:0 frame:0
        TX packets:34208546 errors:0 dropped:0 overruns:685 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:1468 (1.4 KB) TX bytes:27900653088 (27.9 GB)

        vnet7 Link encap:Ethernet HWaddr fe:00:a9:fe:03:a0
        inet6 addr: fe80::fc00:a9ff:fefe:3a0/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:135564 errors:0 dropped:0 overruns:0 frame:0
        TX packets:147052 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:24793427 (24.7 MB) TX bytes:28672478 (28.6 MB)

        vnet8 Link encap:Ethernet HWaddr fe:a0:f2:00:00:14
        inet6 addr: fe80::fca0:f2ff:fe00:14/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:82040 errors:0 dropped:0 overruns:0 frame:0
        TX packets:57646 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:6003378 (6.0 MB) TX bytes:12080972 (12.0 MB)

        vnet9 Link encap:Ethernet HWaddr fe:00:24:fd:00:02
        inet6 addr: fe80::fc00:24ff:fefd:2/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:66828 errors:0 dropped:0 overruns:0 frame:0
        TX packets:95490 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:13116330 (13.1 MB) TX bytes:7101586 (7.1 MB)

        vnet10 Link encap:Ethernet HWaddr fe:00:00:e1:00:02
        inet6 addr: fe80::fc00:ff:fee1:2/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:10738 errors:0 dropped:0 overruns:0 frame:0
        TX packets:27987 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:673028 (673.0 KB) TX bytes:1813058 (1.8 MB)

        vnet11 Link encap:Ethernet HWaddr fe:00:a9:fe:00:de
        inet6 addr: fe80::fc00:a9ff:fefe:de/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:135117 errors:0 dropped:0 overruns:0 frame:0
        TX packets:139794 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:24660919 (24.6 MB) TX bytes:28067120 (28.0 MB)

        vnet12 Link encap:Ethernet HWaddr fe:ab:da:00:00:12
        inet6 addr: fe80::fcab:daff:fe00:12/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:1171 errors:0 dropped:0 overruns:0 frame:0
        TX packets:4797 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:72402 (72.4 KB) TX bytes:479169 (479.1 KB)

        vnet13 Link encap:Ethernet HWaddr fe:00:0b:6c:00:01
        inet6 addr: fe80::fc00:bff:fe6c:1/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:48893 errors:0 dropped:0 overruns:0 frame:0
        TX packets:34489 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:3634304 (3.6 MB) TX bytes:6653621 (6.6 MB)

        vnet14 Link encap:Ethernet HWaddr fe:00:04:eb:00:01
        inet6 addr: fe80::fc00:4ff:feeb:1/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:13998 errors:0 dropped:0 overruns:0 frame:0
        TX packets:7709 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:906754 (906.7 KB) TX bytes:511028 (511.0 KB)

        vnet15 Link encap:Ethernet HWaddr fe:00:22:f3:00:03
        inet6 addr: fe80::fc00:22ff:fef3:3/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:13983 errors:0 dropped:0 overruns:0 frame:0
        TX packets:8692 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:905836 (905.8 KB) TX bytes:531554 (531.5 KB)

        vnet16 Link encap:Ethernet HWaddr fe:00:6e:68:00:03
        inet6 addr: fe80::fc00:6eff:fe68:3/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:46591 errors:0 dropped:0 overruns:0 frame:0
        TX packets:32982 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:500
        RX bytes:3466814 (3.4 MB) TX bytes:6496383 (6.4 MB)

        Show
        angeline shen added a comment - Hugo: Still FAILING configuration. I AM STILL GROPING IN THE DARK AS THERE IS NO DOCUMENTATION on how to configure openvswitch for CENTOS properly. There is only scattered documentation, I have lots of confusion on how to correctly configure host to make this work. Our developer thinks there is bug on openstack openvswitch side. Here is what I have done on the host configuration so far. Your previous comment of following FAILED: DEVICETYPE=ovs TYPE=OVSPort OVS_BRIDGE=ovsbr1 My problem is I am getting little bits of scattered info of various configuration files one at a time, This will take forever to get this working. There is only scattered docs about this configuration. For this round of testing FAILURES, here are all the files on the host: I used 'cloudbr' instead of 'br0' . I am still confused about this device name how it is tied together with command: ovs-vsctl add-br cloudbr Host: 10.223.58.195 1. /etc/sysconfig/ifcfg-eth1: DEVICE=eth1 BOOTPROTO=none HWADDR=BC:30:5B:D4:16:3C IPV6INIT=no #MTU=1500 NM_CONTROLLED=yes ONBOOT=yes TYPE=Ethernet UUID="0142ba15-de11-499c-a1ba-82837b30f890" IPADDR=10.223.58.195 NETMASK=255.255.255.192 GATEWAY=10.223.58.193 DNS1=10.223.110.254 USERCTL=yes IPV4_FAILURE_FATAL=yes DEFROUTE=yes NAME="System eth1" QUESTION: I don't have the line OVS_BRIDGE=cloudbr in /etc/sysconfig/ifcfg-eth1. Do I need this ? Does this mean I have to reinstall the host & reconfigure again at this time ? THIS IS ANOTHER EXAMPLE of HOW I AM GETTING only little bits of info one at a time. CAN YOU PLEASE GIVE ME CONCRETE EXAMPLE OF ALL THE FILES AND COMMANDS ON THE HOST ON HOW TO configure openswitch on CENTOS KVM? I HAVE BEEN REINSTALLING AND RECONFIGURING HOST, groping in the dark, each time discovering & experimenting on my own to move forward a bit. IT's taking too long..... 2. /etc/sysconfig/ifcfg-cloudbr : DEVICE=cloudbr ONBOOT=yes DEVICETYPE=ovs #TYPE=OVSPort OVS_BRIDGE=cloudbr TYPE=OVSBridge BOOTPROTO=none IPADDR=10.223.58.195 GATEWAY=10.23.58.193 NETMASK=255.255.255.0 HOTPLUG=no QUESTION: Originally, I had TYPE=OVSPort. After I executed following commands per Apache_CloudStack-410-installation_Guide-en-US.pdf section 8.1.8.3 page 75-76 ovs-vsctl add-br cloudbr ovs-vsctl add-port cloudbr eth1 The host CAN NO LONGER ping gateway, www.google.com. Then on the host, I commented out TYPE=OVSPort & reentered TYPE=OVSBridge. Please see above content. I did 'service network restart'. After several minutes, the HOST ABLE TO ping gateway, google, the WORLD.. WHAT A DISCOVERY !!!!! BUT I AM STILL ON THE WRONG TRACK , I KNOW. 3. brctl show [root@Rack3Host16 agent] # brctl show bridge name bridge id STP enabled interfaces cloud0 /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory 0000.aa81045f5e4a no cloudbr /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory 0000.2213344e8047 no QUESTION: OUR DEVELOPER edison.su@citrix.com says this is BUG IN OPENVSWITCH. 4. /etc/cloudstack/agent/agent.properties: #Storage #Wed Apr 24 17:35:17 PDT 2013 guest.network.device=cloudbr workers=5 private.network.device=cloudbr port=8250 resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource pod=1 zone=1 guid=be58e605-fec5-3ab0-8afc-6491a99baec3 public.network.device=cloudbr cluster=1 local.storage.uuid=75be87c8-ee41-491b-b261-86d78929e8e6 domr.scripts.dir=scripts/network/domr/kvm LibvirtComputingResource.id=0 host=10.223.195.114 ~ 5. [root@Rack3Host16 agent] # ifconfig cloud0 Link encap:Ethernet HWaddr AA:81:04:5F:5E:4A inet addr:169.254.0.1 Bcast:169.254.255.255 Mask:255.255.0.0 inet6 addr: fe80::a881:4ff:fe5f:5e4a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:23 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:4201 (4.1 KiB) cloudbr Link encap:Ethernet HWaddr 22:13:34:4E:80:47 inet addr:10.223.58.195 Bcast:10.223.58.255 Mask:255.255.255.0 inet6 addr: fe80::2013:34ff:fe4e:8047/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:27 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:4385 (4.2 KiB) eth1 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C inet addr:10.223.58.195 Bcast:10.223.58.255 Mask:255.255.255.192 inet6 addr: fc00:2::be30:5bff:fed4:163c/64 Scope:Global inet6 addr: fe80::be30:5bff:fed4:163c/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:2413713 errors:0 dropped:0 overruns:0 frame:0 TX packets:116725 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2287395217 (2.1 GiB) TX bytes:22431834 (21.3 MiB) Interrupt:16 Memory:da000000-da012800 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:78 errors:0 dropped:0 overruns:0 frame:0 TX packets:78 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:10536 (10.2 KiB) TX bytes:10536 (10.2 KiB) 6. [root@Rack3Host16 ~] # lsmod | grep bridge [root@Rack3Host16 ~] # lsmod | grep kvm kvm_intel 52570 0 kvm 314739 1 kvm_intel [root@Rack3Host16 ~] # [root@Rack3Host16 ~] # lsmod | grep brcom brcompat 5873 0 openvswitch 92544 1 brcompat [root@Rack3Host16 ~] # libvirtd --version libvirtd (libvirt) 0.10.1 [root@Rack3Host16 ~] # ovs-vsctl list-br cloud0 cloudbr [root@Rack3Host16 ~] # ovs-vsctl list-ports cloudbr 7. UBUNTU 12.04 openvswitch configuration SUCCESSFUL: [ashen@localhost ~] $ ssh root@10.223.58.194 root@10.223.58.194's password: Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-39-generic x86_64) Documentation: https://help.ubuntu.com/ System information as of Wed Apr 24 10:17:41 PDT 2013 System load: 1.43 Users logged in: 1 Usage of /: 1.7% of 213.44GB IP address for virbr0: 192.168.122.1 Memory usage: 14% IP address for br0: 10.223.58.194 Swap usage: 0% IP address for cloud0: 169.254.0.1 Processes: 123 root@Rack3Host15:~# brctl show bridge name bridge id STP enabled interfaces br0 0000.bc305bd41d41 no eth0 vnet1 vnet4 vnet6 breth0-1383 0000.bc305bd41d41 no eth0.1383 vnet12 vnet2 vnet5 vnet8 breth0-2422 0000.bc305bd41d41 no eth0.2422 vnet13 vnet16 vnet9 breth0-2425 0000.bc305bd41d41 no eth0.2425 vnet10 vnet14 vnet15 cloud0 0000.02f4ebdb1a4d no vnet0 vnet11 vnet3 vnet7 cloudbr 0000.bc305bd41d42 no eth1 virbr0 0000.229da19a354e no root@Rack3Host15:~# ifconfig br0 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41 inet addr:10.223.58.194 Bcast:10.223.58.255 Mask:255.255.255.192 inet6 addr: fc00:2::19a:fd66:9ec3:ea74/64 Scope:Global inet6 addr: fc00:2::bd85:4e19:5b66:6d5c/64 Scope:Global inet6 addr: fc00:2::9cf2:ae1f:85bd:e0a/64 Scope:Global inet6 addr: fc00:2::a177:4150:7f65:80ad/64 Scope:Global inet6 addr: fc00:2::be30:5bff:fed4:1d41/64 Scope:Global inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link inet6 addr: fc00:2::2c6e:5ef7:ab9a:1fdf/64 Scope:Global inet6 addr: fc00:2::8480:ad43:17eb:dcb5/64 Scope:Global inet6 addr: fc00:2::7c5e:a0d4:667d:d85/64 Scope:Global inet6 addr: fc00:2::2cb4:ca53:443f:e702/64 Scope:Global UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:73329203 errors:0 dropped:1420 overruns:0 frame:0 TX packets:40658387 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:112440304684 (112.4 GB) TX bytes:22914745562 (22.9 GB) breth0-1383 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41 inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5809 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:495433 (495.4 KB) TX bytes:328 (328.0 B) breth0-2422 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41 inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1000 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:66833 (66.8 KB) TX bytes:468 (468.0 B) breth0-2425 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41 inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7301 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:475134 (475.1 KB) TX bytes:468 (468.0 B) cloud0 Link encap:Ethernet HWaddr 02:f4:eb:db:1a:4d inet addr:169.254.0.1 Bcast:169.254.255.255 Mask:255.255.0.0 inet6 addr: fe80::f4:ebff:fedb:1a4d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:365928 errors:0 dropped:0 overruns:0 frame:0 TX packets:377930 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:66976216 (66.9 MB) TX bytes:75909408 (75.9 MB) eth0 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:111437360 errors:0 dropped:79 overruns:0 frame:0 TX packets:48794963 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:115627172648 (115.6 GB) TX bytes:23678303462 (23.6 GB) Interrupt:16 Memory:da000000-da012800 eth0.1383 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41 inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:87508 errors:0 dropped:0 overruns:0 frame:0 TX packets:114481 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:25974124 (25.9 MB) TX bytes:10361115 (10.3 MB) eth0.2422 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41 inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:1012 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:67769 (67.7 KB) eth0.2425 Link encap:Ethernet HWaddr bc:30:5b:d4:1d:41 inet6 addr: fe80::be30:5bff:fed4:1d41/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:7313 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:476070 (476.0 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:302 errors:0 dropped:0 overruns:0 frame:0 TX packets:302 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:27042 (27.0 KB) TX bytes:27042 (27.0 KB) virbr0 Link encap:Ethernet HWaddr 22:9d:a1:9a:35:4e inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) vnet0 Link encap:Ethernet HWaddr fe:00:a9:fe:03:d9 inet6 addr: fe80::fc00:a9ff:fefe:3d9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:14 errors:0 dropped:0 overruns:0 frame:0 TX packets:8035 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:1201 (1.2 KB) TX bytes:724054 (724.0 KB) vnet1 Link encap:Ethernet HWaddr fe:d3:dc:00:00:03 inet6 addr: fe80::fcd3:dcff:fe00:3/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:70743 errors:0 dropped:0 overruns:0 frame:0 TX packets:34281819 errors:0 dropped:0 overruns:758 carrier:0 collisions:0 txqueuelen:500 RX bytes:7165910 (7.1 MB) TX bytes:27917346896 (27.9 GB) vnet2 Link encap:Ethernet HWaddr fe:aa:b6:00:00:10 inet6 addr: fe80::fcaa:b6ff:fe00:10/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5913 errors:0 dropped:0 overruns:0 frame:0 TX packets:11493 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:2405133 (2.4 MB) TX bytes:3308880 (3.3 MB) vnet3 Link encap:Ethernet HWaddr fe:00:a9:fe:00:c2 inet6 addr: fe80::fc00:a9ff:fefe:c2/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:15 errors:0 dropped:0 overruns:0 frame:0 TX packets:8007 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:1251 (1.2 KB) TX bytes:722390 (722.3 KB) vnet4 Link encap:Ethernet HWaddr fe:83:72:00:00:05 inet6 addr: fe80::fc83:72ff:fe00:5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:344248 errors:0 dropped:0 overruns:0 frame:0 TX packets:34523176 errors:0 dropped:0 overruns:682 carrier:0 collisions:0 txqueuelen:500 RX bytes:45023067 (45.0 MB) TX bytes:27946596969 (27.9 GB) vnet5 Link encap:Ethernet HWaddr fe:91:ec:00:00:11 inet6 addr: fe80::fc91:ecff:fe00:11/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:14 errors:0 dropped:0 overruns:0 frame:0 TX packets:4238 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:1264 (1.2 KB) TX bytes:362853 (362.8 KB) vnet6 Link encap:Ethernet HWaddr fe:34:fe:00:00:0d inet6 addr: fe80::fc34:feff:fe00:d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:18 errors:0 dropped:0 overruns:0 frame:0 TX packets:34208546 errors:0 dropped:0 overruns:685 carrier:0 collisions:0 txqueuelen:500 RX bytes:1468 (1.4 KB) TX bytes:27900653088 (27.9 GB) vnet7 Link encap:Ethernet HWaddr fe:00:a9:fe:03:a0 inet6 addr: fe80::fc00:a9ff:fefe:3a0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:135564 errors:0 dropped:0 overruns:0 frame:0 TX packets:147052 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:24793427 (24.7 MB) TX bytes:28672478 (28.6 MB) vnet8 Link encap:Ethernet HWaddr fe:a0:f2:00:00:14 inet6 addr: fe80::fca0:f2ff:fe00:14/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:82040 errors:0 dropped:0 overruns:0 frame:0 TX packets:57646 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:6003378 (6.0 MB) TX bytes:12080972 (12.0 MB) vnet9 Link encap:Ethernet HWaddr fe:00:24:fd:00:02 inet6 addr: fe80::fc00:24ff:fefd:2/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:66828 errors:0 dropped:0 overruns:0 frame:0 TX packets:95490 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:13116330 (13.1 MB) TX bytes:7101586 (7.1 MB) vnet10 Link encap:Ethernet HWaddr fe:00:00:e1:00:02 inet6 addr: fe80::fc00:ff:fee1:2/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:10738 errors:0 dropped:0 overruns:0 frame:0 TX packets:27987 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:673028 (673.0 KB) TX bytes:1813058 (1.8 MB) vnet11 Link encap:Ethernet HWaddr fe:00:a9:fe:00:de inet6 addr: fe80::fc00:a9ff:fefe:de/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:135117 errors:0 dropped:0 overruns:0 frame:0 TX packets:139794 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:24660919 (24.6 MB) TX bytes:28067120 (28.0 MB) vnet12 Link encap:Ethernet HWaddr fe:ab:da:00:00:12 inet6 addr: fe80::fcab:daff:fe00:12/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1171 errors:0 dropped:0 overruns:0 frame:0 TX packets:4797 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:72402 (72.4 KB) TX bytes:479169 (479.1 KB) vnet13 Link encap:Ethernet HWaddr fe:00:0b:6c:00:01 inet6 addr: fe80::fc00:bff:fe6c:1/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:48893 errors:0 dropped:0 overruns:0 frame:0 TX packets:34489 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:3634304 (3.6 MB) TX bytes:6653621 (6.6 MB) vnet14 Link encap:Ethernet HWaddr fe:00:04:eb:00:01 inet6 addr: fe80::fc00:4ff:feeb:1/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:13998 errors:0 dropped:0 overruns:0 frame:0 TX packets:7709 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:906754 (906.7 KB) TX bytes:511028 (511.0 KB) vnet15 Link encap:Ethernet HWaddr fe:00:22:f3:00:03 inet6 addr: fe80::fc00:22ff:fef3:3/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:13983 errors:0 dropped:0 overruns:0 frame:0 TX packets:8692 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:905836 (905.8 KB) TX bytes:531554 (531.5 KB) vnet16 Link encap:Ethernet HWaddr fe:00:6e:68:00:03 inet6 addr: fe80::fc00:6eff:fe68:3/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:46591 errors:0 dropped:0 overruns:0 frame:0 TX packets:32982 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:3466814 (3.4 MB) TX bytes:6496383 (6.4 MB)
        Hide
        angeline shen added a comment -

        Hugo:

        On the host 10.223.58.195 , After I changed /etc/sysconfig/network-scripts/ifcfg-eth1 and
        /etc/sysconfig/network-scripts/ifcfg-cloudbr to your instructions, service network restart ,

        the host can ONLY ping its own gateway 10.223.58.193 and any host in same subnet,
        but CANNOT reach anything beyond the gateway 10.223.58.193.

        1. /etc/sysconfig/network-scripts/ifcfg-eth1:

        DEVICE=eth1
        BOOTPROTO=none
        HWADDR=BC:30:5B:D4:16:3C
        IPV6INIT=no
        #MTU=1500
        NM_CONTROLLED=no
        ONBOOT=yes
        #TYPE=Ethernet
        TYPE=OVSPort
        DEVICETYPE=ovs
        OVS_BRIDGE=cloudbr
        #UUID="0142ba15-de11-499c-a1ba-82837b30f890"
        #IPADDR=10.223.58.195
        #NETMASK=255.255.255.192
        #GATEWAY=10.223.58.193
        #DNS1=10.223.110.254
        #USERCTL=yes
        #IPV4_FAILURE_FATAL=yes
        #DEFROUTE=yes
        #NAME="System eth1"

        2. /etc/sysconfig/network-scripts/ifcfg-cloudbr:

        DEVICE=cloudbr
        ONBOOT=yes
        DEVICETYPE=ovs
        TYPE=OVSBridge
        BOOTPROTO=static
        IPADDR=10.223.58.195
        GATEWAY=10.23.58.193
        NETMASK=255.255.255.0
        HOTPLUG=no

        3. On host 10.223.58.195, service network restart

        4. on host 10.223.58.195:

        [root@Rack3Host16 ~]# ip route
        10.223.58.0/24 dev cloudbr proto kernel scope link src 10.223.58.195
        169.254.0.0/16 dev cloud0 proto kernel scope link src 169.254.0.1
        169.254.0.0/16 dev eth1 scope link metric 1002
        169.254.0.0/16 dev cloudbr scope link metric 1015

        5. It can reach its gateway & other hosts on same gateway:

        [root@Rack3Host16 ~]# ping 10.223.58.193
        PING 10.223.58.193 (10.223.58.193) 56(84) bytes of data.
        64 bytes from 10.223.58.193: icmp_seq=1 ttl=64 time=7.43 ms
        64 bytes from 10.223.58.193: icmp_seq=2 ttl=64 time=2.82 ms
        64 bytes from 10.223.58.193: icmp_seq=3 ttl=64 time=1.06 ms
        ^C
        — 10.223.58.193 ping statistics —
        3 packets transmitted, 3 received, 0% packet loss, time 2364ms
        rtt min/avg/max/mdev = 1.061/3.771/7.430/2.685 ms
        [root@Rack3Host16 ~]# ping 10.223.58.194
        PING 10.223.58.194 (10.223.58.194) 56(84) bytes of data.
        64 bytes from 10.223.58.194: icmp_seq=1 ttl=64 time=0.399 ms
        64 bytes from 10.223.58.194: icmp_seq=2 ttl=64 time=0.181 ms
        ^C
        — 10.223.58.194 ping statistics —
        2 packets transmitted, 2 received, 0% packet loss, time 1446ms
        rtt min/avg/max/mdev = 0.181/0.290/0.399/0.109 ms

        [root@Rack3Host16 ~]# traceroute 10.223.58.194
        traceroute to 10.223.58.194 (10.223.58.194), 30 hops max, 60 byte packets
        1 10.223.58.194 (10.223.58.194) 0.922 ms 0.851 ms 0.830 ms

        [root@Rack3Host16 ~]# traceroute 10.223.58.193
        traceroute to 10.223.58.193 (10.223.58.193), 30 hops max, 60 byte packets
        1 10.223.58.193 (10.223.58.193) 7.017 ms 12.506 ms 7.675 ms

        6. host CANNOT reach anything outside of its subnet:

        [root@Rack3Host16 ~]# ping 10.223.195.114
        connect: Network is unreachable

        [root@Rack3Host16 ~]# ping www.google.com
        ping: unknown host www.google.com

        [root@Rack3Host16 ~]# ping 8.8.8.8
        connect: Network is unreachable

        [root@Rack3Host16 ~]# traceroute 8.8.8.8
        traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
        connect: Network is unreachable

        7. [root@Rack3Host16 ~]# ifconfig
        cloudbr Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C
        inet addr:10.223.58.195 Bcast:10.223.58.255 Mask:255.255.255.0
        inet6 addr: fc00:2::be30:5bff:fed4:163c/64 Scope:Global
        inet6 addr: fe80::68f7:52ff:fe03:a744/64 Scope:Link
        UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
        RX packets:46945 errors:0 dropped:0 overruns:0 frame:0
        TX packets:549 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:39235509 (37.4 MiB) TX bytes:102773 (100.3 KiB)

        cloud0 Link encap:Ethernet HWaddr AA:81:04:5F:5E:4A
        inet addr:169.254.0.1 Bcast:169.254.255.255 Mask:255.255.0.0
        inet6 addr: fe80::a881:4ff:fe5f:5e4a/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:0 errors:0 dropped:0 overruns:0 frame:0
        TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:0 (0.0 b) TX bytes:4201 (4.1 KiB)

        eth1 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C
        inet6 addr: fe80::be30:5bff:fed4:163c/64 Scope:Link
        UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
        RX packets:48123 errors:0 dropped:0 overruns:0 frame:0
        TX packets:532 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:1000
        RX bytes:39518352 (37.6 MiB) TX bytes:101960 (99.5 KiB)
        Interrupt:16 Memory:da000000-da012800

        lo Link encap:Local Loopback
        inet addr:127.0.0.1 Mask:255.0.0.0
        inet6 addr: ::1/128 Scope:Host
        UP LOOPBACK RUNNING MTU:16436 Metric:1
        RX packets:340 errors:0 dropped:0 overruns:0 frame:0
        TX packets:340 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:46264 (45.1 KiB) TX bytes:46264 (45.1 KiB)

        8. [root@Rack3Host16 ~]# brctl show
        bridge name bridge id STP enabled interfaces
        cloud0 /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        /sys/class/net/cloud0/bridge: No such file or directory
        0000.aa81045f5e4a no
        cloudbr /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        /sys/class/net/cloudbr/bridge: No such file or directory
        0000.bc305bd4163c no

        Show
        angeline shen added a comment - Hugo: On the host 10.223.58.195 , After I changed /etc/sysconfig/network-scripts/ifcfg-eth1 and /etc/sysconfig/network-scripts/ifcfg-cloudbr to your instructions, service network restart , the host can ONLY ping its own gateway 10.223.58.193 and any host in same subnet, but CANNOT reach anything beyond the gateway 10.223.58.193. 1. /etc/sysconfig/network-scripts/ifcfg-eth1: DEVICE=eth1 BOOTPROTO=none HWADDR=BC:30:5B:D4:16:3C IPV6INIT=no #MTU=1500 NM_CONTROLLED=no ONBOOT=yes #TYPE=Ethernet TYPE=OVSPort DEVICETYPE=ovs OVS_BRIDGE=cloudbr #UUID="0142ba15-de11-499c-a1ba-82837b30f890" #IPADDR=10.223.58.195 #NETMASK=255.255.255.192 #GATEWAY=10.223.58.193 #DNS1=10.223.110.254 #USERCTL=yes #IPV4_FAILURE_FATAL=yes #DEFROUTE=yes #NAME="System eth1" 2. /etc/sysconfig/network-scripts/ifcfg-cloudbr: DEVICE=cloudbr ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR=10.223.58.195 GATEWAY=10.23.58.193 NETMASK=255.255.255.0 HOTPLUG=no 3. On host 10.223.58.195, service network restart 4. on host 10.223.58.195: [root@Rack3Host16 ~] # ip route 10.223.58.0/24 dev cloudbr proto kernel scope link src 10.223.58.195 169.254.0.0/16 dev cloud0 proto kernel scope link src 169.254.0.1 169.254.0.0/16 dev eth1 scope link metric 1002 169.254.0.0/16 dev cloudbr scope link metric 1015 5. It can reach its gateway & other hosts on same gateway: [root@Rack3Host16 ~] # ping 10.223.58.193 PING 10.223.58.193 (10.223.58.193) 56(84) bytes of data. 64 bytes from 10.223.58.193: icmp_seq=1 ttl=64 time=7.43 ms 64 bytes from 10.223.58.193: icmp_seq=2 ttl=64 time=2.82 ms 64 bytes from 10.223.58.193: icmp_seq=3 ttl=64 time=1.06 ms ^C — 10.223.58.193 ping statistics — 3 packets transmitted, 3 received, 0% packet loss, time 2364ms rtt min/avg/max/mdev = 1.061/3.771/7.430/2.685 ms [root@Rack3Host16 ~] # ping 10.223.58.194 PING 10.223.58.194 (10.223.58.194) 56(84) bytes of data. 64 bytes from 10.223.58.194: icmp_seq=1 ttl=64 time=0.399 ms 64 bytes from 10.223.58.194: icmp_seq=2 ttl=64 time=0.181 ms ^C — 10.223.58.194 ping statistics — 2 packets transmitted, 2 received, 0% packet loss, time 1446ms rtt min/avg/max/mdev = 0.181/0.290/0.399/0.109 ms [root@Rack3Host16 ~] # traceroute 10.223.58.194 traceroute to 10.223.58.194 (10.223.58.194), 30 hops max, 60 byte packets 1 10.223.58.194 (10.223.58.194) 0.922 ms 0.851 ms 0.830 ms [root@Rack3Host16 ~] # traceroute 10.223.58.193 traceroute to 10.223.58.193 (10.223.58.193), 30 hops max, 60 byte packets 1 10.223.58.193 (10.223.58.193) 7.017 ms 12.506 ms 7.675 ms 6. host CANNOT reach anything outside of its subnet: [root@Rack3Host16 ~] # ping 10.223.195.114 connect: Network is unreachable [root@Rack3Host16 ~] # ping www.google.com ping: unknown host www.google.com [root@Rack3Host16 ~] # ping 8.8.8.8 connect: Network is unreachable [root@Rack3Host16 ~] # traceroute 8.8.8.8 traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets connect: Network is unreachable 7. [root@Rack3Host16 ~] # ifconfig cloudbr Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C inet addr:10.223.58.195 Bcast:10.223.58.255 Mask:255.255.255.0 inet6 addr: fc00:2::be30:5bff:fed4:163c/64 Scope:Global inet6 addr: fe80::68f7:52ff:fe03:a744/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:46945 errors:0 dropped:0 overruns:0 frame:0 TX packets:549 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:39235509 (37.4 MiB) TX bytes:102773 (100.3 KiB) cloud0 Link encap:Ethernet HWaddr AA:81:04:5F:5E:4A inet addr:169.254.0.1 Bcast:169.254.255.255 Mask:255.255.0.0 inet6 addr: fe80::a881:4ff:fe5f:5e4a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:23 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:4201 (4.1 KiB) eth1 Link encap:Ethernet HWaddr BC:30:5B:D4:16:3C inet6 addr: fe80::be30:5bff:fed4:163c/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:48123 errors:0 dropped:0 overruns:0 frame:0 TX packets:532 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:39518352 (37.6 MiB) TX bytes:101960 (99.5 KiB) Interrupt:16 Memory:da000000-da012800 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:340 errors:0 dropped:0 overruns:0 frame:0 TX packets:340 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:46264 (45.1 KiB) TX bytes:46264 (45.1 KiB) 8. [root@Rack3Host16 ~] # brctl show bridge name bridge id STP enabled interfaces cloud0 /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory 0000.aa81045f5e4a no cloudbr /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory /sys/class/net/cloudbr/bridge: No such file or directory 0000.bc305bd4163c no
        Hide
        angeline shen added a comment -

        Hugo:

        For your configuration example above:

        1. The physical interface: eth1
          DEVICE=eth1
          BOOTPROTO=none
          HWADDR=BC:30:5B:D4:16:3C
          IPV6INIT=no
          #MTU=1500
          NM_CONTROLLED=no
          ONBOOT=yes
          TYPE=OVSPort
          DEVICETYPE=ovs
          OVS_BRIDGE=br0
        1. The bridge interface: br0
          DEVICE=br0
          ONBOOT=yes
          DEVICETYPE=ovs
          TYPE=OVSBridge
          BOOTPROTO=static
          IPADDR= 10.223.58.195
          GATEWAY = 10.223.58.193
          NETMASK=255.255.255.192
          HOTPLUG=no

        QUESTION: will commands to configure openvswitch network interfaces be as follows:

        ovs-vsctl add-br br0
        ovs-vsctl add-port br0 eth1

        Thanks

        Show
        angeline shen added a comment - Hugo: For your configuration example above: The physical interface: eth1 DEVICE=eth1 BOOTPROTO=none HWADDR=BC:30:5B:D4:16:3C IPV6INIT=no #MTU=1500 NM_CONTROLLED=no ONBOOT=yes TYPE=OVSPort DEVICETYPE=ovs OVS_BRIDGE=br0 The bridge interface: br0 DEVICE=br0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR= 10.223.58.195 GATEWAY = 10.223.58.193 NETMASK=255.255.255.192 HOTPLUG=no QUESTION: will commands to configure openvswitch network interfaces be as follows: ovs-vsctl add-br br0 ovs-vsctl add-port br0 eth1 Thanks

          People

          • Assignee:
            Hugo Trippaers
            Reporter:
            Prasanna Santhanam
          • Votes:
            4 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development