Bigtop
  1. Bigtop
  2. BIGTOP-635

Implement a cluster-abstraction, discovery and manipulation framework for iTest

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.4.0
    • Fix Version/s: None
    • Component/s: tests
    • Labels:
      None

      Description

      We've come to a point where our tests need to have a uniform way of interfacing with the cluster under test. It is no longer ok to assume that the test can be executed on a particular node (and thus have access to services running on it). It is also less than ideal for tests to assume a particular type of interaction with the services since it tends to break in different deployment scenarios.

      A framework that needs to be put in place has to be capable of (regardless of where a test using it is executed on):

      1. representing the abstract configuration of the cluster
      2. representing the abstract topology of the entire cluster (services running on a cluster, nodes hosting the daemons, racks, etc).
      3. giving tests an ability to query this topology
      4. giving tests an ability to affect the nodes in that topology in a particular way (refreshing configuration, restarting services, etc.)

      Of course, the ideal solution here would be to give Bigtop tests a programmatic access to a Hadoop cluster management framework such as Cloudera's CM or Apache Ambari.

      As with any ideal solutions I don't think it is realistic though. Hence we have to cook something up. At this point I'm really focused on getting the API right and I'm totally fine with an implementation of that API to be something as silly as a bunch of ssh-based scripts or something.

      This JIRA is primarily focused on coming up with such an API. Anybody who's willing to help is welcome to.

      1. ClusterManagerAPI.pdf
        27 kB
        Sujay Rau
      2. BigtopClusterManagerv2.zip
        18 kB
        Sujay Rau
      3. BigtopClusterManager.zip
        11 kB
        Sujay Rau
      4. bigtop-635.patch
        37 kB
        Sujay Rau
      5. bigtop-635.patch
        38 kB
        Sujay Rau

        Issue Links

          Activity

          Roman Shaposhnik created issue -
          Roman Shaposhnik made changes -
          Field Original Value New Value
          Assignee Roman Shaposhnik [ rvs ] Sujay Rau [ sujay.rau ]
          Roman Shaposhnik made changes -
          Link This issue blocks BIGTOP-614 [ BIGTOP-614 ]
          Roman Shaposhnik made changes -
          Fix Version/s 0.5.0 [ 12321865 ]
          Enis Soztutar made changes -
          Link This issue is related to HBASE-6201 [ HBASE-6201 ]
          Sujay Rau made changes -
          Attachment ClusterManagerAPI.pdf [ 12535760 ]
          Attachment BigtopClusterManager.zip [ 12535761 ]
          Sujay Rau made changes -
          Attachment BigtopClusterManagerv2.zip [ 12537378 ]
          Sujay Rau made changes -
          Attachment bigtop-635.patch [ 12537620 ]
          Sujay Rau made changes -
          Attachment bigtop-635.patch [ 12537622 ]
          Sujay Rau made changes -
          Attachment bigtop-635.patch [ 12537620 ]
          Sujay Rau made changes -
          Attachment bigtop-635.patch [ 12537750 ]
          Sujay Rau made changes -
          Attachment bigtop-635.patch [ 12537750 ]
          Sujay Rau made changes -
          Attachment bigtop-635.patch [ 12537753 ]
          Stephen Chu made changes -
          Assignee Sujay Rau [ sujay.rau ] Stephen Chu [ schu ]
          Bruno Mahé made changes -
          Link This issue relates to BIGTOP-710 [ BIGTOP-710 ]
          Roman Shaposhnik made changes -
          Fix Version/s 0.6.0 [ 12323895 ]
          Fix Version/s 0.5.0 [ 12321865 ]
          Roman Shaposhnik made changes -
          Fix Version/s 0.6.0 [ 12323895 ]

            People

            • Assignee:
              Stephen Chu
              Reporter:
              Roman Shaposhnik
            • Votes:
              0 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated:

                Development