Bigtop
  1. Bigtop
  2. BIGTOP-414

enable hadoop tests in hadoop-0.23 branch to build and to run

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.3.0
    • Fix Version/s: 0.3.0
    • Component/s: Tests
    • Labels:
      None

      Description

      The tests in bigtop-tests in the hadoop-0.23 branch do not build, and therefore cannot run.
      I am submitting a patch that enables the hadoop-smoke test artifact to build, and the tests to be executed.

      1. BIGTOP-414.patch
        293 kB
        Wing Yew Poon

        Activity

        Hide
        Wing Yew Poon added a comment -

        Patch is available.

        Show
        Wing Yew Poon added a comment - Patch is available.
        Hide
        Wing Yew Poon added a comment -

        This is a first step to make the tests in the hadoop-0.23 branch build, and run. In order to achieve that, I had to comment out the other modules in the top level (for both the test artifact and the test execution pom), since I have only worked on the hadoop module.
        For the hadoop tests, because HADOOP-7730 and HDFS-1762 are only fixed in hadoop 0.22.x and not in hadoop 0.23, I created a TestCLI class in bigtop itself, extending org.apache.hadoop.cli.CLITestHelper, and created the necessary CommandExecutor implementation as well. There are some improvements in this TestCLI, notably the removal of hardcoded /tmp/testcli paths in the expected results, and parameterizing the superuser group in the expected results from the hdfs configuration.
        Aside from changes to enable TestHadoopExamples to run, I took the liberty to modifying the parameters to some examples to make them more friendly to running on a lower resource pseudo-distributed cluster on a vm environment.

        Show
        Wing Yew Poon added a comment - This is a first step to make the tests in the hadoop-0.23 branch build, and run. In order to achieve that, I had to comment out the other modules in the top level (for both the test artifact and the test execution pom), since I have only worked on the hadoop module. For the hadoop tests, because HADOOP-7730 and HDFS-1762 are only fixed in hadoop 0.22.x and not in hadoop 0.23, I created a TestCLI class in bigtop itself, extending org.apache.hadoop.cli.CLITestHelper, and created the necessary CommandExecutor implementation as well. There are some improvements in this TestCLI, notably the removal of hardcoded /tmp/testcli paths in the expected results, and parameterizing the superuser group in the expected results from the hdfs configuration. Aside from changes to enable TestHadoopExamples to run, I took the liberty to modifying the parameters to some examples to make them more friendly to running on a lower resource pseudo-distributed cluster on a vm environment.
        Hide
        Roman Shaposhnik added a comment -

        Good stuff. I'll make sure to review and commit it first thing in the morning tomorrow. Thanks for the patch.

        Show
        Roman Shaposhnik added a comment - Good stuff. I'll make sure to review and commit it first thing in the morning tomorrow. Thanks for the patch.
        Hide
        Roman Shaposhnik added a comment -

        +1. And committed. Thank a lot for the cleanup!

        Show
        Roman Shaposhnik added a comment - +1. And committed. Thank a lot for the cleanup!

          People

          • Assignee:
            Wing Yew Poon
            Reporter:
            Wing Yew Poon
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development