Details

    • Type: Test
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: tests
    • Labels:
      None

      Description

      Add smoke tests for Hadoop HDFS.

      1. 0001-BIGTOP-2232.-Add-smoke-tests-for-HDFS.patch
        4 kB
        Konstantin Boudnik
      2. 0001-BIGTOP-2232.-Add-smoke-tests-for-HDFS.patch
        4 kB
        Konstantin Boudnik
      3. 0001-BIGTOP-2232.-Add-smoke-tests-for-HDFS.patch
        4 kB
        Konstantin Boudnik
      4. 0003-BIGTOP-2232.-Add-smoke-tests-for-HDFS.patch
        3 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Hide
          cos Konstantin Boudnik added a comment -

          Here's the initial patch, but first all the subtasks need to be fixed.
          At the moment, I am excluding all new tests introduced in BIGTOP-2009 as they are full of hardcoded paths to the resources and that needs to be fixed before they can be used in the smoking environment.
          The main source of issues is this block of the code which is repeated again and again

          +    try {
                 JarContent.unpackJarContainer(TestLs.class, "." , null);
          +    } catch (IOException ioe) {
          +        println ("Perhaps running as smoke test environment");
          +    }
               // get namenode hostname from core-site.xml
               Configuration conf = new Configuration();
               namenode = conf.get("fs.defaultFS");
          @@ -53,7 +57,7 @@ public class TestLs {
               }
               assertTrue("Could not find namenode", namenode != null);
           
          -    sh.exec("cp -r test_data test_data_TestLs");
          +    sh.exec("cp -r `find . -name test_data` test_data_TestLs");
          

          The changes illustrate the issue with the current approach. While first fix is somewhat ok, the later one blows, but something of the sort would have to be done.

          Show
          cos Konstantin Boudnik added a comment - Here's the initial patch, but first all the subtasks need to be fixed. At the moment, I am excluding all new tests introduced in BIGTOP-2009 as they are full of hardcoded paths to the resources and that needs to be fixed before they can be used in the smoking environment. The main source of issues is this block of the code which is repeated again and again + try { JarContent.unpackJarContainer(TestLs.class, "." , null ); + } catch (IOException ioe) { + println ( "Perhaps running as smoke test environment" ); + } // get namenode hostname from core-site.xml Configuration conf = new Configuration(); namenode = conf.get( "fs.defaultFS" ); @@ -53,7 +57,7 @@ public class TestLs { } assertTrue( "Could not find namenode" , namenode != null ); - sh.exec( "cp -r test_data test_data_TestLs" ); + sh.exec( "cp -r `find . -name test_data` test_data_TestLs" ); The changes illustrate the issue with the current approach. While first fix is somewhat ok, the later one blows, but something of the sort would have to be done.
          Hide
          cos Konstantin Boudnik added a comment -

          Need to add HADOOP_CONF_DIR to the testRuntime scope.

          Show
          cos Konstantin Boudnik added a comment - Need to add HADOOP_CONF_DIR to the testRuntime scope.
          Hide
          cos Konstantin Boudnik added a comment -

          And some last tweaking. Now all 4 are ready to go.

          Show
          cos Konstantin Boudnik added a comment - And some last tweaking. Now all 4 are ready to go.
          Hide
          cos Konstantin Boudnik added a comment -

          Argh, needed to guard against not-set HADOOP_CONF_DIR as it might break the other parts of the build. This is good now

          Show
          cos Konstantin Boudnik added a comment - Argh, needed to guard against not-set HADOOP_CONF_DIR as it might break the other parts of the build. This is good now
          Hide
          cos Konstantin Boudnik added a comment -

          Pushed to the master

          Show
          cos Konstantin Boudnik added a comment - Pushed to the master

            People

            • Assignee:
              cos Konstantin Boudnik
              Reporter:
              cos Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development