Hadoop Common
  1. Hadoop Common
  2. HADOOP-6879

Provide SSH based (Jsch) remote execution API for system tests

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: build, test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Tags:
      herriot

      Description

      http://mvnrepository.com/
      com.jcraft » jsch
      0.1.42 version needs to be included in the build. This is needed to facilitate implementation of some system (Herriot) testcases .

      Please include this in ivy.

      jsch is originally located in http://www.jcraft.com/jsch/

      1. HADOOP-6879.y20.patch
        14 kB
        Konstantin Boudnik
      2. HADOOP-6879.y20.patch
        14 kB
        Konstantin Boudnik
      3. HADOOP-6879.y20.patch
        14 kB
        Konstantin Boudnik
      4. HADOOP-6879.y20.patch
        21 kB
        Konstantin Boudnik
      5. HADOOP-6879.y20.patch
        21 kB
        Konstantin Boudnik
      6. HADOOP-6879.y20.patch
        21 kB
        Konstantin Boudnik
      7. HADOOP-6879.patch
        0.7 kB
        Iyappan Srinivasan
      8. HADOOP-6879.patch
        0.7 kB
        Iyappan Srinivasan
      9. HADOOP-6879.patch
        5 kB
        Konstantin Boudnik
      10. HADOOP-6879.patch
        7 kB
        Konstantin Boudnik
      11. HADOOP-6879.patch
        7 kB
        Konstantin Boudnik
      12. HADOOP-6879.patch
        14 kB
        Konstantin Boudnik
      13. HADOOP-6879.patch
        11 kB
        Konstantin Boudnik
      14. HADOOP-6879.patch
        11 kB
        Konstantin Boudnik
      15. HADOOP-6879.patch
        11 kB
        Konstantin Boudnik
      16. HADOOP-6879.patch
        11 kB
        Konstantin Boudnik
      17. HADOOP-6879.patch
        11 kB
        Konstantin Boudnik
      18. 6879-ydist-security-patch.txt
        0.7 kB
        Iyappan Srinivasan

        Issue Links

          Activity

          Hide
          Iyappan Srinivasan added a comment -

          patch for 0.20.1.xxx

          Show
          Iyappan Srinivasan added a comment - patch for 0.20.1.xxx
          Hide
          Konstantin Boudnik added a comment -

          Please have the trunk patch as well. For trunk, I think, this jar file needs to go to test->system profile, not master.

          Show
          Konstantin Boudnik added a comment - Please have the trunk patch as well. For trunk, I think, this jar file needs to go to test->system profile, not master.
          Hide
          Balaji Rajagopalan added a comment -

          +1

          Show
          Balaji Rajagopalan added a comment - +1
          Hide
          Iyappan Srinivasan added a comment -

          patch for trunk

          Show
          Iyappan Srinivasan added a comment - patch for trunk
          Hide
          Iyappan Srinivasan added a comment -

          One minor change

          Show
          Iyappan Srinivasan added a comment - One minor change
          Hide
          Konstantin Boudnik added a comment -

          I think, this jar file needs to go to test->system profile, not master.

          Apparently there's no Ivy system profile in Common (unlike Hdfs and MR). However, this dependency needs to go to test instead of master.

          Show
          Konstantin Boudnik added a comment - I think, this jar file needs to go to test->system profile, not master. Apparently there's no Ivy system profile in Common (unlike Hdfs and MR). However, this dependency needs to go to test instead of master.
          Hide
          Konstantin Boudnik added a comment -

          Moving over (and fixing issues in the original patch) proposed functionality from MAPREDUCE-1882 and refitting original patch for the trunk.

          Remote exec API needs some extra work to be done on it especially in the part of getting user's identities, but I just want to have it in the same place for now.

          Show
          Konstantin Boudnik added a comment - Moving over (and fixing issues in the original patch) proposed functionality from MAPREDUCE-1882 and refitting original patch for the trunk. Remote exec API needs some extra work to be done on it especially in the part of getting user's identities, but I just want to have it in the same place for now.
          Hide
          Konstantin Boudnik added a comment -

          Patch for y20, not to commit here.

          Show
          Konstantin Boudnik added a comment - Patch for y20, not to commit here.
          Hide
          Konstantin Boudnik added a comment -

          The patch isn't final yet because there's not way to run framework validation tests from Ant

          Show
          Konstantin Boudnik added a comment - The patch isn't final yet because there's not way to run framework validation tests from Ant
          Hide
          Konstantin Boudnik added a comment -

          Used wrong ivy profile, apparently

          Show
          Konstantin Boudnik added a comment - Used wrong ivy profile, apparently
          Hide
          Konstantin Boudnik added a comment -

          Base implementing method should use portNumber variable instead of a constant.

          Show
          Konstantin Boudnik added a comment - Base implementing method should use portNumber variable instead of a constant.
          Hide
          Konstantin Boudnik added a comment -

          Updated based on Sharad's comments.

          Show
          Konstantin Boudnik added a comment - Updated based on Sharad's comments.
          Hide
          Konstantin Boudnik added a comment -

          Oops, published wrong files.

          Show
          Konstantin Boudnik added a comment - Oops, published wrong files.
          Hide
          Sharad Agarwal added a comment -

          Minor nits:
          1. The method needs to be implemented or removed:

          public void
              testExecuteCommandForRemoteHostNameUserCommandIdentityFilePortNumber()
              throws Exception {
              //TODO: Test goes here...
            }
          

          2. There shouldn't be any @author tags in the javadoc.

          Other than that patch looks fine.

          Show
          Sharad Agarwal added a comment - Minor nits: 1. The method needs to be implemented or removed: public void testExecuteCommandForRemoteHostNameUserCommandIdentityFilePortNumber() throws Exception { //TODO: Test goes here... } 2. There shouldn't be any @author tags in the javadoc. Other than that patch looks fine.
          Hide
          Konstantin Boudnik added a comment -

          Addressing Sharad's comments and adding Apache license boiler-plate to the test class.

          Show
          Konstantin Boudnik added a comment - Addressing Sharad's comments and adding Apache license boiler-plate to the test class.
          Hide
          Konstantin Boudnik added a comment -

          Same as before to make sure that test-patch picks up the correct patch.

          Show
          Konstantin Boudnik added a comment - Same as before to make sure that test-patch picks up the correct patch.
          Hide
          Konstantin Boudnik added a comment -

          Looks like Hudson is semi-dead again... Here's the results of test-patch

          -1 overall.  
          
              +1 @author.  The patch does not contain any @author tags.
          
              +1 tests included.  The patch appears to include 6 new or modified tests.
          
              -1 javadoc.  The javadoc tool appears to have generated 1 warning messages.
          
              +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
          
              +1 findbugs.  The patch does not introduce any new Findbugs warnings.
          
              +1 release audit.  The applied patch does not increase the total number of release audit warnings.
          
              +1 system tests framework.  The patch passed system tests framework compile.
          

          javadoc warning is unrelated: they all seem to be caused by security code

          Show
          Konstantin Boudnik added a comment - Looks like Hudson is semi-dead again... Here's the results of test-patch -1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 system tests framework. The patch passed system tests framework compile. javadoc warning is unrelated: they all seem to be caused by security code
          Hide
          Konstantin Boudnik added a comment -

          I have just committed it.

          Show
          Konstantin Boudnik added a comment - I have just committed it.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #395 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/395/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #395 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/395/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #489 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/489/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #489 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/489/ )

            People

            • Assignee:
              Konstantin Boudnik
              Reporter:
              Iyappan Srinivasan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development