Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: ha
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Fencer implementation that uses PowerShell to fence a remote component.

      1. HADOOP-14309-001.patch
        6 kB
        Íñigo Goiri
      2. HADOOP-14309-000.patch
        6 kB
        Íñigo Goiri

        Issue Links

          Activity

          Hide
          elgoiri Íñigo Goiri added a comment -

          Fencer method that uses PowerShell to remotely connect to a machine and kill the required process.

          Show
          elgoiri Íñigo Goiri added a comment - Fencer method that uses PowerShell to remotely connect to a machine and kill the required process.
          Hide
          elgoiri Íñigo Goiri added a comment -

          To enable the fencing mechanism for HDFS HA, one could set:

          <property>
            <name>dfs.ha.fencing.methods</name>
            <value>powershell(NameNode)</value>
          </property>
          
          Show
          elgoiri Íñigo Goiri added a comment - To enable the fencing mechanism for HDFS HA, one could set: <property> <name>dfs.ha.fencing.methods</name> <value>powershell(NameNode)</value> </property>
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 13m 26s trunk passed
          +1 compile 15m 31s trunk passed
          +1 checkstyle 0m 36s trunk passed
          +1 mvnsite 1m 15s trunk passed
          +1 mvneclipse 0m 21s trunk passed
          +1 findbugs 1m 26s trunk passed
          +1 javadoc 0m 48s trunk passed
          +1 mvninstall 0m 38s the patch passed
          +1 compile 13m 52s the patch passed
          +1 javac 13m 52s the patch passed
          +1 checkstyle 0m 35s the patch passed
          +1 mvnsite 1m 2s the patch passed
          +1 mvneclipse 0m 20s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 38s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 0m 51s the patch passed
          -1 unit 8m 4s hadoop-common in the patch failed.
          +1 asflicense 0m 34s The patch does not generate ASF License warnings.
          63m 4s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            Found reliance on default encoding in org.apache.hadoop.ha.PowerShellFencer.buildPSScript(String, String):in org.apache.hadoop.ha.PowerShellFencer.buildPSScript(String, String): new java.io.FileWriter(File) At PowerShellFencer.java:[line 109]
          Failed junit tests hadoop.security.TestKDiag



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:612578f
          JIRA Issue HADOOP-14309
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863504/HADOOP-14309-000.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3145e4a5d21a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a41f8dd
          Default Java 1.8.0_121
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12105/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12105/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12105/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12105/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 13m 26s trunk passed +1 compile 15m 31s trunk passed +1 checkstyle 0m 36s trunk passed +1 mvnsite 1m 15s trunk passed +1 mvneclipse 0m 21s trunk passed +1 findbugs 1m 26s trunk passed +1 javadoc 0m 48s trunk passed +1 mvninstall 0m 38s the patch passed +1 compile 13m 52s the patch passed +1 javac 13m 52s the patch passed +1 checkstyle 0m 35s the patch passed +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 38s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 0m 51s the patch passed -1 unit 8m 4s hadoop-common in the patch failed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 63m 4s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   Found reliance on default encoding in org.apache.hadoop.ha.PowerShellFencer.buildPSScript(String, String):in org.apache.hadoop.ha.PowerShellFencer.buildPSScript(String, String): new java.io.FileWriter(File) At PowerShellFencer.java: [line 109] Failed junit tests hadoop.security.TestKDiag Subsystem Report/Notes Docker Image:yetus/hadoop:612578f JIRA Issue HADOOP-14309 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863504/HADOOP-14309-000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3145e4a5d21a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a41f8dd Default Java 1.8.0_121 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/12105/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/12105/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/12105/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/12105/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Looks reasonable, given my rudimentary PowerShell. Does the commandline include some unique string? Should this fail if this matches more than one process at the remote host?

          +1 if the findbugs warning is fixed. This could also call deleteOnExit on the temp file instance.

          Show
          chris.douglas Chris Douglas added a comment - Looks reasonable, given my rudimentary PowerShell. Does the commandline include some unique string? Should this fail if this matches more than one process at the remote host? +1 if the findbugs warning is fixed. This could also call deleteOnExit on the temp file instance.
          Hide
          elgoiri Íñigo Goiri added a comment -

          Thanks Chris Douglas for the review.
          The filter is kind of tricky because it should parse the whole class name which is passed to java.exe to start.
          In this case, I'm leaving the option for the administrator to set whatever in the CommandLine filter.
          The current SSH bases it on the port number.

          I'll fix the findbugs and the deleteOnExit.

          Show
          elgoiri Íñigo Goiri added a comment - Thanks Chris Douglas for the review. The filter is kind of tricky because it should parse the whole class name which is passed to java.exe to start. In this case, I'm leaving the option for the administrator to set whatever in the CommandLine filter. The current SSH bases it on the port number. I'll fix the findbugs and the deleteOnExit .
          Hide
          elgoiri Íñigo Goiri added a comment -
          • Fixed findbugs
          • Added deleteOnExit
          • Added more information to the comments
          Show
          elgoiri Íñigo Goiri added a comment - Fixed findbugs Added deleteOnExit Added more information to the comments
          Hide
          chris.douglas Chris Douglas added a comment -

          +1 I committed this. Thanks, Inigo

          Show
          chris.douglas Chris Douglas added a comment - +1 I committed this. Thanks, Inigo
          Hide
          elgoiri Íñigo Goiri added a comment -

          Chris Douglas, thanks for the review and the commit.

          Show
          elgoiri Íñigo Goiri added a comment - Chris Douglas , thanks for the review and the commit.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11644 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11644/)
          HADOOP-14309. Add PowerShell NodeFencer. Contributed by Inigo Goiri (cdouglas: rev 31cf480d6520e259f2a7f29de64ef4c5b5a50142)

          • (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/PowerShellFencer.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/NodeFencer.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11644 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11644/ ) HADOOP-14309 . Add PowerShell NodeFencer. Contributed by Inigo Goiri (cdouglas: rev 31cf480d6520e259f2a7f29de64ef4c5b5a50142) (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/PowerShellFencer.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/NodeFencer.java

            People

            • Assignee:
              elgoiri Íñigo Goiri
              Reporter:
              elgoiri Íñigo Goiri
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development