Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12678

Handle empty rename pending metadata file during atomic rename in redo path

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs/azure
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Handle empty rename pending metadata file during atomic rename in redo path
      During atomic rename we create metadata file for rename(-renamePending.json). We create that in 2 steps
      1. We create an empty blob corresponding to the .json file in its real location
      2. We create a scratch file to which we write the contents of the rename pending which is then copied over into the blob described in 1
      If process crash occurs after step 1 and before step 2 is complete - we will be left with a zero size blob corresponding to the pending rename metadata file.
      This kind of scenario can happen in the /hbase/.tmp folder because it is considered a candidate folder for atomic rename. Now when HMaster starts up it executes listStatus on the .tmp folder to clean up pending data. At this stage due to the lazy pending rename complete process we look for these json files. On seeing an empty file the process simply throws a fatal exception assuming something went wrong.

      1. HADOOP-12678.001.patch
        12 kB
        madhumita chakraborty
      2. HADOOP-12678.002.patch
        6 kB
        madhumita chakraborty
      3. HADOOP-12678.003.patch
        6 kB
        madhumita chakraborty
      4. HADOOP-12678.004.patch
        6 kB
        madhumita chakraborty
      5. HADOOP-12678.005.patch
        6 kB
        madhumita chakraborty
      6. HADOOP-12678.006.patch
        6 kB
        madhumita chakraborty

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 3s HADOOP-12678 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779656/HADOOP-12678.001.patch
        JIRA Issue HADOOP-12678
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8316/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 3s HADOOP-12678 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779656/HADOOP-12678.001.patch JIRA Issue HADOOP-12678 Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8316/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 53s trunk passed
        +1 compile 0m 15s trunk passed with JDK v1.8.0_66
        +1 compile 0m 16s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 9s trunk passed
        +1 mvnsite 0m 22s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 32s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 16s the patch passed
        +1 compile 0m 12s the patch passed with JDK v1.8.0_66
        +1 javac 0m 12s the patch passed
        +1 compile 0m 14s the patch passed with JDK v1.7.0_91
        +1 javac 0m 14s the patch passed
        -1 checkstyle 0m 9s Patch generated 2 new checkstyle issues in hadoop-tools/hadoop-azure (total was 25, now 27).
        +1 mvnsite 0m 20s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        -1 findbugs 0m 38s hadoop-tools/hadoop-azure introduced 1 new FindBugs issues.
        -1 javadoc 1m 3s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 3 new issues (was 26, now 29).
        +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91
        +1 unit 1m 6s hadoop-azure in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 20s hadoop-azure in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 18s Patch does not generate ASF License warnings.
        16m 17s



        Reason Tests
        FindBugs module:hadoop-tools/hadoop-azure
          The method name org.apache.hadoop.fs.azure.NativeAzureFileSystem$FolderRenamePending.DeleteRenamePendingFile(FileSystem, Path) doesn't start with a lower case letter At NativeAzureFileSystem.java:doesn't start with a lower case letter At NativeAzureFileSystem.java:[lines 228-247]



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779672/HADOOP-12678.002.patch
        JIRA Issue HADOOP-12678
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 6ba7674ab390 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a0249da
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt
        findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/artifact/patchprocess/new-findbugs-hadoop-tools_hadoop-azure.html
        javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/testReport/
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Max memory used 75MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 53s trunk passed +1 compile 0m 15s trunk passed with JDK v1.8.0_66 +1 compile 0m 16s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 9s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 32s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 16s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_66 +1 javac 0m 12s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_91 +1 javac 0m 14s the patch passed -1 checkstyle 0m 9s Patch generated 2 new checkstyle issues in hadoop-tools/hadoop-azure (total was 25, now 27). +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 0m 38s hadoop-tools/hadoop-azure introduced 1 new FindBugs issues. -1 javadoc 1m 3s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 3 new issues (was 26, now 29). +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91 +1 unit 1m 6s hadoop-azure in the patch passed with JDK v1.8.0_66. +1 unit 1m 20s hadoop-azure in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 16m 17s Reason Tests FindBugs module:hadoop-tools/hadoop-azure   The method name org.apache.hadoop.fs.azure.NativeAzureFileSystem$FolderRenamePending.DeleteRenamePendingFile(FileSystem, Path) doesn't start with a lower case letter At NativeAzureFileSystem.java:doesn't start with a lower case letter At NativeAzureFileSystem.java: [lines 228-247] Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779672/HADOOP-12678.002.patch JIRA Issue HADOOP-12678 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6ba7674ab390 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a0249da Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/artifact/patchprocess/new-findbugs-hadoop-tools_hadoop-azure.html javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8318/console This message was automatically generated.
        Hide
        madhuch-ms madhumita chakraborty added a comment -

        Chris Nauroth,Gaurav Kanade,Pravin Mittal,Dushyanth could you guys please take a look at the patch?

        Show
        madhuch-ms madhumita chakraborty added a comment - Chris Nauroth , Gaurav Kanade , Pravin Mittal , Dushyanth could you guys please take a look at the patch?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 59s trunk passed
        +1 compile 0m 15s trunk passed with JDK v1.8.0_66
        +1 compile 0m 16s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 9s trunk passed
        +1 mvnsite 0m 23s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 32s trunk passed
        +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 16s the patch passed
        +1 compile 0m 12s the patch passed with JDK v1.8.0_66
        +1 javac 0m 12s the patch passed
        +1 compile 0m 14s the patch passed with JDK v1.7.0_91
        +1 javac 0m 14s the patch passed
        +1 checkstyle 0m 9s the patch passed
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 35s the patch passed
        -1 javadoc 1m 0s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26).
        +1 javadoc 0m 11s the patch passed with JDK v1.8.0_66
        -1 javadoc 1m 28s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1).
        +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91
        +1 unit 1m 6s hadoop-azure in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 20s hadoop-azure in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        16m 19s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779754/HADOOP-12678.003.patch
        JIRA Issue HADOOP-12678
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 4c967eb417ba 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d0a22ba
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8320/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt
        javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8320/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8320/testReport/
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Max memory used 75MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8320/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 59s trunk passed +1 compile 0m 15s trunk passed with JDK v1.8.0_66 +1 compile 0m 16s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 9s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 32s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 16s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_66 +1 javac 0m 12s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_91 +1 javac 0m 14s the patch passed +1 checkstyle 0m 9s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 35s the patch passed -1 javadoc 1m 0s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26). +1 javadoc 0m 11s the patch passed with JDK v1.8.0_66 -1 javadoc 1m 28s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1). +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91 +1 unit 1m 6s hadoop-azure in the patch passed with JDK v1.8.0_66. +1 unit 1m 20s hadoop-azure in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 16m 19s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779754/HADOOP-12678.003.patch JIRA Issue HADOOP-12678 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4c967eb417ba 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d0a22ba Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8320/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8320/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8320/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8320/console This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        madhumita chakraborty, thank you for the patch. I have a few notes.

        1. In NativeAzureFileSystem.FolderRenamePending#deleteRenamePendingFile, the exception handling logic is quite complex. There is an unchecked downcast to StorageException, which could cause a CastClassException. There is no check that the cause is non-null. I don't see why there is a nested catch (Exception e2), because I don't see a possibility of any exception being thrown in that block, unless this was put in place to mask the CastClassException. There is no need to wrap the original IOException in a whole new IOException before throwing it. This will only make the stack trace longer without adding new information. The code was also incorrectly indented, which made it difficult to read. I suggest simplifying the catch (IOException e) block to this:
                  Throwable cause = e.getCause();
                  if (cause != null && cause instanceof StorageException &&
                      "BlobNotFound".equals(((StorageException)cause).getErrorCode())) {
                    LOG.warn("rename pending file " + redoFile + " is already deleted");
                  } else {
                    throw e;
                  }
          
        2. Is NativeAzureFileSystem.FolderRenamePending#deleteRenamePendingFile marked public only so that the tests can call it? If so, then please make it package-private (remove public) and apply the VisibleForTesting annotation.
        3. Please ensure any lines that you are changing or adding are shorter than 80 characters.
        Show
        cnauroth Chris Nauroth added a comment - madhumita chakraborty , thank you for the patch. I have a few notes. In NativeAzureFileSystem.FolderRenamePending#deleteRenamePendingFile , the exception handling logic is quite complex. There is an unchecked downcast to StorageException , which could cause a CastClassException . There is no check that the cause is non-null. I don't see why there is a nested catch (Exception e2) , because I don't see a possibility of any exception being thrown in that block, unless this was put in place to mask the CastClassException . There is no need to wrap the original IOException in a whole new IOException before throwing it. This will only make the stack trace longer without adding new information. The code was also incorrectly indented, which made it difficult to read. I suggest simplifying the catch (IOException e) block to this: Throwable cause = e.getCause(); if (cause != null && cause instanceof StorageException && "BlobNotFound" .equals(((StorageException)cause).getErrorCode())) { LOG.warn( "rename pending file " + redoFile + " is already deleted" ); } else { throw e; } Is NativeAzureFileSystem.FolderRenamePending#deleteRenamePendingFile marked public only so that the tests can call it? If so, then please make it package-private (remove public ) and apply the VisibleForTesting annotation. Please ensure any lines that you are changing or adding are shorter than 80 characters.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 39s trunk passed
        +1 compile 0m 15s trunk passed with JDK v1.8.0_66
        +1 compile 0m 15s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 9s trunk passed
        +1 mvnsite 0m 21s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 15s the patch passed
        +1 compile 0m 12s the patch passed with JDK v1.8.0_66
        +1 javac 0m 12s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_91
        +1 javac 0m 13s the patch passed
        +1 checkstyle 0m 8s the patch passed
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 32s the patch passed
        -1 javadoc 1m 0s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26).
        +1 javadoc 0m 12s the patch passed with JDK v1.8.0_66
        -1 javadoc 1m 26s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1).
        +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91
        +1 unit 1m 10s hadoop-azure in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 14s hadoop-azure in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 16s Patch does not generate ASF License warnings.
        15m 42s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780770/HADOOP-12678.004.patch
        JIRA Issue HADOOP-12678
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c1e1817106f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c1462a6
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/artifact/patchprocess/whitespace-eol.txt
        javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt
        javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/testReport/
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Max memory used 75MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 39s trunk passed +1 compile 0m 15s trunk passed with JDK v1.8.0_66 +1 compile 0m 15s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 9s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 15s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_66 +1 javac 0m 12s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_91 +1 javac 0m 13s the patch passed +1 checkstyle 0m 8s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 9s the patch passed -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 32s the patch passed -1 javadoc 1m 0s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26). +1 javadoc 0m 12s the patch passed with JDK v1.8.0_66 -1 javadoc 1m 26s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1). +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91 +1 unit 1m 10s hadoop-azure in the patch passed with JDK v1.8.0_66. +1 unit 1m 14s hadoop-azure in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 16s Patch does not generate ASF License warnings. 15m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780770/HADOOP-12678.004.patch JIRA Issue HADOOP-12678 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c1e1817106f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c1462a6 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/artifact/patchprocess/whitespace-eol.txt javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8348/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 26s trunk passed
        +1 compile 0m 14s trunk passed with JDK v1.8.0_66
        +1 compile 0m 17s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 9s trunk passed
        +1 mvnsite 0m 23s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 30s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 17s the patch passed
        +1 compile 0m 12s the patch passed with JDK v1.8.0_66
        +1 javac 0m 12s the patch passed
        +1 compile 0m 14s the patch passed with JDK v1.7.0_91
        +1 javac 0m 14s the patch passed
        +1 checkstyle 0m 8s the patch passed
        +1 mvnsite 0m 20s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 37s the patch passed
        -1 javadoc 1m 3s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26).
        +1 javadoc 0m 11s the patch passed with JDK v1.8.0_66
        -1 javadoc 1m 32s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1).
        +1 javadoc 0m 13s the patch passed with JDK v1.7.0_91
        +1 unit 1m 12s hadoop-azure in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 23s hadoop-azure in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        17m 0s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780788/HADOOP-12678.005.patch
        JIRA Issue HADOOP-12678
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b10c208759b0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c1462a6
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8349/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt
        javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8349/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8349/testReport/
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8349/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 26s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_66 +1 compile 0m 17s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 9s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 17s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_66 +1 javac 0m 12s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_91 +1 javac 0m 14s the patch passed +1 checkstyle 0m 8s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 37s the patch passed -1 javadoc 1m 3s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26). +1 javadoc 0m 11s the patch passed with JDK v1.8.0_66 -1 javadoc 1m 32s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1). +1 javadoc 0m 13s the patch passed with JDK v1.7.0_91 +1 unit 1m 12s hadoop-azure in the patch passed with JDK v1.8.0_66. +1 unit 1m 23s hadoop-azure in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 17m 0s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780788/HADOOP-12678.005.patch JIRA Issue HADOOP-12678 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b10c208759b0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c1462a6 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8349/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8349/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8349/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8349/console This message was automatically generated.
        Hide
        madhuch-ms madhumita chakraborty added a comment -

        Chris Nauroth I have addressed your comments. Could you please take a look?

        Show
        madhuch-ms madhumita chakraborty added a comment - Chris Nauroth I have addressed your comments. Could you please take a look?
        Hide
        cnauroth Chris Nauroth added a comment -

        Thank you, madhumita chakraborty. The error handling in deleteRenamePendingFile still needs some work. Here is the code from your v005 patch.

              } catch (IOException e) {
                // If the rename metadata was not found then somebody probably
                // raced with us and finished the delete first
                Throwable t = e.getCause();
                if (t != null && t instanceof StorageException) {
                  StorageException se = (StorageException) t;
                  if (se.getErrorCode().equals(("BlobNotFound"))) {
                    LOG.warn("rename pending file " + redoFile + " is already deleted");
                  } else {
                    throw e;
                  }
                }
              }
        

        If there is a general IOException not caused by an Azure StorageException, then this logic would stifle the exception without either throwing it or logging it. An example of this could be loss of network connectivity to the Azure Storage backend, which Java would report as an IOException with no cause and a message describing the network error. We'd want to make sure errors like this propagate to the caller, so please stick with the code I gave in my last comment:

              } catch (IOException e) {
                Throwable cause = e.getCause();
                if (cause != null && cause instanceof StorageException &&
                    "BlobNotFound".equals(((StorageException)cause).getErrorCode())) {
                  LOG.warn("rename pending file " + redoFile + " is already deleted");
                } else {
                  throw e;
                }
              }
        

        This ensures that only the BlobNotFound error would get swallowed, and any other IOException, whether or not its root cause is in Azure Storage, would propagate to the caller. It also clarifies that there are really only two cases for this code: swallow BlobNotFound, else rethrow.

        The JavaDoc warnings from the last pre-commit run don't require any action. These are pre-existing warnings unrelated to this patch. The patch is shifting the line numbers and therefore making it appear that new warnings were introduced.

        Show
        cnauroth Chris Nauroth added a comment - Thank you, madhumita chakraborty . The error handling in deleteRenamePendingFile still needs some work. Here is the code from your v005 patch. } catch (IOException e) { // If the rename metadata was not found then somebody probably // raced with us and finished the delete first Throwable t = e.getCause(); if (t != null && t instanceof StorageException) { StorageException se = (StorageException) t; if (se.getErrorCode().equals(( "BlobNotFound" ))) { LOG.warn( "rename pending file " + redoFile + " is already deleted" ); } else { throw e; } } } If there is a general IOException not caused by an Azure StorageException , then this logic would stifle the exception without either throwing it or logging it. An example of this could be loss of network connectivity to the Azure Storage backend, which Java would report as an IOException with no cause and a message describing the network error. We'd want to make sure errors like this propagate to the caller, so please stick with the code I gave in my last comment: } catch (IOException e) { Throwable cause = e.getCause(); if (cause != null && cause instanceof StorageException && "BlobNotFound" .equals(((StorageException)cause).getErrorCode())) { LOG.warn( "rename pending file " + redoFile + " is already deleted" ); } else { throw e; } } This ensures that only the BlobNotFound error would get swallowed, and any other IOException , whether or not its root cause is in Azure Storage, would propagate to the caller. It also clarifies that there are really only two cases for this code: swallow BlobNotFound, else rethrow. The JavaDoc warnings from the last pre-commit run don't require any action. These are pre-existing warnings unrelated to this patch. The patch is shifting the line numbers and therefore making it appear that new warnings were introduced.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 44s trunk passed
        +1 compile 0m 14s trunk passed with JDK v1.8.0_66
        +1 compile 0m 16s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 8s trunk passed
        +1 mvnsite 0m 22s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 30s trunk passed
        +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 15s the patch passed
        +1 compile 0m 11s the patch passed with JDK v1.8.0_66
        +1 javac 0m 11s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_91
        +1 javac 0m 13s the patch passed
        -1 checkstyle 0m 8s Patch generated 2 new checkstyle issues in hadoop-tools/hadoop-azure (total was 25, now 27).
        +1 mvnsite 0m 18s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 35s the patch passed
        -1 javadoc 0m 59s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26).
        +1 javadoc 0m 9s the patch passed with JDK v1.8.0_66
        -1 javadoc 1m 26s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1).
        +1 javadoc 0m 11s the patch passed with JDK v1.7.0_91
        +1 unit 1m 1s hadoop-azure in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 17s hadoop-azure in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        15m 44s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780895/HADOOP-12678.006.patch
        JIRA Issue HADOOP-12678
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 25de3887313a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 791c163
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt
        javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt
        javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/testReport/
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 44s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_66 +1 compile 0m 16s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 8s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 15s the patch passed +1 compile 0m 11s the patch passed with JDK v1.8.0_66 +1 javac 0m 11s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_91 +1 javac 0m 13s the patch passed -1 checkstyle 0m 8s Patch generated 2 new checkstyle issues in hadoop-tools/hadoop-azure (total was 25, now 27). +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 35s the patch passed -1 javadoc 0m 59s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26). +1 javadoc 0m 9s the patch passed with JDK v1.8.0_66 -1 javadoc 1m 26s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1). +1 javadoc 0m 11s the patch passed with JDK v1.7.0_91 +1 unit 1m 1s hadoop-azure in the patch passed with JDK v1.8.0_66. +1 unit 1m 17s hadoop-azure in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 15m 44s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780895/HADOOP-12678.006.patch JIRA Issue HADOOP-12678 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 25de3887313a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 791c163 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8351/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 43s trunk passed
        +1 compile 0m 15s trunk passed with JDK v1.8.0_66
        +1 compile 0m 16s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 8s trunk passed
        +1 mvnsite 0m 21s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 15s the patch passed
        +1 compile 0m 11s the patch passed with JDK v1.8.0_66
        +1 javac 0m 11s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_91
        +1 javac 0m 13s the patch passed
        +1 checkstyle 0m 8s the patch passed
        +1 mvnsite 0m 18s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 34s the patch passed
        -1 javadoc 0m 57s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26).
        +1 javadoc 0m 9s the patch passed with JDK v1.8.0_66
        -1 javadoc 1m 24s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1).
        +1 javadoc 0m 11s the patch passed with JDK v1.7.0_91
        +1 unit 1m 1s hadoop-azure in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 14s hadoop-azure in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 18s Patch does not generate ASF License warnings.
        15m 36s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780899/HADOOP-12678.006.patch
        JIRA Issue HADOOP-12678
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 96b06f85ca1d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 791c163
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8352/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt
        javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8352/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8352/testReport/
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8352/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 43s trunk passed +1 compile 0m 15s trunk passed with JDK v1.8.0_66 +1 compile 0m 16s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 8s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 15s the patch passed +1 compile 0m 11s the patch passed with JDK v1.8.0_66 +1 javac 0m 11s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_91 +1 javac 0m 13s the patch passed +1 checkstyle 0m 8s the patch passed +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 34s the patch passed -1 javadoc 0m 57s hadoop-tools_hadoop-azure-jdk1.8.0_66 with JDK v1.8.0_66 generated 18 new issues (was 26, now 26). +1 javadoc 0m 9s the patch passed with JDK v1.8.0_66 -1 javadoc 1m 24s hadoop-tools_hadoop-azure-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 1, now 1). +1 javadoc 0m 11s the patch passed with JDK v1.7.0_91 +1 unit 1m 1s hadoop-azure in the patch passed with JDK v1.8.0_66. +1 unit 1m 14s hadoop-azure in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 15m 36s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780899/HADOOP-12678.006.patch JIRA Issue HADOOP-12678 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 96b06f85ca1d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 791c163 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 javadoc hadoop-tools_hadoop-azure-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8352/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.8.0_66.txt javadoc hadoop-tools_hadoop-azure-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8352/artifact/patchprocess/diff-javadoc-javadoc-hadoop-tools_hadoop-azure-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8352/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8352/console This message was automatically generated.
        Hide
        madhuch-ms madhumita chakraborty added a comment -

        Chris Nauroth Thanks Chris! It was my bad. I have fixed it. Could you please take a look?

        Show
        madhuch-ms madhumita chakraborty added a comment - Chris Nauroth Thanks Chris! It was my bad. I have fixed it. Could you please take a look?
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for patch v06. I have committed this to trunk, branch-2 and branch-2.8. madhumita chakraborty, thank you for contributing this patch.

        Show
        cnauroth Chris Nauroth added a comment - +1 for patch v06. I have committed this to trunk, branch-2 and branch-2.8. madhumita chakraborty , thank you for contributing this patch.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9076 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9076/)
        HADOOP-12678. Handle empty rename pending metadata file during atomic (cnauroth: rev f0fa6d869b9abb5a900ea1c9eb4eb19ec9831dc4)

        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/NativeAzureFileSystemBaseTest.java
        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9076 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9076/ ) HADOOP-12678 . Handle empty rename pending metadata file during atomic (cnauroth: rev f0fa6d869b9abb5a900ea1c9eb4eb19ec9831dc4) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/NativeAzureFileSystemBaseTest.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt

          People

          • Assignee:
            madhuch-ms madhumita chakraborty
            Reporter:
            madhuch-ms madhumita chakraborty
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development