Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12717

NPE when trying to rename a directory in Windows Azure Storage FileSystem

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Encountered an NPE when trying to use the HBase utility ExportSnapshot with Azure as the target.

      It turns out verifyAndConvertToStandardFormat is returning null when determining the hbaseRoot, and this is being added to the atomicRenameDirs set.

      java.lang.NullPointerException
      at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.isKeyForDirectorySet(AzureNativeFileSystemStore.java:1059)
      at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.isAtomicRenameKey(AzureNativeFileSystemStore.java:1053)
      at org.apache.hadoop.fs.azure.NativeAzureFileSystem.prepareAtomicFolderRename(NativeAzureFileSystem.java:2098)
      at org.apache.hadoop.fs.azure.NativeAzureFileSystem.rename(NativeAzureFileSystem.java:1996)
      at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:944)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      at com.yammer.calmie.snapshot.AbstractSnapshotUtil.exportSnapshot(AbstractSnapshotUtil.java:210)
      at com.yammer.calmie.snapshot.AbstractSnapshotUtil.run(AbstractSnapshotUtil.java:79)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      at com.yammer.calmie.snapshot.SnapshotAzureBlobUtil.main(SnapshotAzureBlobUtil.java:85)

      1. HADOOP-12717.03.patch
        3 kB
        Gaurav Kanade
      2. HADOOP-12717.02.patch
        3 kB
        Gaurav Kanade
      3. HADOOP-12717.004.patch
        3 kB
        Chris Nauroth
      4. HADOOP-12717.001.patch
        1.0 kB
        Chris Nauroth
      5. diff.txt
        1.0 kB
        Robert Yokota

        Activity

        Hide
        gouravk Gaurav Kanade added a comment -

        Can you specify how exactly you are trying to use the ExportSnapshot - i.e. what is the copy-to parameter in your ExportSnapshot? Just want to get a better idea to correctly identify the interactions with our WASB code path

        Show
        gouravk Gaurav Kanade added a comment - Can you specify how exactly you are trying to use the ExportSnapshot - i.e. what is the copy-to parameter in your ExportSnapshot? Just want to get a better idea to correctly identify the interactions with our WASB code path
        Hide
        rayokota Robert Yokota added a comment -

        The copy-to param was of the form wasbs://<container>@<account>.blob.core.windows.net.

        I had to workaround the NPE by specifying "snapshot.export.skip.tmp" as "true", to avoid the code path that tries to do a rename.

        Show
        rayokota Robert Yokota added a comment - The copy-to param was of the form wasbs://<container>@<account>.blob.core.windows.net. I had to workaround the NPE by specifying "snapshot.export.skip.tmp" as "true", to avoid the code path that tries to do a rename.
        Hide
        rayokota Robert Yokota added a comment -

        Or rather wasbs://<container>@<account>.blob.core.windows.net/<path>

        Show
        rayokota Robert Yokota added a comment - Or rather wasbs://<container>@<account>.blob.core.windows.net/<path>
        Hide
        gouravk Gaurav Kanade added a comment -

        Robert YokotaChris Nauroth It seems other customers are too hitting this issue and is pretty annoying; this needs to be fixed. At first glance the patch seems fine to me.

        However not sure if it may break some other dependencies in the WASB driver so adding Dushyanth to take a look and review

        Show
        gouravk Gaurav Kanade added a comment - Robert Yokota Chris Nauroth It seems other customers are too hitting this issue and is pretty annoying; this needs to be fixed. At first glance the patch seems fine to me. However not sure if it may break some other dependencies in the WASB driver so adding Dushyanth to take a look and review
        Hide
        cnauroth Chris Nauroth added a comment -

        I'm attaching the exact same patch file, renamed to conform to the pre-commit test job's expectations.

        Show
        cnauroth Chris Nauroth added a comment - I'm attaching the exact same patch file, renamed to conform to the pre-commit test job's expectations.
        Hide
        gouravk Gaurav Kanade added a comment -

        madhumita chakraborty could you please also review? pending any concerns you or Dushyanth may find it has a +1 from me

        Show
        gouravk Gaurav Kanade added a comment - madhumita chakraborty could you please also review? pending any concerns you or Dushyanth may find it has a +1 from me
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 26s trunk passed
        +1 compile 0m 18s trunk passed with JDK v1.8.0_72
        +1 compile 0m 15s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 13s trunk passed
        +1 mvnsite 0m 22s trunk passed
        +1 mvneclipse 0m 11s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 15s trunk passed with JDK v1.8.0_72
        +1 javadoc 0m 13s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 15s the patch passed
        +1 compile 0m 16s the patch passed with JDK v1.8.0_72
        +1 javac 0m 16s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_95
        +1 javac 0m 13s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 39s the patch passed
        +1 javadoc 0m 12s the patch passed with JDK v1.8.0_72
        +1 javadoc 0m 11s the patch passed with JDK v1.7.0_95
        +1 unit 1m 24s hadoop-azure in the patch passed with JDK v1.8.0_72.
        +1 unit 1m 29s hadoop-azure in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        16m 30s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790815/HADOOP-12717.001.patch
        JIRA Issue HADOOP-12717
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 6d1d45018570 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4abb2fa
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8759/testReport/
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8759/console
        Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 26s trunk passed +1 compile 0m 18s trunk passed with JDK v1.8.0_72 +1 compile 0m 15s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 11s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 15s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 13s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 15s the patch passed +1 compile 0m 16s the patch passed with JDK v1.8.0_72 +1 javac 0m 16s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_95 +1 javac 0m 13s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 39s the patch passed +1 javadoc 0m 12s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 11s the patch passed with JDK v1.7.0_95 +1 unit 1m 24s hadoop-azure in the patch passed with JDK v1.8.0_72. +1 unit 1m 29s hadoop-azure in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 16m 30s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790815/HADOOP-12717.001.patch JIRA Issue HADOOP-12717 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6d1d45018570 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4abb2fa Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8759/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8759/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        madhuch-ms madhumita chakraborty added a comment -

        Looks good to me. Would it be possible to add one unit test for this scenario?

        Show
        madhuch-ms madhumita chakraborty added a comment - Looks good to me. Would it be possible to add one unit test for this scenario?
        Hide
        gouravk Gaurav Kanade added a comment -

        Thanks madhumita chakraborty; Robert Yokota has provided the original patch and I'm not sure what test we can add; however I spoke to Dushyanth to get this signed off and +1 from me again; it seems this issue will always be hit when multiple file systems interact in scenarios such as export and so on; recently several people are facing this and we need to move fast on this.

        Show
        gouravk Gaurav Kanade added a comment - Thanks madhumita chakraborty ; Robert Yokota has provided the original patch and I'm not sure what test we can add; however I spoke to Dushyanth to get this signed off and +1 from me again; it seems this issue will always be hit when multiple file systems interact in scenarios such as export and so on; recently several people are facing this and we need to move fast on this.
        Hide
        dchickabasapa Dushyanth added a comment -

        +1 on the Patch.

        Show
        dchickabasapa Dushyanth added a comment - +1 on the Patch.
        Hide
        cnauroth Chris Nauroth added a comment -

        The proposed change makes sense to me. (Note that AzureNativeFileSystemStore#getDirectorySet also calls verifyAndConvertToStandardFormat, and that already has a similar null check.)

        I'm not sure what test we can add

        It seems like you ought to be able to write a unit test that initializes an instance of AzureNativeFileSystemStore using configuration that would cause verifyAndConvertToStandardFormat to return null. Calling isAtomicRenameKey right after that would trigger a NullPointerException like shown in the issue description. Then, after applying the null check fix, that same test would no longer throw a NullPointerException, so the test would pass. Did you try exploring this?

        Show
        cnauroth Chris Nauroth added a comment - The proposed change makes sense to me. (Note that AzureNativeFileSystemStore#getDirectorySet also calls verifyAndConvertToStandardFormat , and that already has a similar null check.) I'm not sure what test we can add It seems like you ought to be able to write a unit test that initializes an instance of AzureNativeFileSystemStore using configuration that would cause verifyAndConvertToStandardFormat to return null . Calling isAtomicRenameKey right after that would trigger a NullPointerException like shown in the issue description. Then, after applying the null check fix, that same test would no longer throw a NullPointerException , so the test would pass. Did you try exploring this?
        Hide
        gouravk Gaurav Kanade added a comment -

        Chris Nauroth Added new patch with test with given specs

        Show
        gouravk Gaurav Kanade added a comment - Chris Nauroth Added new patch with test with given specs
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 31s trunk passed
        +1 compile 0m 14s trunk passed with JDK v1.8.0_74
        +1 compile 0m 16s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 13s trunk passed
        +1 mvnsite 0m 20s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 30s trunk passed
        +1 javadoc 0m 12s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 16s the patch passed
        +1 compile 0m 11s the patch passed with JDK v1.8.0_74
        +1 javac 0m 11s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_95
        +1 javac 0m 13s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 18s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 37s the patch passed
        +1 javadoc 0m 9s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 11s the patch passed with JDK v1.7.0_95
        +1 unit 1m 9s hadoop-azure in the patch passed with JDK v1.8.0_74.
        +1 unit 1m 26s hadoop-azure in the patch passed with JDK v1.7.0_95.
        -1 asflicense 0m 17s Patch generated 1 ASF License warnings.
        14m 56s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791595/HADOOP-12717.02.patch
        JIRA Issue HADOOP-12717
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 4dbdbf9ff067 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2759689
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8794/artifact/patchprocess/whitespace-eol.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8794/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8794/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8794/console
        Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 31s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_74 +1 compile 0m 16s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 20s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 16s the patch passed +1 compile 0m 11s the patch passed with JDK v1.8.0_74 +1 javac 0m 11s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_95 +1 javac 0m 13s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 9s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 37s the patch passed +1 javadoc 0m 9s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 11s the patch passed with JDK v1.7.0_95 +1 unit 1m 9s hadoop-azure in the patch passed with JDK v1.8.0_74. +1 unit 1m 26s hadoop-azure in the patch passed with JDK v1.7.0_95. -1 asflicense 0m 17s Patch generated 1 ASF License warnings. 14m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791595/HADOOP-12717.02.patch JIRA Issue HADOOP-12717 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4dbdbf9ff067 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2759689 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8794/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8794/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8794/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8794/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 8s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 37s trunk passed
        +1 compile 0m 14s trunk passed with JDK v1.8.0_74
        +1 compile 0m 16s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 13s trunk passed
        +1 mvnsite 0m 20s trunk passed
        +1 mvneclipse 0m 11s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 12s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 15s the patch passed
        +1 compile 0m 12s the patch passed with JDK v1.8.0_74
        +1 javac 0m 12s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_95
        +1 javac 0m 13s the patch passed
        +1 checkstyle 0m 10s the patch passed
        +1 mvnsite 0m 18s the patch passed
        +1 mvneclipse 0m 8s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 38s the patch passed
        +1 javadoc 0m 10s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 11s the patch passed with JDK v1.7.0_95
        +1 unit 1m 14s hadoop-azure in the patch passed with JDK v1.8.0_74.
        +1 unit 1m 29s hadoop-azure in the patch passed with JDK v1.7.0_95.
        -1 asflicense 0m 16s Patch generated 1 ASF License warnings.
        15m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791600/HADOOP-12717.03.patch
        JIRA Issue HADOOP-12717
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 136fa363ae95 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2759689
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8795/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8795/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8795/console
        Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 37s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_74 +1 compile 0m 16s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 20s trunk passed +1 mvneclipse 0m 11s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 15s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_74 +1 javac 0m 12s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_95 +1 javac 0m 13s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 8s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 38s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 11s the patch passed with JDK v1.7.0_95 +1 unit 1m 14s hadoop-azure in the patch passed with JDK v1.8.0_74. +1 unit 1m 29s hadoop-azure in the patch passed with JDK v1.7.0_95. -1 asflicense 0m 16s Patch generated 1 ASF License warnings. 15m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791600/HADOOP-12717.03.patch JIRA Issue HADOOP-12717 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 136fa363ae95 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2759689 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8795/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8795/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8795/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        The patch looks good aside from the license violation that was flagged by the last pre-commit run. I am attaching patch v004, which is the same code, with the addition of the Apache license to the new test class.

        Show
        cnauroth Chris Nauroth added a comment - The patch looks good aside from the license violation that was flagged by the last pre-commit run. I am attaching patch v004, which is the same code, with the addition of the Apache license to the new test class.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 2s trunk passed
        +1 compile 0m 15s trunk passed with JDK v1.8.0_74
        +1 compile 0m 17s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 21s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 34s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 16s the patch passed
        +1 compile 0m 11s the patch passed with JDK v1.8.0_74
        +1 javac 0m 11s the patch passed
        +1 compile 0m 14s the patch passed with JDK v1.7.0_95
        +1 javac 0m 14s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 42s the patch passed
        +1 javadoc 0m 10s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95
        +1 unit 1m 17s hadoop-azure in the patch passed with JDK v1.8.0_74.
        +1 unit 1m 36s hadoop-azure in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        17m 18s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791610/HADOOP-12717.004.patch
        JIRA Issue HADOOP-12717
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c5bf47318a2f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2759689
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8797/testReport/
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8797/console
        Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 2s trunk passed +1 compile 0m 15s trunk passed with JDK v1.8.0_74 +1 compile 0m 17s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 34s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 16s the patch passed +1 compile 0m 11s the patch passed with JDK v1.8.0_74 +1 javac 0m 11s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_95 +1 javac 0m 14s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 42s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95 +1 unit 1m 17s hadoop-azure in the patch passed with JDK v1.8.0_74. +1 unit 1m 36s hadoop-azure in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 17m 18s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791610/HADOOP-12717.004.patch JIRA Issue HADOOP-12717 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c5bf47318a2f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2759689 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8797/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8797/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 17s trunk passed
        +1 compile 0m 19s trunk passed with JDK v1.8.0_74
        +1 compile 0m 17s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 23s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 34s trunk passed
        +1 javadoc 0m 16s trunk passed with JDK v1.8.0_74
        +1 javadoc 0m 19s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 24s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.8.0_74
        +1 javac 0m 26s the patch passed
        +1 compile 0m 22s the patch passed with JDK v1.7.0_95
        +1 javac 0m 22s the patch passed
        +1 checkstyle 0m 16s the patch passed
        +1 mvnsite 0m 28s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 54s the patch passed
        +1 javadoc 0m 21s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 18s the patch passed with JDK v1.7.0_95
        +1 unit 1m 54s hadoop-azure in the patch passed with JDK v1.8.0_74.
        +1 unit 1m 49s hadoop-azure in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        20m 26s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791610/HADOOP-12717.004.patch
        JIRA Issue HADOOP-12717
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux fd3b05b8b14c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2759689
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8796/testReport/
        modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8796/console
        Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 17s trunk passed +1 compile 0m 19s trunk passed with JDK v1.8.0_74 +1 compile 0m 17s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 34s trunk passed +1 javadoc 0m 16s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 19s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 24s the patch passed +1 compile 0m 26s the patch passed with JDK v1.8.0_74 +1 javac 0m 26s the patch passed +1 compile 0m 22s the patch passed with JDK v1.7.0_95 +1 javac 0m 22s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 28s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 54s the patch passed +1 javadoc 0m 21s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 18s the patch passed with JDK v1.7.0_95 +1 unit 1m 54s hadoop-azure in the patch passed with JDK v1.8.0_74. +1 unit 1m 49s hadoop-azure in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 20m 26s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791610/HADOOP-12717.004.patch JIRA Issue HADOOP-12717 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fd3b05b8b14c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2759689 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8796/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8796/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for patch v004. I have committed this to trunk, branch-2 and branch-2.8. Robert Yokota and Gaurav Kanade, thank you for the patch.

        Show
        cnauroth Chris Nauroth added a comment - +1 for patch v004. I have committed this to trunk, branch-2 and branch-2.8. Robert Yokota and Gaurav Kanade , thank you for the patch.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9427 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9427/)
        HADOOP-12717. NPE when trying to rename a directory in Windows Azure (cnauroth: rev c50aad0f854b74ede9668e35db314b0a93be81b2)

        • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
        • hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestNativeAzureFileSystemAtomicRenameDirList.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9427 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9427/ ) HADOOP-12717 . NPE when trying to rename a directory in Windows Azure (cnauroth: rev c50aad0f854b74ede9668e35db314b0a93be81b2) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestNativeAzureFileSystemAtomicRenameDirList.java

          People

          • Assignee:
            rayokota Robert Yokota
            Reporter:
            rayokota Robert Yokota
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development