Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: fs/azure
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Current implementation of WASB, only supports Azure storage keys and SAS key being provided via org.apache.hadoop.conf.Configuration, which results in these secrets residing in the same address space as the WASB process and providing complete access to the Azure storage account and its containers. Added to the fact that WASB does not inherently support ACL's, WASB is its current implementation cannot be securely used for environments like secure hadoop cluster. This JIRA is created to add a new mode in WASB, which operates on Azure Storage SAS keys, which can provide fine grained timed access to containers and blobs, providing a segway into supporting WASB for secure hadoop cluster.

      More details about the issue and the proposal are provided in the design proposal document.

      1. HADOOP-13863.001.patch
        47 kB
        Dushyanth
      2. HADOOP-13863.002.patch
        50 kB
        Dushyanth
      3. HADOOP-13863.003.patch
        70 kB
        Dushyanth
      4. HADOOP-13863.004.patch
        72 kB
        Dushyanth
      5. HADOOP-13863.005.patch
        72 kB
        Dushyanth
      6. HADOOP-13863.006.patch
        70 kB
        Mingliang Liu
      7. HADOOP-13863.007.patch
        73 kB
        Dushyanth
      8. Proposal-Document.pdf
        459 kB
        Dushyanth

        Issue Links

          Activity

          Hide
          dchickabasapa Dushyanth added a comment -

          Attaching the first draft of the design proposal to support SAS Key mode in WASB.

          Show
          dchickabasapa Dushyanth added a comment - Attaching the first draft of the design proposal to support SAS Key mode in WASB.
          Hide
          stevel@apache.org Steve Loughran added a comment -
          1. any link for more info on SAS keys?
          2. this new storage interface: is it going to be a think layer of extra auth, or a copy & paste of the existing code? Because C&P is something to absolutely avoid...if it can't be done without some changes for better extensibility in the NativeAzureStorage classes, I'd prefer adding those extension points.
          3. testing. The local one works with today's keys, so can be run on anyone's desktop? If so, that's essential. It does mean that the other codepath, the httpclient stuff, is going to have to be tested in the release process by volunteers with access to that feature

          S3 has a similar problem there and its IAM credential auth on EC2 machines. We trust AWS to test their HTTP client, but even there, in HADOOP-13727, we had to do some patching to deal with how IAM/AWS throttled clients —the kind of problem which we didn't see during our integration tests, even running in-EC2. It'd be good to make sure that the Azure client avoids the same problem

          Show
          stevel@apache.org Steve Loughran added a comment - any link for more info on SAS keys? this new storage interface: is it going to be a think layer of extra auth, or a copy & paste of the existing code? Because C&P is something to absolutely avoid...if it can't be done without some changes for better extensibility in the NativeAzureStorage classes, I'd prefer adding those extension points. testing. The local one works with today's keys, so can be run on anyone's desktop? If so, that's essential. It does mean that the other codepath, the httpclient stuff, is going to have to be tested in the release process by volunteers with access to that feature S3 has a similar problem there and its IAM credential auth on EC2 machines. We trust AWS to test their HTTP client, but even there, in HADOOP-13727 , we had to do some patching to deal with how IAM/AWS throttled clients —the kind of problem which we didn't see during our integration tests, even running in-EC2. It'd be good to make sure that the Azure client avoids the same problem
          Hide
          dchickabasapa Dushyanth added a comment -

          Thanks Steve Loughran for the review of the draft. To answer your questions:

          1) https://docs.microsoft.com/en-us/azure/storage/storage-dotnet-shared-access-signature-part-1
          2) The layer is going to be a layer with auth support, as well as logic to support renewel of the SAS keys. Hence it made more sense to have a separate implementation rather than polluting the current StorageInterfaceImpl implementation. We expect the feature to be added iteratively, the first iteration is expected to be introduction of the SAS key mode with just copy-paste and get an initial read on the approach, and then the plan is to add the renewel process.
          3) Yes, the local SAS key mode will allow to desk the SAS key mode of execution on anyone's desktop. The HttpClient mode, will have to be tested is specific setups.

          Show
          dchickabasapa Dushyanth added a comment - Thanks Steve Loughran for the review of the draft. To answer your questions: 1) https://docs.microsoft.com/en-us/azure/storage/storage-dotnet-shared-access-signature-part-1 2) The layer is going to be a layer with auth support, as well as logic to support renewel of the SAS keys. Hence it made more sense to have a separate implementation rather than polluting the current StorageInterfaceImpl implementation. We expect the feature to be added iteratively, the first iteration is expected to be introduction of the SAS key mode with just copy-paste and get an initial read on the approach, and then the plan is to add the renewel process. 3) Yes, the local SAS key mode will allow to desk the SAS key mode of execution on anyone's desktop. The HttpClient mode, will have to be tested is specific setups.
          Hide
          dchickabasapa Dushyanth added a comment -

          First iteration of the support to add SAS key mode for WASB.

          This patch has following limitations:

          1) Supports only LocalSASKey mode.
          2) No support for renewal of SAS keys

          Testing: This patch has been tested for both Non-SAS key mode and SAS-key mode for live azure storage tests.

          Show
          dchickabasapa Dushyanth added a comment - First iteration of the support to add SAS key mode for WASB. This patch has following limitations: 1) Supports only LocalSASKey mode. 2) No support for renewal of SAS keys Testing: This patch has been tested for both Non-SAS key mode and SAS-key mode for live azure storage tests.
          Hide
          stevel@apache.org Steve Loughran added a comment -
          Show
          stevel@apache.org Steve Loughran added a comment - + Larry McCay
          Hide
          liuml07 Mingliang Liu added a comment -

          This patch does not apply.

          978	      if (!(this.storageInteractionLayer instanceof MockStorageInterface) && useSasKeyMode) {
          979	        connectToAzureStorageInSASKeyMode(accountName, containerName, sessionUri);
          980	        return;
          981	      }
          

          Probing the implementation details is not ideal, not to mention it's a mocked class in tests.

          Show
          liuml07 Mingliang Liu added a comment - This patch does not apply. 978 if (!( this .storageInteractionLayer instanceof MockStorageInterface) && useSasKeyMode) { 979 connectToAzureStorageInSASKeyMode(accountName, containerName, sessionUri); 980 return ; 981 } Probing the implementation details is not ideal, not to mention it's a mocked class in tests.
          Hide
          liuml07 Mingliang Liu added a comment -
          java version "1.8.0_65"
          Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
          Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)
          [ERROR] COMPILATION ERROR :
          [ERROR] /Users/mliu/Workspace/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java:[978,53] cannot find symbol
            symbol:   class MockStorageInterface
            location: class org.apache.hadoop.fs.azure.AzureNativeFileSystemStore
          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hadoop-azure: Compilation failure
          [ERROR] /Users/mliu/Workspace/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java:[978,53] cannot find symbol
          [ERROR] symbol:   class MockStorageInterface
          [ERROR] location: class org.apache.hadoop.fs.azure.AzureNativeFileSystemStore
          [ERROR] -> [Help 1]
          [ERROR]
          [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
          [ERROR] Re-run Maven using the -X switch to enable full debug logging.
          [ERROR]
          [ERROR] For more information about the errors and possible solutions, please read the following articles:
          [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
          [ERROR]
          [ERROR] After correcting the problems, you can resume the build with the command
          [ERROR]   mvn <goals> -rf :hadoop-azure
          
          Show
          liuml07 Mingliang Liu added a comment - java version "1.8.0_65" Java(TM) SE Runtime Environment (build 1.8.0_65-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode) [ERROR] COMPILATION ERROR : [ERROR] /Users/mliu/Workspace/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java:[978,53] cannot find symbol symbol: class MockStorageInterface location: class org.apache.hadoop.fs.azure.AzureNativeFileSystemStore [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile ( default -compile) on project hadoop-azure: Compilation failure [ERROR] /Users/mliu/Workspace/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java:[978,53] cannot find symbol [ERROR] symbol: class MockStorageInterface [ERROR] location: class org.apache.hadoop.fs.azure.AzureNativeFileSystemStore [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch . [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http: //cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hadoop-azure
          Hide
          dchickabasapa Dushyanth added a comment -

          I apologize, I was not seeing this issue after the initial build was successful. Was able to repro the problem with a clean build. I have fixed the issue, and removed the implementation checks. The check was put in place to not affect Mock tests, but have disabled those tests for SAS mode in the new patch.

          Show
          dchickabasapa Dushyanth added a comment - I apologize, I was not seeing this issue after the initial build was successful. Was able to repro the problem with a clean build. I have fixed the issue, and removed the implementation checks. The check was put in place to not affect Mock tests, but have disabled those tests for SAS mode in the new patch.
          Hide
          liuml07 Mingliang Liu added a comment -

          Good, thanks Dushyanth for updating the patch. I'll review this code this week (or before the holiday) if no reviews from others.

          Show
          liuml07 Mingliang Liu added a comment - Good, thanks Dushyanth for updating the patch. I'll review this code this week (or before the holiday) if no reviews from others.
          Hide
          stevel@apache.org Steve Loughran added a comment - - edited

          Any reason why SASKeyGenerationException extends Exception and SASKeyModeNonSupportedException extends RuntimeException?

          Generally we like everything to extend IOException, as that tends to get relayed around the most

          Show
          stevel@apache.org Steve Loughran added a comment - - edited Any reason why SASKeyGenerationException extends Exception and SASKeyModeNonSupportedException extends RuntimeException ? Generally we like everything to extend IOException, as that tends to get relayed around the most
          Hide
          stevel@apache.org Steve Loughran added a comment -
          • Needs some docs, ideally a section in hadoop-tools/hadoop-azure/src/site/markdown/index.md
          • default values MAY go into hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          • I'd like to have all exceptions subclass AzureException unless there's a particularly compelling reason why not

          I ran all the tests for this locally, they did complete but it did take a long time. I should check to see if there's any different when run without this patch.

          Results :
          
          Tests run: 646, Failures: 0, Errors: 0, Skipped: 8
          
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD SUCCESS
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 29:23 min
          [INFO] Finished at: 2016-12-14T17:58:13+00:00
          
          Show
          stevel@apache.org Steve Loughran added a comment - Needs some docs, ideally a section in hadoop-tools/hadoop-azure/src/site/markdown/index.md default values MAY go into hadoop-common-project/hadoop-common/src/main/resources/core-default.xml I'd like to have all exceptions subclass AzureException unless there's a particularly compelling reason why not I ran all the tests for this locally, they did complete but it did take a long time. I should check to see if there's any different when run without this patch. Results : Tests run: 646, Failures: 0, Errors: 0, Skipped: 8 [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 29:23 min [INFO] Finished at: 2016-12-14T17:58:13+00:00
          Hide
          dchickabasapa Dushyanth added a comment -

          Modified the proposal document with the Remote service requirements

          Show
          dchickabasapa Dushyanth added a comment - Modified the proposal document with the Remote service requirements
          Hide
          dchickabasapa Dushyanth added a comment - - edited

          Thanks Steve Loughran for the review.

          1) I have added documentation to index.md file for the change.
          2) These are azure specific configs name as "fs.zure.*" so not sure if it is a good idean to add them to hadoop-common.
          3) I have modified the newly added Exceptions to extend AzureException

          The new patch also has support for the RemoteSASKeyGenerationImpl.

          Show
          dchickabasapa Dushyanth added a comment - - edited Thanks Steve Loughran for the review. 1) I have added documentation to index.md file for the change. 2) These are azure specific configs name as "fs.zure.*" so not sure if it is a good idean to add them to hadoop-common. 3) I have modified the newly added Exceptions to extend AzureException The new patch also has support for the RemoteSASKeyGenerationImpl.
          Hide
          liuml07 Mingliang Liu added a comment -

          Submitting the patch (again)...

          Show
          liuml07 Mingliang Liu added a comment - Submitting the patch (again)...
          Hide
          liuml07 Mingliang Liu added a comment - - edited

          These are azure specific configs name as "fs.zure.*" so not sure if it is a good idean to add them to hadoop-common.

          Configuration will load the core-default.xml file for default values in hadoop common and tools, e.g. fs.s3.. I also think it's a good idea to have fs.azure. configs in core-defaults.xml if their default values are not loaded elsewhere.

          Show
          liuml07 Mingliang Liu added a comment - - edited These are azure specific configs name as "fs.zure.*" so not sure if it is a good idean to add them to hadoop-common. Configuration will load the core-default.xml file for default values in hadoop common and tools, e.g. fs.s3. . I also think it's a good idea to have fs.azure. configs in core-defaults.xml if their default values are not loaded elsewhere.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 7m 27s trunk passed
          +1 compile 0m 21s trunk passed
          +1 checkstyle 0m 15s trunk passed
          +1 mvnsite 0m 22s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 0m 30s trunk passed
          +1 javadoc 0m 13s trunk passed
          +1 mvninstall 0m 19s the patch passed
          +1 compile 0m 19s the patch passed
          +1 javac 0m 19s the patch passed
          -0 checkstyle 0m 12s hadoop-tools/hadoop-azure: The patch generated 32 new + 35 unchanged - 0 fixed = 67 total (was 35)
          -1 mvnsite 0m 20s hadoop-azure in the patch failed.
          +1 mvneclipse 0m 12s the patch passed
          -1 whitespace 0m 0s The patch 21 line(s) with tabs.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          -1 findbugs 0m 38s hadoop-tools/hadoop-azure generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
          -1 javadoc 0m 12s hadoop-azure in the patch failed.
          +1 unit 1m 31s hadoop-azure in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          15m 1s



          Reason Tests
          FindBugs module:hadoop-tools/hadoop-azure
            Found reliance on default encoding in org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet):in org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet): new java.io.InputStreamReader(InputStream) At WasbRemoteCallHelper.java:[line 93]
            org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet) may fail to close stream At WasbRemoteCallHelper.java:stream At WasbRemoteCallHelper.java:[line 92]



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13863
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843334/HADOOP-13863.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 130d5cc11e6d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 64a2d5b
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-azure.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/whitespace-tabs.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/new-findbugs-hadoop-tools_hadoop-azure.html
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/testReport/
          modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 7m 27s trunk passed +1 compile 0m 21s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 19s the patch passed +1 compile 0m 19s the patch passed +1 javac 0m 19s the patch passed -0 checkstyle 0m 12s hadoop-tools/hadoop-azure: The patch generated 32 new + 35 unchanged - 0 fixed = 67 total (was 35) -1 mvnsite 0m 20s hadoop-azure in the patch failed. +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch 21 line(s) with tabs. +1 xml 0m 2s The patch has no ill-formed XML file. -1 findbugs 0m 38s hadoop-tools/hadoop-azure generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) -1 javadoc 0m 12s hadoop-azure in the patch failed. +1 unit 1m 31s hadoop-azure in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 15m 1s Reason Tests FindBugs module:hadoop-tools/hadoop-azure   Found reliance on default encoding in org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet):in org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet): new java.io.InputStreamReader(InputStream) At WasbRemoteCallHelper.java: [line 93]   org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet) may fail to close stream At WasbRemoteCallHelper.java:stream At WasbRemoteCallHelper.java: [line 92] Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13863 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843334/HADOOP-13863.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 130d5cc11e6d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 64a2d5b Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/patch-mvnsite-hadoop-tools_hadoop-azure.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/whitespace-tabs.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/new-findbugs-hadoop-tools_hadoop-azure.html javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/testReport/ modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11283/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          + that file is used to auto-generate the documentation of properties in the hadoop site.

          Show
          stevel@apache.org Steve Loughran added a comment - + that file is used to auto-generate the documentation of properties in the hadoop site.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Coming along nicely.

          • "fs.azure.sas.expiry.period" should be read using getTimeDuration; caller is free to choose a time. Just use a default like "1d" or "24h" for the per-day value.
          • docs haven't marked example json and URLs as code. Either indent by 4 spaces or surround code sections with ``` triples.
          • As discussed, feel free to put the defaults into core-defaults.xml. We do that for other object stores too.

          Minor style issues

          • Checkstyle is complaining a lot; ideally it should be down to ~0 complaints.
          • RemoteSASKeyGeneratorImpl could just import static WasbRemoteCallHelper.* for the constants
          • lines are all too wide. It's been discussed recently, and preferred length is just 80 chars, with the odd overrun allowed. Why? Makes side-by-side patch comparison easier?
          • SASKeyGeneratorInterface appears to be indenting with tabs, or is 8+ spaces wide.
          • Could the javadocs for the constants all use{{@value}} to declare the defined value. IDEs which popup javadocs like this.
          Show
          stevel@apache.org Steve Loughran added a comment - Coming along nicely. "fs.azure.sas.expiry.period" should be read using getTimeDuration ; caller is free to choose a time. Just use a default like "1d" or "24h" for the per-day value. docs haven't marked example json and URLs as code. Either indent by 4 spaces or surround code sections with ``` triples. As discussed, feel free to put the defaults into core-defaults.xml. We do that for other object stores too. Minor style issues Checkstyle is complaining a lot; ideally it should be down to ~0 complaints. RemoteSASKeyGeneratorImpl could just import static WasbRemoteCallHelper.* for the constants lines are all too wide. It's been discussed recently, and preferred length is just 80 chars, with the odd overrun allowed. Why? Makes side-by-side patch comparison easier? SASKeyGeneratorInterface appears to be indenting with tabs, or is 8+ spaces wide. Could the javadocs for the constants all use{{@value}} to declare the defined value. IDEs which popup javadocs like this.
          Hide
          dchickabasapa Dushyanth added a comment -

          Thanks Steve Loughran for the review. I have addressed your comments in Patch v4. Also the lines have been reduced 80 chars per line.

          Show
          dchickabasapa Dushyanth added a comment - Thanks Steve Loughran for the review. I have addressed your comments in Patch v4. Also the lines have been reduced 80 chars per line.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 1m 47s Maven dependency ordering for branch
          +1 mvninstall 12m 36s trunk passed
          +1 compile 9m 34s trunk passed
          +1 checkstyle 1m 34s trunk passed
          +1 mvnsite 1m 26s trunk passed
          +1 mvneclipse 0m 37s trunk passed
          +1 findbugs 1m 58s trunk passed
          +1 javadoc 1m 10s trunk passed
          0 mvndep 0m 18s Maven dependency ordering for patch
          +1 mvninstall 0m 57s the patch passed
          +1 compile 9m 18s the patch passed
          +1 javac 9m 18s the patch passed
          -0 checkstyle 1m 35s root: The patch generated 8 new + 35 unchanged - 0 fixed = 43 total (was 35)
          +1 mvnsite 1m 28s the patch passed
          +1 mvneclipse 0m 43s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 xml 0m 2s The patch has 1 ill-formed XML file(s).
          -1 findbugs 0m 45s hadoop-tools/hadoop-azure generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
          -1 javadoc 0m 24s hadoop-azure in the patch failed.
          -1 unit 17m 14s hadoop-common in the patch failed.
          -1 unit 1m 3s hadoop-azure in the patch failed.
          -1 asflicense 0m 37s The patch generated 3 ASF License warnings.
          92m 12s



          Reason Tests
          FindBugs module:hadoop-tools/hadoop-azure
            Found reliance on default encoding in org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet):in org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet): new java.io.InputStreamReader(InputStream) At WasbRemoteCallHelper.java:[line 99]
            org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet) may fail to close stream At WasbRemoteCallHelper.java:stream At WasbRemoteCallHelper.java:[line 98]
          Failed junit tests hadoop.io.file.tfile.TestVLong
            hadoop.util.TestApplicationClassLoader
            hadoop.ipc.TestDecayRpcScheduler
            hadoop.io.file.tfile.TestTFileByteArrays
            hadoop.http.TestGlobalFilter
            hadoop.fs.TestAvroFSInput
            hadoop.http.TestPathFilter
            hadoop.fs.viewfs.TestViewFileSystemDelegation
            hadoop.fs.TestFileSystemInitialization
            hadoop.fs.TestDefaultUri
            hadoop.io.erasurecode.TestCodecRawCoderMapping
            hadoop.io.retry.TestDefaultRetryPolicy
            hadoop.io.compress.TestCompressionStreamReuse
            hadoop.fs.viewfs.TestFcPermissionsLocalFs
            hadoop.io.file.tfile.TestTFileLzoCodecsStreams
            hadoop.ipc.TestRPCCompatibility
            hadoop.fs.viewfs.TestViewFsURIs
            hadoop.fs.shell.TestCopy
            hadoop.fs.shell.TestPathData
            hadoop.ha.TestHealthMonitor
            hadoop.ha.TestFailoverController
            hadoop.io.compress.TestCodecPool
            hadoop.fs.TestGetFileBlockLocations
            hadoop.fs.TestGetSpaceUsed
            hadoop.fs.contract.rawlocal.TestRawlocalContractSetTimes
            hadoop.security.TestLdapGroupsMappingWithPosixGroup
            hadoop.security.TestGroupsCaching
            hadoop.fs.permission.TestFsPermission
            hadoop.http.TestHttpServer
            hadoop.crypto.TestCryptoStreamsWithJceAesCtrCryptoCodec
            hadoop.security.TestAuthenticationFilter
            hadoop.security.token.delegation.web.TestDelegationTokenAuthenticationHandlerWithMocks
            hadoop.fs.TestChecksumFileSystem
            hadoop.crypto.key.TestKeyProviderFactory
            hadoop.conf.TestDeprecatedKeys
            hadoop.security.authorize.TestProxyUsers
            hadoop.security.token.delegation.web.TestDelegationTokenManager
            hadoop.crypto.random.TestOsSecureRandom
            hadoop.conf.TestReconfiguration
            hadoop.tracing.TestTraceUtils
            hadoop.net.TestStaticMapping
            hadoop.fs.TestHarFileSystemBasics
            hadoop.io.TestSetFile
            hadoop.fs.contract.rawlocal.TestRawlocalContractDelete
            hadoop.fs.TestDU
            hadoop.fs.contract.localfs.TestLocalFSContractSeek
            hadoop.fs.TestTruncatedInputBug
            hadoop.fs.TestFSMainOperationsLocalFileSystem
            hadoop.fs.contract.localfs.TestLocalFSContractSetTimes
            hadoop.fs.TestPath
            hadoop.io.erasurecode.coder.TestRSErasureCoder
            hadoop.util.TestGenericOptionsParser
            hadoop.metrics2.sink.TestRollingFileSystemSink
            hadoop.crypto.key.TestKeyShell
            hadoop.io.file.tfile.TestTFile
            hadoop.fs.sftp.TestSFTPFileSystem
            hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec
            hadoop.fs.contract.rawlocal.TestRawlocalContractAppend
            hadoop.io.TestWritable
            hadoop.ipc.TestProtoBufRPCCompatibility
            hadoop.fs.viewfs.TestViewfsFileStatus
            hadoop.fs.TestTrash
            hadoop.security.TestHttpCrossOriginFilterInitializer
            hadoop.ha.TestSshFenceByTcpPort
            hadoop.security.authorize.TestProxyServers
            hadoop.fs.TestLocalDirAllocator
            hadoop.security.alias.TestCredShell
            hadoop.ipc.TestIdentityProviders
            hadoop.ipc.TestIPCServerResponder
            hadoop.ha.TestShellCommandFencer
            hadoop.crypto.TestCryptoOutputStreamClosing
            hadoop.util.TestNativeLibraryChecker
            hadoop.fs.TestFileContextResolveAfs
            hadoop.io.TestMapFile
            hadoop.fs.shell.TestCopyPreserveFlag
            hadoop.cli.TestCLI
            hadoop.crypto.key.TestKeyProviderCryptoExtension
            hadoop.http.TestSSLHttpServer
            hadoop.fs.TestFsShellCopy
            hadoop.io.file.tfile.TestTFileSplit
            hadoop.ipc.TestAsyncIPC
            hadoop.conf.TestConfigRedactor
            hadoop.fs.TestFsShell
            hadoop.io.file.tfile.TestTFileLzoCodecsByteArrays
            hadoop.io.TestBloomMapFile
            hadoop.fs.shell.find.TestName
            hadoop.io.compress.zlib.TestZlibCompressorDecompressor
            hadoop.io.TestSecureIOUtils
            hadoop.fs.contract.ftp.TestFTPContractCreate
            hadoop.fs.contract.localfs.TestLocalFSContractOpen
            hadoop.io.file.tfile.TestTFileComparators
            hadoop.fs.contract.localfs.TestLocalFSContractMkdir
            hadoop.crypto.TestCryptoStreamsForLocalFS
            hadoop.fs.shell.TestTextCommand
            hadoop.fs.contract.ftp.TestFTPContractOpen
            hadoop.fs.shell.TestCount
            hadoop.fs.contract.ftp.TestFTPContractMkdir
            hadoop.ipc.TestCallQueueManager
            hadoop.io.compress.TestCompressorDecompressor
            hadoop.io.file.tfile.TestTFileStreams
            hadoop.fs.viewfs.TestViewFsTrash
            hadoop.ipc.TestRPCServerShutdown
            hadoop.fs.viewfs.TestFcCreateMkdirLocalFs
            hadoop.fs.contract.rawlocal.TestRawlocalContractMkdir
            hadoop.http.TestServletFilter
            hadoop.metrics2.sink.TestRollingFileSystemSinkWithLocal
            hadoop.fs.TestDelegateToFileSystem
            hadoop.fs.contract.localfs.TestLocalFSContractRename
            hadoop.fs.contract.localfs.TestLocalFSContractDelete
            hadoop.fs.viewfs.TestFcMainOperationsLocalFs
            hadoop.ipc.TestRPCWaitForProxy
            hadoop.ha.TestZKFailoverController
            hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem
            hadoop.security.token.TestDtUtilShell
            hadoop.conf.TestConfigurationDeprecation
            hadoop.fs.TestRawLocalFileSystemContract
            hadoop.security.authorize.TestAccessControlList
            hadoop.fs.TestFilterFileSystem
            hadoop.security.ssl.TestSSLFactory
            hadoop.fs.shell.find.TestPrint0
            hadoop.io.erasurecode.coder.TestHHXORErasureCoder
            hadoop.io.TestSequenceFile
            hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays
            hadoop.service.TestServiceLifecycle
            hadoop.fs.viewfs.TestViewFileSystemDelegationTokenSupport
            hadoop.security.TestUserFromEnv
            hadoop.io.file.tfile.TestTFileUnsortedByteArrays
            hadoop.fs.shell.TestXAttrCommands
            hadoop.ipc.TestMiniRPCBenchmark
            hadoop.http.TestHttpServerLogs
            hadoop.fs.shell.find.TestIname
            hadoop.io.file.tfile.TestTFileSeek
            hadoop.util.TestGenericsUtil
            hadoop.io.serializer.TestWritableSerialization
            hadoop.io.TestSequenceFileSync
            hadoop.crypto.key.TestKeyProvider
            hadoop.ha.TestNodeFencer
            hadoop.fs.TestLocalFSFileContextCreateMkdir
            hadoop.crypto.key.TestKeyProviderDelegationTokenExtension
            hadoop.ipc.TestRPC
            hadoop.fs.contract.ftp.TestFTPContractDelete
            hadoop.fs.TestFcLocalFsUtil
            hadoop.fs.TestStat
            hadoop.io.TestGenericWritable
            hadoop.http.TestAuthenticationSessionCookie
            hadoop.conf.TestConfServlet
            hadoop.fs.contract.rawlocal.TestRawlocalContractSeek
            hadoop.util.TestNativeCrc32
            hadoop.fs.TestDelegateToFsCheckPath
            hadoop.ipc.TestProtoBufRpc
            hadoop.security.TestWhitelistBasedResolver
            hadoop.io.retry.TestConnectionRetryPolicy
            hadoop.net.TestNetUtils
            hadoop.fs.shell.find.TestFind
            hadoop.util.TestDiskChecker
            hadoop.ipc.TestSocketFactory
            hadoop.fs.TestLocalFSFileContextMainOperations
            hadoop.security.TestLdapGroupsMappingWithOneQuery
            hadoop.security.TestKDiag
            hadoop.io.TestSequenceFileAppend
            hadoop.util.TestBasicDiskValidator
            hadoop.fs.contract.localfs.TestLocalFSContractGetFileStatus
            hadoop.security.alias.TestCredentialProviderFactory
            hadoop.http.TestHttpServerWebapps
            hadoop.ipc.TestIPC
            hadoop.fs.contract.localfs.TestLocalFSContractAppend
            hadoop.ha.TestHAAdmin
            hadoop.crypto.key.kms.TestLoadBalancingKMSClientProvider
            hadoop.ipc.TestMultipleProtocolServer
            hadoop.fs.viewfs.TestChRootedFs
            hadoop.ipc.TestWeightedRoundRobinMultiplexer
            hadoop.security.token.delegation.TestZKDelegationTokenSecretManager
            hadoop.fs.TestFileSystemCanonicalization
            hadoop.fs.TestFsShellReturnCode
            hadoop.fs.TestLocalFsFCStatistics
            hadoop.io.file.tfile.TestTFileNoneCodecsStreams
            hadoop.fs.TestHarFileSystem
            hadoop.fs.TestSymlinkLocalFSFileContext
            hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem
            hadoop.fs.contract.localfs.TestLocalFSContractLoaded
            hadoop.security.TestUGIWithMiniKdc
            hadoop.fs.shell.find.TestPrint
            hadoop.net.TestScriptBasedMappingWithDependency
            hadoop.util.TestFindClass
            hadoop.fs.contract.rawlocal.TestRawlocalContractOpen
            hadoop.util.hash.TestHash
            hadoop.io.file.tfile.TestTFileComparator2
            hadoop.io.compress.bzip2.TestBzip2CompressorDecompressor
            hadoop.ipc.TestSaslRPC
            hadoop.util.TestRunJar
            hadoop.net.TestSwitchMapping
            hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs
            hadoop.fs.TestLocalFileSystemPermission
            hadoop.security.TestJNIGroupsMapping
            hadoop.http.lib.TestStaticUserWebFilter
            hadoop.security.TestShellBasedIdMapping
            hadoop.ipc.TestRPCCallBenchmark
            hadoop.io.serializer.avro.TestAvroSerialization
            hadoop.fs.shell.TestMove
            hadoop.conf.TestGetInstances
            hadoop.fs.shell.TestLs
            hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
            hadoop.ha.TestHealthMonitorWithDedicatedHealthAddress
            hadoop.security.TestKDiagNoKDC
            hadoop.conf.TestConfigurationSubclass
            hadoop.fs.viewfs.TestViewFsConfig
            hadoop.security.TestCompositeGroupMapping
            hadoop.fs.contract.localfs.TestLocalFSContractCreate
            hadoop.fs.TestListFiles
            hadoop.fs.contract.rawlocal.TestRawlocalContractRename
            hadoop.security.TestUserGroupInformation
            hadoop.security.authorize.TestServiceAuthorization
            hadoop.net.TestTableMapping
            hadoop.crypto.TestCryptoCodec
            hadoop.fs.TestFsShellTouch
            hadoop.fs.TestFileUtil
            hadoop.security.TestDoAsEffectiveUser
            hadoop.fs.TestFcLocalFsPermission
            hadoop.util.TestNodeHealthScriptRunner
            hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays
            hadoop.crypto.TestCryptoStreamsNormal
            hadoop.ha.TestZKFailoverControllerStress
            hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem
            hadoop.fs.contract.rawlocal.TestRawlocalContractCreate
            hadoop.io.file.tfile.TestCompression
            hadoop.fs.viewfs.TestChRootedFileSystem
            hadoop.fs.TestFileContext
            hadoop.ipc.TestReuseRpcConnections
            hadoop.http.TestHttpCookieFlag
            hadoop.security.TestGroupFallback
            hadoop.security.token.delegation.web.TestWebDelegationToken
            hadoop.net.TestScriptBasedMapping
            hadoop.fs.viewfs.TestViewFsLocalFs
            hadoop.security.TestProxyUserFromEnv
            hadoop.crypto.TestCryptoStreams
            hadoop.fs.TestLocalFileSystem
            hadoop.io.TestSequenceFileSerialization
            hadoop.fs.contract.rawlocal.TestRawlocalContractGetFileStatus
            hadoop.io.TestArrayFile
            hadoop.conf.TestConfiguration
            hadoop.http.TestHttpServerLifecycle
            hadoop.fs.shell.TestAclCommands
            hadoop.ipc.TestProtoBufRpcServerHandoff
            hadoop.security.TestSecurityUtil
            hadoop.jmx.TestJMXJsonServlet
            hadoop.io.TestDefaultStringifier
            hadoop.ipc.TestFairCallQueue
            hadoop.fs.TestFsShellList
            hadoop.io.erasurecode.coder.TestXORCoder
            hadoop.fs.TestFileContextDeleteOnExit
            hadoop.conf.TestCommonConfigurationFields
            hadoop.security.TestUGILoginFromKeytab
            hadoop.io.file.tfile.TestTFileSeqFileComparison
            hadoop.io.compress.TestCodec
            hadoop.io.serializer.TestSerializationFactory
            hadoop.log.TestLogLevel
            hadoop.security.TestLdapGroupsMapping
            hadoop.metrics2.source.TestJvmMetrics
            hadoop.fs.contract.ftp.TestFTPContractRename
            hadoop.fs.TestSymlinkLocalFSFileSystem
            hadoop.io.compress.TestCodecFactory
            hadoop.ipc.TestServer
            hadoop.ipc.TestRpcServerHandoff
            hadoop.fs.azure.TestFileSystemOperationsExceptionHandlingMultiThreaded
            hadoop.fs.azure.contract.TestAzureNativeContractMkdir
            hadoop.fs.azure.TestNativeAzureFileSystemConcurrency
            hadoop.fs.azure.TestAzureConcurrentOutOfBandIo
            hadoop.fs.azure.TestFileSystemOperationExceptionHandling
            hadoop.fs.azure.metrics.TestBandwidthGaugeUpdater
            hadoop.fs.azure.TestWasbFsck
            hadoop.fs.azure.TestNativeAzureFileSystemClientLogging
            hadoop.fs.azure.TestOutOfBandAzureBlobOperations
            hadoop.fs.azure.TestReadAndSeekPageBlobAfterWrite
            hadoop.fs.azure.TestFileSystemOperationExceptionMessage
            hadoop.fs.azure.contract.TestAzureNativeContractCreate
            hadoop.fs.azure.TestNativeAzureFileSystemContractMocked
            hadoop.fs.azure.TestNativeAzureFileSystemContractPageBlobLive
            hadoop.fs.azure.TestOutOfBandAzureBlobOperationsLive
            hadoop.fs.azure.TestContainerChecks
            hadoop.fs.azure.TestNativeAzureFileSystemLive
            hadoop.fs.azure.contract.TestAzureNativeContractAppend
            hadoop.fs.azure.contract.TestAzureNativeContractDelete
            hadoop.fs.azure.TestNativeAzureFileSystemContractLive
            hadoop.fs.azure.TestBlobMetadata
            hadoop.fs.azure.metrics.TestAzureFileSystemInstrumentation
            hadoop.fs.azure.TestNativeAzureFileSystemConcurrencyLive
            hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked
            hadoop.fs.azure.metrics.TestNativeAzureFileSystemMetricsSystem
            hadoop.fs.azure.contract.TestAzureNativeContractOpen
            hadoop.fs.azure.TestNativeAzureFileSystemContractEmulator
            hadoop.fs.azure.TestNativeAzureFileSystemAtomicRenameDirList
            hadoop.fs.azure.contract.TestAzureNativeContractSeek
            hadoop.fs.azure.TestNativeAzureFSPageBlobLive
            hadoop.fs.azure.TestBlobDataValidation
            hadoop.fs.azure.TestBlobTypeSpeedDifference
            hadoop.fs.azure.TestWasbUriAndConfiguration
            hadoop.fs.azure.contract.TestAzureNativeContractRename
            hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck
            hadoop.fs.azure.TestNativeAzureFileSystemAppend
            hadoop.fs.azure.TestNativeAzureFileSystemMocked
            hadoop.fs.azure.TestNativeAzureFileSystemBlockLocations
            hadoop.fs.azure.contract.TestAzureNativeContractDistCp
            hadoop.fs.azure.contract.TestAzureNativeContractGetFileStatus
            hadoop.fs.azure.TestAzureFileSystemErrorConditions
            hadoop.fs.azure.TestFileSystemOperationsWithThreads
          Timed out junit tests org.apache.hadoop.fs.TestFileSystemCaching
            org.apache.hadoop.io.nativeio.TestNativeIO



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13863
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843733/HADOOP-13863.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 72d9da9cb4dc 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fcbe152
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/diff-checkstyle-root.txt
          xml https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/xml.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/new-findbugs-hadoop-tools_hadoop-azure.html
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-azure.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 1m 47s Maven dependency ordering for branch +1 mvninstall 12m 36s trunk passed +1 compile 9m 34s trunk passed +1 checkstyle 1m 34s trunk passed +1 mvnsite 1m 26s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 1m 58s trunk passed +1 javadoc 1m 10s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 0m 57s the patch passed +1 compile 9m 18s the patch passed +1 javac 9m 18s the patch passed -0 checkstyle 1m 35s root: The patch generated 8 new + 35 unchanged - 0 fixed = 43 total (was 35) +1 mvnsite 1m 28s the patch passed +1 mvneclipse 0m 43s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 xml 0m 2s The patch has 1 ill-formed XML file(s). -1 findbugs 0m 45s hadoop-tools/hadoop-azure generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) -1 javadoc 0m 24s hadoop-azure in the patch failed. -1 unit 17m 14s hadoop-common in the patch failed. -1 unit 1m 3s hadoop-azure in the patch failed. -1 asflicense 0m 37s The patch generated 3 ASF License warnings. 92m 12s Reason Tests FindBugs module:hadoop-tools/hadoop-azure   Found reliance on default encoding in org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet):in org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet): new java.io.InputStreamReader(InputStream) At WasbRemoteCallHelper.java: [line 99]   org.apache.hadoop.fs.azure.WasbRemoteCallHelper.makeRemoteGetRequest(HttpGet) may fail to close stream At WasbRemoteCallHelper.java:stream At WasbRemoteCallHelper.java: [line 98] Failed junit tests hadoop.io.file.tfile.TestVLong   hadoop.util.TestApplicationClassLoader   hadoop.ipc.TestDecayRpcScheduler   hadoop.io.file.tfile.TestTFileByteArrays   hadoop.http.TestGlobalFilter   hadoop.fs.TestAvroFSInput   hadoop.http.TestPathFilter   hadoop.fs.viewfs.TestViewFileSystemDelegation   hadoop.fs.TestFileSystemInitialization   hadoop.fs.TestDefaultUri   hadoop.io.erasurecode.TestCodecRawCoderMapping   hadoop.io.retry.TestDefaultRetryPolicy   hadoop.io.compress.TestCompressionStreamReuse   hadoop.fs.viewfs.TestFcPermissionsLocalFs   hadoop.io.file.tfile.TestTFileLzoCodecsStreams   hadoop.ipc.TestRPCCompatibility   hadoop.fs.viewfs.TestViewFsURIs   hadoop.fs.shell.TestCopy   hadoop.fs.shell.TestPathData   hadoop.ha.TestHealthMonitor   hadoop.ha.TestFailoverController   hadoop.io.compress.TestCodecPool   hadoop.fs.TestGetFileBlockLocations   hadoop.fs.TestGetSpaceUsed   hadoop.fs.contract.rawlocal.TestRawlocalContractSetTimes   hadoop.security.TestLdapGroupsMappingWithPosixGroup   hadoop.security.TestGroupsCaching   hadoop.fs.permission.TestFsPermission   hadoop.http.TestHttpServer   hadoop.crypto.TestCryptoStreamsWithJceAesCtrCryptoCodec   hadoop.security.TestAuthenticationFilter   hadoop.security.token.delegation.web.TestDelegationTokenAuthenticationHandlerWithMocks   hadoop.fs.TestChecksumFileSystem   hadoop.crypto.key.TestKeyProviderFactory   hadoop.conf.TestDeprecatedKeys   hadoop.security.authorize.TestProxyUsers   hadoop.security.token.delegation.web.TestDelegationTokenManager   hadoop.crypto.random.TestOsSecureRandom   hadoop.conf.TestReconfiguration   hadoop.tracing.TestTraceUtils   hadoop.net.TestStaticMapping   hadoop.fs.TestHarFileSystemBasics   hadoop.io.TestSetFile   hadoop.fs.contract.rawlocal.TestRawlocalContractDelete   hadoop.fs.TestDU   hadoop.fs.contract.localfs.TestLocalFSContractSeek   hadoop.fs.TestTruncatedInputBug   hadoop.fs.TestFSMainOperationsLocalFileSystem   hadoop.fs.contract.localfs.TestLocalFSContractSetTimes   hadoop.fs.TestPath   hadoop.io.erasurecode.coder.TestRSErasureCoder   hadoop.util.TestGenericOptionsParser   hadoop.metrics2.sink.TestRollingFileSystemSink   hadoop.crypto.key.TestKeyShell   hadoop.io.file.tfile.TestTFile   hadoop.fs.sftp.TestSFTPFileSystem   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec   hadoop.fs.contract.rawlocal.TestRawlocalContractAppend   hadoop.io.TestWritable   hadoop.ipc.TestProtoBufRPCCompatibility   hadoop.fs.viewfs.TestViewfsFileStatus   hadoop.fs.TestTrash   hadoop.security.TestHttpCrossOriginFilterInitializer   hadoop.ha.TestSshFenceByTcpPort   hadoop.security.authorize.TestProxyServers   hadoop.fs.TestLocalDirAllocator   hadoop.security.alias.TestCredShell   hadoop.ipc.TestIdentityProviders   hadoop.ipc.TestIPCServerResponder   hadoop.ha.TestShellCommandFencer   hadoop.crypto.TestCryptoOutputStreamClosing   hadoop.util.TestNativeLibraryChecker   hadoop.fs.TestFileContextResolveAfs   hadoop.io.TestMapFile   hadoop.fs.shell.TestCopyPreserveFlag   hadoop.cli.TestCLI   hadoop.crypto.key.TestKeyProviderCryptoExtension   hadoop.http.TestSSLHttpServer   hadoop.fs.TestFsShellCopy   hadoop.io.file.tfile.TestTFileSplit   hadoop.ipc.TestAsyncIPC   hadoop.conf.TestConfigRedactor   hadoop.fs.TestFsShell   hadoop.io.file.tfile.TestTFileLzoCodecsByteArrays   hadoop.io.TestBloomMapFile   hadoop.fs.shell.find.TestName   hadoop.io.compress.zlib.TestZlibCompressorDecompressor   hadoop.io.TestSecureIOUtils   hadoop.fs.contract.ftp.TestFTPContractCreate   hadoop.fs.contract.localfs.TestLocalFSContractOpen   hadoop.io.file.tfile.TestTFileComparators   hadoop.fs.contract.localfs.TestLocalFSContractMkdir   hadoop.crypto.TestCryptoStreamsForLocalFS   hadoop.fs.shell.TestTextCommand   hadoop.fs.contract.ftp.TestFTPContractOpen   hadoop.fs.shell.TestCount   hadoop.fs.contract.ftp.TestFTPContractMkdir   hadoop.ipc.TestCallQueueManager   hadoop.io.compress.TestCompressorDecompressor   hadoop.io.file.tfile.TestTFileStreams   hadoop.fs.viewfs.TestViewFsTrash   hadoop.ipc.TestRPCServerShutdown   hadoop.fs.viewfs.TestFcCreateMkdirLocalFs   hadoop.fs.contract.rawlocal.TestRawlocalContractMkdir   hadoop.http.TestServletFilter   hadoop.metrics2.sink.TestRollingFileSystemSinkWithLocal   hadoop.fs.TestDelegateToFileSystem   hadoop.fs.contract.localfs.TestLocalFSContractRename   hadoop.fs.contract.localfs.TestLocalFSContractDelete   hadoop.fs.viewfs.TestFcMainOperationsLocalFs   hadoop.ipc.TestRPCWaitForProxy   hadoop.ha.TestZKFailoverController   hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem   hadoop.security.token.TestDtUtilShell   hadoop.conf.TestConfigurationDeprecation   hadoop.fs.TestRawLocalFileSystemContract   hadoop.security.authorize.TestAccessControlList   hadoop.fs.TestFilterFileSystem   hadoop.security.ssl.TestSSLFactory   hadoop.fs.shell.find.TestPrint0   hadoop.io.erasurecode.coder.TestHHXORErasureCoder   hadoop.io.TestSequenceFile   hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays   hadoop.service.TestServiceLifecycle   hadoop.fs.viewfs.TestViewFileSystemDelegationTokenSupport   hadoop.security.TestUserFromEnv   hadoop.io.file.tfile.TestTFileUnsortedByteArrays   hadoop.fs.shell.TestXAttrCommands   hadoop.ipc.TestMiniRPCBenchmark   hadoop.http.TestHttpServerLogs   hadoop.fs.shell.find.TestIname   hadoop.io.file.tfile.TestTFileSeek   hadoop.util.TestGenericsUtil   hadoop.io.serializer.TestWritableSerialization   hadoop.io.TestSequenceFileSync   hadoop.crypto.key.TestKeyProvider   hadoop.ha.TestNodeFencer   hadoop.fs.TestLocalFSFileContextCreateMkdir   hadoop.crypto.key.TestKeyProviderDelegationTokenExtension   hadoop.ipc.TestRPC   hadoop.fs.contract.ftp.TestFTPContractDelete   hadoop.fs.TestFcLocalFsUtil   hadoop.fs.TestStat   hadoop.io.TestGenericWritable   hadoop.http.TestAuthenticationSessionCookie   hadoop.conf.TestConfServlet   hadoop.fs.contract.rawlocal.TestRawlocalContractSeek   hadoop.util.TestNativeCrc32   hadoop.fs.TestDelegateToFsCheckPath   hadoop.ipc.TestProtoBufRpc   hadoop.security.TestWhitelistBasedResolver   hadoop.io.retry.TestConnectionRetryPolicy   hadoop.net.TestNetUtils   hadoop.fs.shell.find.TestFind   hadoop.util.TestDiskChecker   hadoop.ipc.TestSocketFactory   hadoop.fs.TestLocalFSFileContextMainOperations   hadoop.security.TestLdapGroupsMappingWithOneQuery   hadoop.security.TestKDiag   hadoop.io.TestSequenceFileAppend   hadoop.util.TestBasicDiskValidator   hadoop.fs.contract.localfs.TestLocalFSContractGetFileStatus   hadoop.security.alias.TestCredentialProviderFactory   hadoop.http.TestHttpServerWebapps   hadoop.ipc.TestIPC   hadoop.fs.contract.localfs.TestLocalFSContractAppend   hadoop.ha.TestHAAdmin   hadoop.crypto.key.kms.TestLoadBalancingKMSClientProvider   hadoop.ipc.TestMultipleProtocolServer   hadoop.fs.viewfs.TestChRootedFs   hadoop.ipc.TestWeightedRoundRobinMultiplexer   hadoop.security.token.delegation.TestZKDelegationTokenSecretManager   hadoop.fs.TestFileSystemCanonicalization   hadoop.fs.TestFsShellReturnCode   hadoop.fs.TestLocalFsFCStatistics   hadoop.io.file.tfile.TestTFileNoneCodecsStreams   hadoop.fs.TestHarFileSystem   hadoop.fs.TestSymlinkLocalFSFileContext   hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem   hadoop.fs.contract.localfs.TestLocalFSContractLoaded   hadoop.security.TestUGIWithMiniKdc   hadoop.fs.shell.find.TestPrint   hadoop.net.TestScriptBasedMappingWithDependency   hadoop.util.TestFindClass   hadoop.fs.contract.rawlocal.TestRawlocalContractOpen   hadoop.util.hash.TestHash   hadoop.io.file.tfile.TestTFileComparator2   hadoop.io.compress.bzip2.TestBzip2CompressorDecompressor   hadoop.ipc.TestSaslRPC   hadoop.util.TestRunJar   hadoop.net.TestSwitchMapping   hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs   hadoop.fs.TestLocalFileSystemPermission   hadoop.security.TestJNIGroupsMapping   hadoop.http.lib.TestStaticUserWebFilter   hadoop.security.TestShellBasedIdMapping   hadoop.ipc.TestRPCCallBenchmark   hadoop.io.serializer.avro.TestAvroSerialization   hadoop.fs.shell.TestMove   hadoop.conf.TestGetInstances   hadoop.fs.shell.TestLs   hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays   hadoop.ha.TestHealthMonitorWithDedicatedHealthAddress   hadoop.security.TestKDiagNoKDC   hadoop.conf.TestConfigurationSubclass   hadoop.fs.viewfs.TestViewFsConfig   hadoop.security.TestCompositeGroupMapping   hadoop.fs.contract.localfs.TestLocalFSContractCreate   hadoop.fs.TestListFiles   hadoop.fs.contract.rawlocal.TestRawlocalContractRename   hadoop.security.TestUserGroupInformation   hadoop.security.authorize.TestServiceAuthorization   hadoop.net.TestTableMapping   hadoop.crypto.TestCryptoCodec   hadoop.fs.TestFsShellTouch   hadoop.fs.TestFileUtil   hadoop.security.TestDoAsEffectiveUser   hadoop.fs.TestFcLocalFsPermission   hadoop.util.TestNodeHealthScriptRunner   hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays   hadoop.crypto.TestCryptoStreamsNormal   hadoop.ha.TestZKFailoverControllerStress   hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem   hadoop.fs.contract.rawlocal.TestRawlocalContractCreate   hadoop.io.file.tfile.TestCompression   hadoop.fs.viewfs.TestChRootedFileSystem   hadoop.fs.TestFileContext   hadoop.ipc.TestReuseRpcConnections   hadoop.http.TestHttpCookieFlag   hadoop.security.TestGroupFallback   hadoop.security.token.delegation.web.TestWebDelegationToken   hadoop.net.TestScriptBasedMapping   hadoop.fs.viewfs.TestViewFsLocalFs   hadoop.security.TestProxyUserFromEnv   hadoop.crypto.TestCryptoStreams   hadoop.fs.TestLocalFileSystem   hadoop.io.TestSequenceFileSerialization   hadoop.fs.contract.rawlocal.TestRawlocalContractGetFileStatus   hadoop.io.TestArrayFile   hadoop.conf.TestConfiguration   hadoop.http.TestHttpServerLifecycle   hadoop.fs.shell.TestAclCommands   hadoop.ipc.TestProtoBufRpcServerHandoff   hadoop.security.TestSecurityUtil   hadoop.jmx.TestJMXJsonServlet   hadoop.io.TestDefaultStringifier   hadoop.ipc.TestFairCallQueue   hadoop.fs.TestFsShellList   hadoop.io.erasurecode.coder.TestXORCoder   hadoop.fs.TestFileContextDeleteOnExit   hadoop.conf.TestCommonConfigurationFields   hadoop.security.TestUGILoginFromKeytab   hadoop.io.file.tfile.TestTFileSeqFileComparison   hadoop.io.compress.TestCodec   hadoop.io.serializer.TestSerializationFactory   hadoop.log.TestLogLevel   hadoop.security.TestLdapGroupsMapping   hadoop.metrics2.source.TestJvmMetrics   hadoop.fs.contract.ftp.TestFTPContractRename   hadoop.fs.TestSymlinkLocalFSFileSystem   hadoop.io.compress.TestCodecFactory   hadoop.ipc.TestServer   hadoop.ipc.TestRpcServerHandoff   hadoop.fs.azure.TestFileSystemOperationsExceptionHandlingMultiThreaded   hadoop.fs.azure.contract.TestAzureNativeContractMkdir   hadoop.fs.azure.TestNativeAzureFileSystemConcurrency   hadoop.fs.azure.TestAzureConcurrentOutOfBandIo   hadoop.fs.azure.TestFileSystemOperationExceptionHandling   hadoop.fs.azure.metrics.TestBandwidthGaugeUpdater   hadoop.fs.azure.TestWasbFsck   hadoop.fs.azure.TestNativeAzureFileSystemClientLogging   hadoop.fs.azure.TestOutOfBandAzureBlobOperations   hadoop.fs.azure.TestReadAndSeekPageBlobAfterWrite   hadoop.fs.azure.TestFileSystemOperationExceptionMessage   hadoop.fs.azure.contract.TestAzureNativeContractCreate   hadoop.fs.azure.TestNativeAzureFileSystemContractMocked   hadoop.fs.azure.TestNativeAzureFileSystemContractPageBlobLive   hadoop.fs.azure.TestOutOfBandAzureBlobOperationsLive   hadoop.fs.azure.TestContainerChecks   hadoop.fs.azure.TestNativeAzureFileSystemLive   hadoop.fs.azure.contract.TestAzureNativeContractAppend   hadoop.fs.azure.contract.TestAzureNativeContractDelete   hadoop.fs.azure.TestNativeAzureFileSystemContractLive   hadoop.fs.azure.TestBlobMetadata   hadoop.fs.azure.metrics.TestAzureFileSystemInstrumentation   hadoop.fs.azure.TestNativeAzureFileSystemConcurrencyLive   hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked   hadoop.fs.azure.metrics.TestNativeAzureFileSystemMetricsSystem   hadoop.fs.azure.contract.TestAzureNativeContractOpen   hadoop.fs.azure.TestNativeAzureFileSystemContractEmulator   hadoop.fs.azure.TestNativeAzureFileSystemAtomicRenameDirList   hadoop.fs.azure.contract.TestAzureNativeContractSeek   hadoop.fs.azure.TestNativeAzureFSPageBlobLive   hadoop.fs.azure.TestBlobDataValidation   hadoop.fs.azure.TestBlobTypeSpeedDifference   hadoop.fs.azure.TestWasbUriAndConfiguration   hadoop.fs.azure.contract.TestAzureNativeContractRename   hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck   hadoop.fs.azure.TestNativeAzureFileSystemAppend   hadoop.fs.azure.TestNativeAzureFileSystemMocked   hadoop.fs.azure.TestNativeAzureFileSystemBlockLocations   hadoop.fs.azure.contract.TestAzureNativeContractDistCp   hadoop.fs.azure.contract.TestAzureNativeContractGetFileStatus   hadoop.fs.azure.TestAzureFileSystemErrorConditions   hadoop.fs.azure.TestFileSystemOperationsWithThreads Timed out junit tests org.apache.hadoop.fs.TestFileSystemCaching   org.apache.hadoop.io.nativeio.TestNativeIO Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13863 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843733/HADOOP-13863.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 72d9da9cb4dc 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fcbe152 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/diff-checkstyle-root.txt xml https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/xml.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/new-findbugs-hadoop-tools_hadoop-azure.html javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11297/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          dchickabasapa Dushyanth added a comment -

          Attaching Patch v5 with fixes for stylecheck, findbugs and xml encoding errors.

          Show
          dchickabasapa Dushyanth added a comment - Attaching Patch v5 with fixes for stylecheck, findbugs and xml encoding errors.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 30s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 2m 11s Maven dependency ordering for branch
          +1 mvninstall 14m 1s trunk passed
          +1 compile 9m 56s trunk passed
          +1 checkstyle 1m 56s trunk passed
          +1 mvnsite 1m 31s trunk passed
          +1 mvneclipse 0m 37s trunk passed
          +1 findbugs 2m 0s trunk passed
          +1 javadoc 1m 9s trunk passed
          0 mvndep 0m 19s Maven dependency ordering for patch
          +1 mvninstall 0m 57s the patch passed
          +1 compile 9m 45s the patch passed
          +1 javac 9m 45s the patch passed
          +1 checkstyle 1m 36s the patch passed
          +1 mvnsite 1m 31s the patch passed
          +1 mvneclipse 0m 43s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 xml 0m 3s The patch has 1 ill-formed XML file(s).
          +1 findbugs 2m 20s the patch passed
          -1 javadoc 0m 24s hadoop-azure in the patch failed.
          -1 unit 18m 2s hadoop-common in the patch failed.
          -1 unit 1m 4s hadoop-azure in the patch failed.
          -1 asflicense 0m 40s The patch generated 3 ASF License warnings.
          96m 15s



          Reason Tests
          Failed junit tests hadoop.crypto.TestCryptoStreamsWithJceAesCtrCryptoCodec
            hadoop.io.compress.TestCodec
            hadoop.io.TestSequenceFileSync
            hadoop.io.file.tfile.TestCompression
            hadoop.http.TestHttpServerWebapps
            hadoop.io.TestMapFile
            hadoop.fs.TestDelegateToFsCheckPath
            hadoop.ipc.TestIdentityProviders
            hadoop.conf.TestConfiguration
            hadoop.util.TestNativeCrc32
            hadoop.conf.TestConfigRedactor
            hadoop.fs.TestFsShellCopy
            hadoop.fs.shell.TestCopyPreserveFlag
            hadoop.io.file.tfile.TestTFileComparator2
            hadoop.fs.contract.rawlocal.TestRawlocalContractMkdir
            hadoop.io.file.tfile.TestTFileLzoCodecsByteArrays
            hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem
            hadoop.io.serializer.avro.TestAvroSerialization
            hadoop.ipc.TestServer
            hadoop.fs.viewfs.TestChRootedFileSystem
            hadoop.ha.TestZKFailoverControllerStress
            hadoop.fs.TestHarFileSystemBasics
            hadoop.security.TestProxyUserFromEnv
            hadoop.security.token.delegation.web.TestDelegationTokenAuthenticationHandlerWithMocks
            hadoop.ipc.TestDecayRpcScheduler
            hadoop.fs.shell.TestMove
            hadoop.cli.TestCLI
            hadoop.crypto.key.TestKeyProviderCryptoExtension
            hadoop.fs.permission.TestFsPermission
            hadoop.fs.contract.localfs.TestLocalFSContractSeek
            hadoop.net.TestNetUtils
            hadoop.tracing.TestTraceUtils
            hadoop.fs.viewfs.TestViewFsURIs
            hadoop.crypto.key.TestKeyShell
            hadoop.metrics2.sink.TestRollingFileSystemSink
            hadoop.security.alias.TestCredShell
            hadoop.security.TestUGILoginFromKeytab
            hadoop.fs.TestFileSystemCanonicalization
            hadoop.ha.TestHAAdmin
            hadoop.fs.shell.find.TestName
            hadoop.fs.TestSymlinkLocalFSFileContext
            hadoop.ipc.TestProtoBufRPCCompatibility
            hadoop.http.TestGlobalFilter
            hadoop.crypto.key.TestKeyProviderFactory
            hadoop.ipc.TestSaslRPC
            hadoop.security.TestLdapGroupsMappingWithPosixGroup
            hadoop.fs.contract.ftp.TestFTPContractCreate
            hadoop.ipc.TestSocketFactory
            hadoop.fs.TestLocalFSFileContextMainOperations
            hadoop.security.alias.TestCredentialProviderFactory
            hadoop.fs.TestChecksumFileSystem
            hadoop.fs.contract.localfs.TestLocalFSContractDelete
            hadoop.ipc.TestRPC
            hadoop.io.TestSecureIOUtils
            hadoop.fs.contract.localfs.TestLocalFSContractLoaded
            hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem
            hadoop.io.TestDefaultStringifier
            hadoop.io.compress.TestCompressionStreamReuse
            hadoop.security.TestUserGroupInformation
            hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem
            hadoop.crypto.TestCryptoCodec
            hadoop.ipc.TestProtoBufRpc
            hadoop.fs.TestDelegateToFileSystem
            hadoop.io.erasurecode.coder.TestHHXORErasureCoder
            hadoop.io.TestSetFile
            hadoop.io.serializer.TestWritableSerialization
            hadoop.ipc.TestProtoBufRpcServerHandoff
            hadoop.crypto.TestCryptoStreamsForLocalFS
            hadoop.http.TestServletFilter
            hadoop.security.TestWhitelistBasedResolver
            hadoop.util.TestBasicDiskValidator
            hadoop.conf.TestCommonConfigurationFields
            hadoop.http.TestAuthenticationSessionCookie
            hadoop.fs.TestFcLocalFsUtil
            hadoop.security.TestKDiagNoKDC
            hadoop.net.TestScriptBasedMapping
            hadoop.security.TestShellBasedIdMapping
            hadoop.ipc.TestFairCallQueue
            hadoop.fs.TestTrash
            hadoop.fs.TestFileContextDeleteOnExit
            hadoop.metrics2.sink.TestRollingFileSystemSinkWithLocal
            hadoop.io.TestSequenceFileSerialization
            hadoop.io.file.tfile.TestTFileStreams
            hadoop.jmx.TestJMXJsonServlet
            hadoop.fs.viewfs.TestChRootedFs
            hadoop.security.TestJNIGroupsMapping
            hadoop.security.TestDoAsEffectiveUser
            hadoop.io.file.tfile.TestVLong
            hadoop.util.TestGenericsUtil
            hadoop.security.token.delegation.web.TestWebDelegationToken
            hadoop.security.TestGroupFallback
            hadoop.fs.TestRawLocalFileSystemContract
            hadoop.fs.viewfs.TestViewFsTrash
            hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays
            hadoop.security.TestKDiag
            hadoop.util.TestGenericOptionsParser
            hadoop.fs.shell.TestCount
            hadoop.security.TestUGIWithMiniKdc
            hadoop.ipc.TestRpcServerHandoff
            hadoop.fs.shell.find.TestIname
            hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec
            hadoop.security.ssl.TestSSLFactory
            hadoop.fs.TestLocalFsFCStatistics
            hadoop.fs.TestLocalFSFileContextCreateMkdir
            hadoop.net.TestStaticMapping
            hadoop.fs.TestSymlinkLocalFSFileSystem
            hadoop.fs.TestFilterFileSystem
            hadoop.crypto.TestCryptoStreams
            hadoop.io.serializer.TestSerializationFactory
            hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
            hadoop.fs.TestFileContext
            hadoop.fs.viewfs.TestViewFsLocalFs
            hadoop.fs.TestAvroFSInput
            hadoop.security.TestLdapGroupsMappingWithOneQuery
            hadoop.ipc.TestCallQueueManager
            hadoop.http.TestHttpServerLifecycle
            hadoop.security.TestSecurityUtil
            hadoop.fs.contract.localfs.TestLocalFSContractAppend
            hadoop.ipc.TestRPCCallBenchmark
            hadoop.fs.TestDefaultUri
            hadoop.io.retry.TestConnectionRetryPolicy
            hadoop.security.TestGroupsCaching
            hadoop.io.file.tfile.TestTFileUnsortedByteArrays
            hadoop.conf.TestReconfiguration
            hadoop.ipc.TestAsyncIPC
            hadoop.io.erasurecode.TestCodecRawCoderMapping
            hadoop.util.TestDiskChecker
            hadoop.crypto.key.kms.TestLoadBalancingKMSClientProvider
            hadoop.io.file.tfile.TestTFileLzoCodecsStreams
            hadoop.http.TestHttpServer
            hadoop.fs.shell.find.TestFind
            hadoop.log.TestLogLevel
            hadoop.http.TestHttpCookieFlag
            hadoop.conf.TestGetInstances
            hadoop.fs.viewfs.TestViewfsFileStatus
            hadoop.security.TestAuthenticationFilter
            hadoop.security.token.delegation.web.TestDelegationTokenManager
            hadoop.ha.TestNodeFencer
            hadoop.io.file.tfile.TestTFileNoneCodecsStreams
            hadoop.io.compress.zlib.TestZlibCompressorDecompressor
            hadoop.security.TestCompositeGroupMapping
            hadoop.fs.shell.TestAclCommands
            hadoop.security.authorize.TestProxyUsers
            hadoop.fs.TestFileSystemInitialization
            hadoop.net.TestSwitchMapping
            hadoop.ipc.TestIPC
            hadoop.fs.viewfs.TestFcMainOperationsLocalFs
            hadoop.io.file.tfile.TestTFileSeqFileComparison
            hadoop.fs.contract.ftp.TestFTPContractMkdir
            hadoop.net.TestScriptBasedMappingWithDependency
            hadoop.fs.contract.rawlocal.TestRawlocalContractDelete
            hadoop.security.TestHttpCrossOriginFilterInitializer
            hadoop.util.TestRunJar
            hadoop.fs.TestListFiles
            hadoop.net.TestTableMapping
            hadoop.ha.TestHealthMonitor
            hadoop.crypto.TestCryptoOutputStreamClosing
            hadoop.ha.TestFailoverController
            hadoop.fs.contract.rawlocal.TestRawlocalContractOpen
            hadoop.io.retry.TestDefaultRetryPolicy
            hadoop.security.authorize.TestServiceAuthorization
            hadoop.fs.contract.ftp.TestFTPContractRename
            hadoop.fs.TestPath
            hadoop.fs.shell.TestCopy
            hadoop.fs.shell.TestLs
            hadoop.fs.shell.TestTextCommand
            hadoop.fs.TestFsShellTouch
            hadoop.fs.TestLocalFileSystem
            hadoop.security.token.delegation.TestZKDelegationTokenSecretManager
            hadoop.ha.TestZKFailoverController
            hadoop.io.TestSequenceFileAppend
            hadoop.io.TestArrayFile
            hadoop.io.TestSequenceFile
            hadoop.io.TestWritable
            hadoop.fs.viewfs.TestFcPermissionsLocalFs
            hadoop.fs.contract.rawlocal.TestRawlocalContractRename
            hadoop.http.TestSSLHttpServer
            hadoop.fs.TestStat
            hadoop.fs.shell.TestXAttrCommands
            hadoop.security.TestLdapGroupsMapping
            hadoop.fs.viewfs.TestViewFsConfig
            hadoop.service.TestServiceLifecycle
            hadoop.fs.TestHarFileSystem
            hadoop.fs.contract.localfs.TestLocalFSContractGetFileStatus
            hadoop.ipc.TestReuseRpcConnections
            hadoop.ipc.TestMiniRPCBenchmark
            hadoop.fs.contract.ftp.TestFTPContractOpen
            hadoop.fs.TestFSMainOperationsLocalFileSystem
            hadoop.ha.TestHealthMonitorWithDedicatedHealthAddress
            hadoop.util.TestNativeLibraryChecker
            hadoop.fs.TestGetFileBlockLocations
            hadoop.fs.contract.rawlocal.TestRawlocalContractCreate
            hadoop.fs.contract.localfs.TestLocalFSContractMkdir
            hadoop.io.file.tfile.TestTFileComparators
            hadoop.ipc.TestRPCCompatibility
            hadoop.util.hash.TestHash
            hadoop.fs.contract.rawlocal.TestRawlocalContractSetTimes
            hadoop.http.TestHttpServerLogs
            hadoop.crypto.key.TestKeyProvider
            hadoop.io.file.tfile.TestTFileByteArrays
            hadoop.io.erasurecode.coder.TestRSErasureCoder
            hadoop.fs.contract.rawlocal.TestRawlocalContractGetFileStatus
            hadoop.util.TestApplicationClassLoader
            hadoop.security.authorize.TestProxyServers
            hadoop.ipc.TestRPCServerShutdown
            hadoop.conf.TestConfServlet
            hadoop.crypto.key.TestKeyProviderDelegationTokenExtension
            hadoop.fs.viewfs.TestFcCreateMkdirLocalFs
            hadoop.ipc.TestIPCServerResponder
            hadoop.fs.TestTruncatedInputBug
            hadoop.fs.contract.rawlocal.TestRawlocalContractAppend
            hadoop.fs.TestFsShellList
            hadoop.ha.TestSshFenceByTcpPort
            hadoop.fs.contract.rawlocal.TestRawlocalContractSeek
            hadoop.crypto.random.TestOsSecureRandom
            hadoop.fs.TestFsShellReturnCode
            hadoop.io.compress.TestCodecPool
            hadoop.io.file.tfile.TestTFileSeek
            hadoop.fs.TestFcLocalFsPermission
            hadoop.security.authorize.TestAccessControlList
            hadoop.http.TestPathFilter
            hadoop.ipc.TestWeightedRoundRobinMultiplexer
            hadoop.fs.viewfs.TestViewFileSystemDelegationTokenSupport
            hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays
            hadoop.metrics2.source.TestJvmMetrics
            hadoop.http.lib.TestStaticUserWebFilter
            hadoop.fs.TestLocalDirAllocator
            hadoop.fs.shell.find.TestPrint0
            hadoop.io.TestGenericWritable
            hadoop.fs.contract.ftp.TestFTPContractDelete
            hadoop.io.compress.TestCompressorDecompressor
            hadoop.fs.contract.localfs.TestLocalFSContractCreate
            hadoop.io.TestBloomMapFile
            hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs
            hadoop.fs.viewfs.TestViewFileSystemDelegation
            hadoop.crypto.TestCryptoStreamsNormal
            hadoop.fs.TestGetSpaceUsed
            hadoop.fs.contract.localfs.TestLocalFSContractOpen
            hadoop.ipc.TestMultipleProtocolServer
            hadoop.fs.sftp.TestSFTPFileSystem
            hadoop.io.erasurecode.coder.TestXORCoder
            hadoop.fs.TestDU
            hadoop.security.TestUserFromEnv
            hadoop.fs.shell.TestPathData
            hadoop.io.file.tfile.TestTFileSplit
            hadoop.io.compress.bzip2.TestBzip2CompressorDecompressor
            hadoop.fs.shell.find.TestPrint
            hadoop.util.TestNodeHealthScriptRunner
            hadoop.fs.TestFileContextResolveAfs
            hadoop.security.token.TestDtUtilShell
            hadoop.conf.TestConfigurationSubclass
            hadoop.conf.TestConfigurationDeprecation
            hadoop.io.file.tfile.TestTFile
            hadoop.fs.contract.localfs.TestLocalFSContractSetTimes
            hadoop.fs.TestFileUtil
            hadoop.io.compress.TestCodecFactory
            hadoop.ipc.TestRPCWaitForProxy
            hadoop.fs.contract.localfs.TestLocalFSContractRename
            hadoop.conf.TestDeprecatedKeys
            hadoop.ha.TestShellCommandFencer
            hadoop.util.TestFindClass
            hadoop.fs.TestFsShell
            hadoop.fs.TestLocalFileSystemPermission
            hadoop.fs.azure.contract.TestAzureNativeContractOpen
            hadoop.fs.azure.TestNativeAzureFileSystemContractEmulator
            hadoop.fs.azure.TestNativeAzureFileSystemBlockLocations
            hadoop.fs.azure.TestWasbUriAndConfiguration
            hadoop.fs.azure.TestAzureConcurrentOutOfBandIo
            hadoop.fs.azure.contract.TestAzureNativeContractGetFileStatus
            hadoop.fs.azure.TestFileSystemOperationsExceptionHandlingMultiThreaded
            hadoop.fs.azure.contract.TestAzureNativeContractSeek
            hadoop.fs.azure.contract.TestAzureNativeContractCreate
            hadoop.fs.azure.TestWasbFsck
            hadoop.fs.azure.contract.TestAzureNativeContractRename
            hadoop.fs.azure.TestNativeAzureFileSystemAppend
            hadoop.fs.azure.TestContainerChecks
            hadoop.fs.azure.TestNativeAzureFSPageBlobLive
            hadoop.fs.azure.TestBlobDataValidation
            hadoop.fs.azure.TestFileSystemOperationExceptionHandling
            hadoop.fs.azure.TestAzureFileSystemErrorConditions
            hadoop.fs.azure.metrics.TestAzureFileSystemInstrumentation
            hadoop.fs.azure.TestNativeAzureFileSystemLive
            hadoop.fs.azure.TestFileSystemOperationExceptionMessage
            hadoop.fs.azure.contract.TestAzureNativeContractDelete
            hadoop.fs.azure.TestOutOfBandAzureBlobOperations
            hadoop.fs.azure.metrics.TestBandwidthGaugeUpdater
            hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked
            hadoop.fs.azure.TestNativeAzureFileSystemMocked
            hadoop.fs.azure.TestNativeAzureFileSystemClientLogging
            hadoop.fs.azure.TestBlobTypeSpeedDifference
            hadoop.fs.azure.contract.TestAzureNativeContractAppend
            hadoop.fs.azure.TestReadAndSeekPageBlobAfterWrite
            hadoop.fs.azure.contract.TestAzureNativeContractMkdir
            hadoop.fs.azure.TestOutOfBandAzureBlobOperationsLive
            hadoop.fs.azure.TestNativeAzureFileSystemContractPageBlobLive
            hadoop.fs.azure.TestFileSystemOperationsWithThreads
            hadoop.fs.azure.TestNativeAzureFileSystemContractMocked
            hadoop.fs.azure.TestBlobMetadata
            hadoop.fs.azure.TestNativeAzureFileSystemContractLive
            hadoop.fs.azure.TestNativeAzureFileSystemConcurrency
            hadoop.fs.azure.TestNativeAzureFileSystemConcurrencyLive
            hadoop.fs.azure.contract.TestAzureNativeContractDistCp
            hadoop.fs.azure.metrics.TestNativeAzureFileSystemMetricsSystem
            hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck
            hadoop.fs.azure.TestNativeAzureFileSystemAtomicRenameDirList
          Timed out junit tests org.apache.hadoop.fs.TestFileSystemCaching
            org.apache.hadoop.io.nativeio.TestNativeIO



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13863
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844151/HADOOP-13863.005.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 5499b8f1e051 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f678080
          Default Java 1.8.0_111
          findbugs v3.0.0
          xml https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/xml.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-azure.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 30s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 2m 11s Maven dependency ordering for branch +1 mvninstall 14m 1s trunk passed +1 compile 9m 56s trunk passed +1 checkstyle 1m 56s trunk passed +1 mvnsite 1m 31s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 2m 0s trunk passed +1 javadoc 1m 9s trunk passed 0 mvndep 0m 19s Maven dependency ordering for patch +1 mvninstall 0m 57s the patch passed +1 compile 9m 45s the patch passed +1 javac 9m 45s the patch passed +1 checkstyle 1m 36s the patch passed +1 mvnsite 1m 31s the patch passed +1 mvneclipse 0m 43s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 xml 0m 3s The patch has 1 ill-formed XML file(s). +1 findbugs 2m 20s the patch passed -1 javadoc 0m 24s hadoop-azure in the patch failed. -1 unit 18m 2s hadoop-common in the patch failed. -1 unit 1m 4s hadoop-azure in the patch failed. -1 asflicense 0m 40s The patch generated 3 ASF License warnings. 96m 15s Reason Tests Failed junit tests hadoop.crypto.TestCryptoStreamsWithJceAesCtrCryptoCodec   hadoop.io.compress.TestCodec   hadoop.io.TestSequenceFileSync   hadoop.io.file.tfile.TestCompression   hadoop.http.TestHttpServerWebapps   hadoop.io.TestMapFile   hadoop.fs.TestDelegateToFsCheckPath   hadoop.ipc.TestIdentityProviders   hadoop.conf.TestConfiguration   hadoop.util.TestNativeCrc32   hadoop.conf.TestConfigRedactor   hadoop.fs.TestFsShellCopy   hadoop.fs.shell.TestCopyPreserveFlag   hadoop.io.file.tfile.TestTFileComparator2   hadoop.fs.contract.rawlocal.TestRawlocalContractMkdir   hadoop.io.file.tfile.TestTFileLzoCodecsByteArrays   hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem   hadoop.io.serializer.avro.TestAvroSerialization   hadoop.ipc.TestServer   hadoop.fs.viewfs.TestChRootedFileSystem   hadoop.ha.TestZKFailoverControllerStress   hadoop.fs.TestHarFileSystemBasics   hadoop.security.TestProxyUserFromEnv   hadoop.security.token.delegation.web.TestDelegationTokenAuthenticationHandlerWithMocks   hadoop.ipc.TestDecayRpcScheduler   hadoop.fs.shell.TestMove   hadoop.cli.TestCLI   hadoop.crypto.key.TestKeyProviderCryptoExtension   hadoop.fs.permission.TestFsPermission   hadoop.fs.contract.localfs.TestLocalFSContractSeek   hadoop.net.TestNetUtils   hadoop.tracing.TestTraceUtils   hadoop.fs.viewfs.TestViewFsURIs   hadoop.crypto.key.TestKeyShell   hadoop.metrics2.sink.TestRollingFileSystemSink   hadoop.security.alias.TestCredShell   hadoop.security.TestUGILoginFromKeytab   hadoop.fs.TestFileSystemCanonicalization   hadoop.ha.TestHAAdmin   hadoop.fs.shell.find.TestName   hadoop.fs.TestSymlinkLocalFSFileContext   hadoop.ipc.TestProtoBufRPCCompatibility   hadoop.http.TestGlobalFilter   hadoop.crypto.key.TestKeyProviderFactory   hadoop.ipc.TestSaslRPC   hadoop.security.TestLdapGroupsMappingWithPosixGroup   hadoop.fs.contract.ftp.TestFTPContractCreate   hadoop.ipc.TestSocketFactory   hadoop.fs.TestLocalFSFileContextMainOperations   hadoop.security.alias.TestCredentialProviderFactory   hadoop.fs.TestChecksumFileSystem   hadoop.fs.contract.localfs.TestLocalFSContractDelete   hadoop.ipc.TestRPC   hadoop.io.TestSecureIOUtils   hadoop.fs.contract.localfs.TestLocalFSContractLoaded   hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem   hadoop.io.TestDefaultStringifier   hadoop.io.compress.TestCompressionStreamReuse   hadoop.security.TestUserGroupInformation   hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem   hadoop.crypto.TestCryptoCodec   hadoop.ipc.TestProtoBufRpc   hadoop.fs.TestDelegateToFileSystem   hadoop.io.erasurecode.coder.TestHHXORErasureCoder   hadoop.io.TestSetFile   hadoop.io.serializer.TestWritableSerialization   hadoop.ipc.TestProtoBufRpcServerHandoff   hadoop.crypto.TestCryptoStreamsForLocalFS   hadoop.http.TestServletFilter   hadoop.security.TestWhitelistBasedResolver   hadoop.util.TestBasicDiskValidator   hadoop.conf.TestCommonConfigurationFields   hadoop.http.TestAuthenticationSessionCookie   hadoop.fs.TestFcLocalFsUtil   hadoop.security.TestKDiagNoKDC   hadoop.net.TestScriptBasedMapping   hadoop.security.TestShellBasedIdMapping   hadoop.ipc.TestFairCallQueue   hadoop.fs.TestTrash   hadoop.fs.TestFileContextDeleteOnExit   hadoop.metrics2.sink.TestRollingFileSystemSinkWithLocal   hadoop.io.TestSequenceFileSerialization   hadoop.io.file.tfile.TestTFileStreams   hadoop.jmx.TestJMXJsonServlet   hadoop.fs.viewfs.TestChRootedFs   hadoop.security.TestJNIGroupsMapping   hadoop.security.TestDoAsEffectiveUser   hadoop.io.file.tfile.TestVLong   hadoop.util.TestGenericsUtil   hadoop.security.token.delegation.web.TestWebDelegationToken   hadoop.security.TestGroupFallback   hadoop.fs.TestRawLocalFileSystemContract   hadoop.fs.viewfs.TestViewFsTrash   hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays   hadoop.security.TestKDiag   hadoop.util.TestGenericOptionsParser   hadoop.fs.shell.TestCount   hadoop.security.TestUGIWithMiniKdc   hadoop.ipc.TestRpcServerHandoff   hadoop.fs.shell.find.TestIname   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec   hadoop.security.ssl.TestSSLFactory   hadoop.fs.TestLocalFsFCStatistics   hadoop.fs.TestLocalFSFileContextCreateMkdir   hadoop.net.TestStaticMapping   hadoop.fs.TestSymlinkLocalFSFileSystem   hadoop.fs.TestFilterFileSystem   hadoop.crypto.TestCryptoStreams   hadoop.io.serializer.TestSerializationFactory   hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays   hadoop.fs.TestFileContext   hadoop.fs.viewfs.TestViewFsLocalFs   hadoop.fs.TestAvroFSInput   hadoop.security.TestLdapGroupsMappingWithOneQuery   hadoop.ipc.TestCallQueueManager   hadoop.http.TestHttpServerLifecycle   hadoop.security.TestSecurityUtil   hadoop.fs.contract.localfs.TestLocalFSContractAppend   hadoop.ipc.TestRPCCallBenchmark   hadoop.fs.TestDefaultUri   hadoop.io.retry.TestConnectionRetryPolicy   hadoop.security.TestGroupsCaching   hadoop.io.file.tfile.TestTFileUnsortedByteArrays   hadoop.conf.TestReconfiguration   hadoop.ipc.TestAsyncIPC   hadoop.io.erasurecode.TestCodecRawCoderMapping   hadoop.util.TestDiskChecker   hadoop.crypto.key.kms.TestLoadBalancingKMSClientProvider   hadoop.io.file.tfile.TestTFileLzoCodecsStreams   hadoop.http.TestHttpServer   hadoop.fs.shell.find.TestFind   hadoop.log.TestLogLevel   hadoop.http.TestHttpCookieFlag   hadoop.conf.TestGetInstances   hadoop.fs.viewfs.TestViewfsFileStatus   hadoop.security.TestAuthenticationFilter   hadoop.security.token.delegation.web.TestDelegationTokenManager   hadoop.ha.TestNodeFencer   hadoop.io.file.tfile.TestTFileNoneCodecsStreams   hadoop.io.compress.zlib.TestZlibCompressorDecompressor   hadoop.security.TestCompositeGroupMapping   hadoop.fs.shell.TestAclCommands   hadoop.security.authorize.TestProxyUsers   hadoop.fs.TestFileSystemInitialization   hadoop.net.TestSwitchMapping   hadoop.ipc.TestIPC   hadoop.fs.viewfs.TestFcMainOperationsLocalFs   hadoop.io.file.tfile.TestTFileSeqFileComparison   hadoop.fs.contract.ftp.TestFTPContractMkdir   hadoop.net.TestScriptBasedMappingWithDependency   hadoop.fs.contract.rawlocal.TestRawlocalContractDelete   hadoop.security.TestHttpCrossOriginFilterInitializer   hadoop.util.TestRunJar   hadoop.fs.TestListFiles   hadoop.net.TestTableMapping   hadoop.ha.TestHealthMonitor   hadoop.crypto.TestCryptoOutputStreamClosing   hadoop.ha.TestFailoverController   hadoop.fs.contract.rawlocal.TestRawlocalContractOpen   hadoop.io.retry.TestDefaultRetryPolicy   hadoop.security.authorize.TestServiceAuthorization   hadoop.fs.contract.ftp.TestFTPContractRename   hadoop.fs.TestPath   hadoop.fs.shell.TestCopy   hadoop.fs.shell.TestLs   hadoop.fs.shell.TestTextCommand   hadoop.fs.TestFsShellTouch   hadoop.fs.TestLocalFileSystem   hadoop.security.token.delegation.TestZKDelegationTokenSecretManager   hadoop.ha.TestZKFailoverController   hadoop.io.TestSequenceFileAppend   hadoop.io.TestArrayFile   hadoop.io.TestSequenceFile   hadoop.io.TestWritable   hadoop.fs.viewfs.TestFcPermissionsLocalFs   hadoop.fs.contract.rawlocal.TestRawlocalContractRename   hadoop.http.TestSSLHttpServer   hadoop.fs.TestStat   hadoop.fs.shell.TestXAttrCommands   hadoop.security.TestLdapGroupsMapping   hadoop.fs.viewfs.TestViewFsConfig   hadoop.service.TestServiceLifecycle   hadoop.fs.TestHarFileSystem   hadoop.fs.contract.localfs.TestLocalFSContractGetFileStatus   hadoop.ipc.TestReuseRpcConnections   hadoop.ipc.TestMiniRPCBenchmark   hadoop.fs.contract.ftp.TestFTPContractOpen   hadoop.fs.TestFSMainOperationsLocalFileSystem   hadoop.ha.TestHealthMonitorWithDedicatedHealthAddress   hadoop.util.TestNativeLibraryChecker   hadoop.fs.TestGetFileBlockLocations   hadoop.fs.contract.rawlocal.TestRawlocalContractCreate   hadoop.fs.contract.localfs.TestLocalFSContractMkdir   hadoop.io.file.tfile.TestTFileComparators   hadoop.ipc.TestRPCCompatibility   hadoop.util.hash.TestHash   hadoop.fs.contract.rawlocal.TestRawlocalContractSetTimes   hadoop.http.TestHttpServerLogs   hadoop.crypto.key.TestKeyProvider   hadoop.io.file.tfile.TestTFileByteArrays   hadoop.io.erasurecode.coder.TestRSErasureCoder   hadoop.fs.contract.rawlocal.TestRawlocalContractGetFileStatus   hadoop.util.TestApplicationClassLoader   hadoop.security.authorize.TestProxyServers   hadoop.ipc.TestRPCServerShutdown   hadoop.conf.TestConfServlet   hadoop.crypto.key.TestKeyProviderDelegationTokenExtension   hadoop.fs.viewfs.TestFcCreateMkdirLocalFs   hadoop.ipc.TestIPCServerResponder   hadoop.fs.TestTruncatedInputBug   hadoop.fs.contract.rawlocal.TestRawlocalContractAppend   hadoop.fs.TestFsShellList   hadoop.ha.TestSshFenceByTcpPort   hadoop.fs.contract.rawlocal.TestRawlocalContractSeek   hadoop.crypto.random.TestOsSecureRandom   hadoop.fs.TestFsShellReturnCode   hadoop.io.compress.TestCodecPool   hadoop.io.file.tfile.TestTFileSeek   hadoop.fs.TestFcLocalFsPermission   hadoop.security.authorize.TestAccessControlList   hadoop.http.TestPathFilter   hadoop.ipc.TestWeightedRoundRobinMultiplexer   hadoop.fs.viewfs.TestViewFileSystemDelegationTokenSupport   hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays   hadoop.metrics2.source.TestJvmMetrics   hadoop.http.lib.TestStaticUserWebFilter   hadoop.fs.TestLocalDirAllocator   hadoop.fs.shell.find.TestPrint0   hadoop.io.TestGenericWritable   hadoop.fs.contract.ftp.TestFTPContractDelete   hadoop.io.compress.TestCompressorDecompressor   hadoop.fs.contract.localfs.TestLocalFSContractCreate   hadoop.io.TestBloomMapFile   hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs   hadoop.fs.viewfs.TestViewFileSystemDelegation   hadoop.crypto.TestCryptoStreamsNormal   hadoop.fs.TestGetSpaceUsed   hadoop.fs.contract.localfs.TestLocalFSContractOpen   hadoop.ipc.TestMultipleProtocolServer   hadoop.fs.sftp.TestSFTPFileSystem   hadoop.io.erasurecode.coder.TestXORCoder   hadoop.fs.TestDU   hadoop.security.TestUserFromEnv   hadoop.fs.shell.TestPathData   hadoop.io.file.tfile.TestTFileSplit   hadoop.io.compress.bzip2.TestBzip2CompressorDecompressor   hadoop.fs.shell.find.TestPrint   hadoop.util.TestNodeHealthScriptRunner   hadoop.fs.TestFileContextResolveAfs   hadoop.security.token.TestDtUtilShell   hadoop.conf.TestConfigurationSubclass   hadoop.conf.TestConfigurationDeprecation   hadoop.io.file.tfile.TestTFile   hadoop.fs.contract.localfs.TestLocalFSContractSetTimes   hadoop.fs.TestFileUtil   hadoop.io.compress.TestCodecFactory   hadoop.ipc.TestRPCWaitForProxy   hadoop.fs.contract.localfs.TestLocalFSContractRename   hadoop.conf.TestDeprecatedKeys   hadoop.ha.TestShellCommandFencer   hadoop.util.TestFindClass   hadoop.fs.TestFsShell   hadoop.fs.TestLocalFileSystemPermission   hadoop.fs.azure.contract.TestAzureNativeContractOpen   hadoop.fs.azure.TestNativeAzureFileSystemContractEmulator   hadoop.fs.azure.TestNativeAzureFileSystemBlockLocations   hadoop.fs.azure.TestWasbUriAndConfiguration   hadoop.fs.azure.TestAzureConcurrentOutOfBandIo   hadoop.fs.azure.contract.TestAzureNativeContractGetFileStatus   hadoop.fs.azure.TestFileSystemOperationsExceptionHandlingMultiThreaded   hadoop.fs.azure.contract.TestAzureNativeContractSeek   hadoop.fs.azure.contract.TestAzureNativeContractCreate   hadoop.fs.azure.TestWasbFsck   hadoop.fs.azure.contract.TestAzureNativeContractRename   hadoop.fs.azure.TestNativeAzureFileSystemAppend   hadoop.fs.azure.TestContainerChecks   hadoop.fs.azure.TestNativeAzureFSPageBlobLive   hadoop.fs.azure.TestBlobDataValidation   hadoop.fs.azure.TestFileSystemOperationExceptionHandling   hadoop.fs.azure.TestAzureFileSystemErrorConditions   hadoop.fs.azure.metrics.TestAzureFileSystemInstrumentation   hadoop.fs.azure.TestNativeAzureFileSystemLive   hadoop.fs.azure.TestFileSystemOperationExceptionMessage   hadoop.fs.azure.contract.TestAzureNativeContractDelete   hadoop.fs.azure.TestOutOfBandAzureBlobOperations   hadoop.fs.azure.metrics.TestBandwidthGaugeUpdater   hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked   hadoop.fs.azure.TestNativeAzureFileSystemMocked   hadoop.fs.azure.TestNativeAzureFileSystemClientLogging   hadoop.fs.azure.TestBlobTypeSpeedDifference   hadoop.fs.azure.contract.TestAzureNativeContractAppend   hadoop.fs.azure.TestReadAndSeekPageBlobAfterWrite   hadoop.fs.azure.contract.TestAzureNativeContractMkdir   hadoop.fs.azure.TestOutOfBandAzureBlobOperationsLive   hadoop.fs.azure.TestNativeAzureFileSystemContractPageBlobLive   hadoop.fs.azure.TestFileSystemOperationsWithThreads   hadoop.fs.azure.TestNativeAzureFileSystemContractMocked   hadoop.fs.azure.TestBlobMetadata   hadoop.fs.azure.TestNativeAzureFileSystemContractLive   hadoop.fs.azure.TestNativeAzureFileSystemConcurrency   hadoop.fs.azure.TestNativeAzureFileSystemConcurrencyLive   hadoop.fs.azure.contract.TestAzureNativeContractDistCp   hadoop.fs.azure.metrics.TestNativeAzureFileSystemMetricsSystem   hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck   hadoop.fs.azure.TestNativeAzureFileSystemAtomicRenameDirList Timed out junit tests org.apache.hadoop.fs.TestFileSystemCaching   org.apache.hadoop.io.nativeio.TestNativeIO Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13863 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844151/HADOOP-13863.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 5499b8f1e051 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f678080 Default Java 1.8.0_111 findbugs v3.0.0 xml https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/xml.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-azure.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11307/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Uploading a refined patch for addressing the unit test failures.

          Show
          liuml07 Mingliang Liu added a comment - Uploading a refined patch for addressing the unit test failures.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 1m 49s Maven dependency ordering for branch
          +1 mvninstall 12m 47s trunk passed
          +1 compile 9m 36s trunk passed
          +1 checkstyle 1m 33s trunk passed
          +1 mvnsite 1m 26s trunk passed
          +1 mvneclipse 0m 38s trunk passed
          +1 findbugs 1m 59s trunk passed
          +1 javadoc 1m 10s trunk passed
          0 mvndep 0m 18s Maven dependency ordering for patch
          +1 mvninstall 0m 57s the patch passed
          +1 compile 9m 21s the patch passed
          +1 javac 9m 21s the patch passed
          +1 checkstyle 1m 34s the patch passed
          +1 mvnsite 1m 28s the patch passed
          +1 mvneclipse 0m 44s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 2m 19s the patch passed
          -1 javadoc 0m 24s hadoop-azure in the patch failed.
          -1 unit 8m 1s hadoop-common in the patch failed.
          +1 unit 1m 35s hadoop-azure in the patch passed.
          +1 asflicense 0m 38s The patch does not generate ASF License warnings.
          83m 40s



          Reason Tests
          Failed junit tests hadoop.conf.TestCommonConfigurationFields



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13863
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844171/HADOOP-13863.006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux e091f81698dc 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 30f85d7
          Default Java 1.8.0_111
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11308/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11308/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11308/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11308/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 1m 49s Maven dependency ordering for branch +1 mvninstall 12m 47s trunk passed +1 compile 9m 36s trunk passed +1 checkstyle 1m 33s trunk passed +1 mvnsite 1m 26s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 1m 59s trunk passed +1 javadoc 1m 10s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 0m 57s the patch passed +1 compile 9m 21s the patch passed +1 javac 9m 21s the patch passed +1 checkstyle 1m 34s the patch passed +1 mvnsite 1m 28s the patch passed +1 mvneclipse 0m 44s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 2m 19s the patch passed -1 javadoc 0m 24s hadoop-azure in the patch failed. -1 unit 8m 1s hadoop-common in the patch failed. +1 unit 1m 35s hadoop-azure in the patch passed. +1 asflicense 0m 38s The patch does not generate ASF License warnings. 83m 40s Reason Tests Failed junit tests hadoop.conf.TestCommonConfigurationFields Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13863 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844171/HADOOP-13863.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux e091f81698dc 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 30f85d7 Default Java 1.8.0_111 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11308/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11308/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11308/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11308/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment - - edited

          The patch looks good to me overall.

          1. In core-default.xml,

            If fs.azure.secure flag is set to false, this flag has no effect.

            I think here we should refer to fs.azure.secure.mode?

          2. Should we also add the default value of fs.azure.sas.expiry.period to the core-default.xml file? Also please document that the config key supports multiple time unit suffix(case insensitive), as described in dfs.heartbeat.interval.
          3. The variable is declared public for visibility in Tests this comment can be replaced with annotation @VisibleForTesting
          4. Can you file separate JIRA to address the javac errors? I think they're mostly not related to this patch. Can you confirm that?
          5. The follow code throws an exception. Is it actually an assert (bug in implementation code instead of usage/configuration)? I'm not very sure about this.
            AzureNativeFileSystemStore#connectToAzureStorageInSecureMode()
            839	    if (!(this.storageInteractionLayer instanceof SecureStorageInterfaceImpl)) {
            840	      throw new AzureException("Invalid State reached."
            841	          + " connectToAzureStorageInSASKeyMode called when"
            842	          + " SASStorageInterfaceImpl not used");
            843	    }
            
          6. Can you add comment for the following code? It's not very straightforward; other code in this method is documented well.
            AzureNativeFileSystemStore#createAzureStorageSession()
            984	      if (useSecureMode) {
            985	        connectToAzureStorageInSecureMode(accountName, containerName, sessionUri);
            986	        return;
            987	      }
            
          7. WasbRemoteCallHelper class be package local?
          Show
          liuml07 Mingliang Liu added a comment - - edited The patch looks good to me overall. In core-default.xml , If fs.azure.secure flag is set to false, this flag has no effect. I think here we should refer to fs.azure.secure.mode ? Should we also add the default value of fs.azure.sas.expiry.period to the core-default.xml file? Also please document that the config key supports multiple time unit suffix(case insensitive), as described in dfs.heartbeat.interval. The variable is declared public for visibility in Tests this comment can be replaced with annotation @VisibleForTesting Can you file separate JIRA to address the javac errors? I think they're mostly not related to this patch. Can you confirm that? The follow code throws an exception. Is it actually an assert (bug in implementation code instead of usage/configuration)? I'm not very sure about this. AzureNativeFileSystemStore#connectToAzureStorageInSecureMode() 839 if (!( this .storageInteractionLayer instanceof SecureStorageInterfaceImpl)) { 840 throw new AzureException( "Invalid State reached." 841 + " connectToAzureStorageInSASKeyMode called when" 842 + " SASStorageInterfaceImpl not used" ); 843 } Can you add comment for the following code? It's not very straightforward; other code in this method is documented well. AzureNativeFileSystemStore#createAzureStorageSession() 984 if (useSecureMode) { 985 connectToAzureStorageInSecureMode(accountName, containerName, sessionUri); 986 return ; 987 } WasbRemoteCallHelper class be package local?
          Hide
          dchickabasapa Dushyanth added a comment -

          Mingliang Liu Thanks a lot for the review.

          I have addressed comments in Patch v7

          Show
          dchickabasapa Dushyanth added a comment - Mingliang Liu Thanks a lot for the review. I have addressed comments in Patch v7
          Hide
          dchickabasapa Dushyanth added a comment -

          I have created HADOOP-13938 to fix the javadoc errors.

          Show
          dchickabasapa Dushyanth added a comment - I have created HADOOP-13938 to fix the javadoc errors.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 41s Maven dependency ordering for branch
          +1 mvninstall 12m 32s trunk passed
          +1 compile 9m 30s trunk passed
          +1 checkstyle 1m 32s trunk passed
          +1 mvnsite 1m 27s trunk passed
          +1 mvneclipse 0m 38s trunk passed
          +1 findbugs 2m 10s trunk passed
          +1 javadoc 1m 16s trunk passed
          0 mvndep 0m 20s Maven dependency ordering for patch
          +1 mvninstall 1m 17s the patch passed
          +1 compile 9m 53s the patch passed
          +1 javac 9m 53s the patch passed
          +1 checkstyle 1m 42s the patch passed
          +1 mvnsite 1m 30s the patch passed
          +1 mvneclipse 0m 44s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 2m 21s the patch passed
          -1 javadoc 0m 24s hadoop-azure in the patch failed.
          -1 unit 9m 20s hadoop-common in the patch failed.
          +1 unit 1m 42s hadoop-azure in the patch passed.
          +1 asflicense 0m 34s The patch does not generate ASF License warnings.
          84m 38s



          Reason Tests
          Failed junit tests hadoop.ha.TestZKFailoverController
            hadoop.security.token.delegation.TestZKDelegationTokenSecretManager
            hadoop.conf.TestCommonConfigurationFields



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13863
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844498/HADOOP-13863.007.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 76c0d12f3cfb 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 22befbd
          Default Java 1.8.0_111
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11314/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11314/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11314/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11314/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 41s Maven dependency ordering for branch +1 mvninstall 12m 32s trunk passed +1 compile 9m 30s trunk passed +1 checkstyle 1m 32s trunk passed +1 mvnsite 1m 27s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 2m 10s trunk passed +1 javadoc 1m 16s trunk passed 0 mvndep 0m 20s Maven dependency ordering for patch +1 mvninstall 1m 17s the patch passed +1 compile 9m 53s the patch passed +1 javac 9m 53s the patch passed +1 checkstyle 1m 42s the patch passed +1 mvnsite 1m 30s the patch passed +1 mvneclipse 0m 44s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 2m 21s the patch passed -1 javadoc 0m 24s hadoop-azure in the patch failed. -1 unit 9m 20s hadoop-common in the patch failed. +1 unit 1m 42s hadoop-azure in the patch passed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 84m 38s Reason Tests Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.security.token.delegation.TestZKDelegationTokenSecretManager   hadoop.conf.TestCommonConfigurationFields Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13863 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844498/HADOOP-13863.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 76c0d12f3cfb 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 22befbd Default Java 1.8.0_111 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/11314/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11314/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11314/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11314/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Committed to trunk and branch-2 branches. Thanks for your contribution Dushyanth. Thanks for your review Steve Loughran.

          Show
          liuml07 Mingliang Liu added a comment - Committed to trunk and branch-2 branches. Thanks for your contribution Dushyanth . Thanks for your review Steve Loughran .
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11032 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11032/)
          HADOOP-13863. Azure: Add a new SAS key mode for WASB. Contributed by (liuml07: rev e92a77099b91620cee84513cc879089907468075)

          • (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SecureModeException.java
          • (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/RemoteSASKeyGeneratorImpl.java
          • (edit) hadoop-tools/hadoop-azure/src/site/markdown/index.md
          • (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SASKeyGeneratorInterface.java
          • (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          • (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/LocalSASKeyGeneratorImpl.java
          • (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallException.java
          • (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestWasbUriAndConfiguration.java
          • (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java
          • (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SASKeyGeneratorImpl.java
          • (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SecureStorageInterfaceImpl.java
          • (edit) hadoop-tools/hadoop-azure/src/test/resources/azure-test.xml
          • (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java
          • (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestContainerChecks.java
          • (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SASKeyGenerationException.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11032 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11032/ ) HADOOP-13863 . Azure: Add a new SAS key mode for WASB. Contributed by (liuml07: rev e92a77099b91620cee84513cc879089907468075) (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SecureModeException.java (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/RemoteSASKeyGeneratorImpl.java (edit) hadoop-tools/hadoop-azure/src/site/markdown/index.md (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SASKeyGeneratorInterface.java (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/LocalSASKeyGeneratorImpl.java (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallException.java (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestWasbUriAndConfiguration.java (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SASKeyGeneratorImpl.java (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SecureStorageInterfaceImpl.java (edit) hadoop-tools/hadoop-azure/src/test/resources/azure-test.xml (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestContainerChecks.java (add) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/SASKeyGenerationException.java
          Hide
          lewuathe Kai Sasaki added a comment -

          This patch seems to broke hadoop.conf.TestCommonConfigurationFields. Do we need to include AzureNativeFileSystemStore in TestCommonConfigurationFields?
          But it will introduce cyclic dependencies between hadoop-common and hadoop-azure modules.

          Show
          lewuathe Kai Sasaki added a comment - This patch seems to broke hadoop.conf.TestCommonConfigurationFields . Do we need to include AzureNativeFileSystemStore in TestCommonConfigurationFields ? But it will introduce cyclic dependencies between hadoop-common and hadoop-azure modules.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks for reporting this. For fixing it, we just need to ignore those newly added config keys in the unit test. No need to bring the module dependency or the cyclic problem.

          Show
          liuml07 Mingliang Liu added a comment - Thanks for reporting this. For fixing it, we just need to ignore those newly added config keys in the unit test. No need to bring the module dependency or the cyclic problem.
          Hide
          lewuathe Kai Sasaki added a comment -

          I found the issue is now tried to be fixed in HADOOP-13943. Thanks for taking care.
          https://issues.apache.org/jira/browse/HADOOP-13943

          Show
          lewuathe Kai Sasaki added a comment - I found the issue is now tried to be fixed in HADOOP-13943 . Thanks for taking care. https://issues.apache.org/jira/browse/HADOOP-13943

            People

            • Assignee:
              dchickabasapa Dushyanth
              Reporter:
              dchickabasapa Dushyanth
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development