Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13929

ADLS connector should not check in contract-test-options.xml

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha2
    • Fix Version/s: 2.8.0, 3.0.0-alpha4
    • Component/s: fs/adl, test
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      To run live unit tests, create src/test/resources/auth-keys.xml with the same properties as in the deprecated contract-test-options.xml.

      Description

      Should not check in the file contract-test-options.xml. Make sure the file is excluded by .gitignore. Make sure ADLS index.md provides a complete example of this file.

      1. HADOOP-13929.001.patch
        7 kB
        John Zhuge
      2. HADOOP-13929.002.patch
        7 kB
        John Zhuge
      3. HADOOP-13929.003.patch
        19 kB
        John Zhuge
      4. HADOOP-13929.004.patch
        19 kB
        John Zhuge
      5. HADOOP-13929.005.patch
        19 kB
        John Zhuge
      6. HADOOP-13929.006.patch
        20 kB
        John Zhuge
      7. HADOOP-13929.007.patch
        23 kB
        John Zhuge
      8. HADOOP-13929.008.patch
        24 kB
        John Zhuge
      9. HADOOP-13929.009.patch
        12 kB
        John Zhuge
      10. HADOOP-13929.010.patch
        11 kB
        John Zhuge
      11. HADOOP-13929.011.patch
        11 kB
        John Zhuge

        Issue Links

          Activity

          Hide
          jzhuge John Zhuge added a comment -

          Patch 001

          • Remove contract-test-options.xml from source control
          • Add sample contract test properties to index.md
          • Add contract-test-options.xml and auth-keys.xml without path to .gitignore

          Chris Nauroth, Steve Loughran, the patch modifies .gitignore so that git ignores contract-test-options.xml and auth-keys.xml in any directory. Let me know whether it is desirable because currently only full paths are specified.

          Show
          jzhuge John Zhuge added a comment - Patch 001 Remove contract-test-options.xml from source control Add sample contract test properties to index.md Add contract-test-options.xml and auth-keys.xml without path to .gitignore Chris Nauroth , Steve Loughran , the patch modifies .gitignore so that git ignores contract-test-options.xml and auth-keys.xml in any directory. Let me know whether it is desirable because currently only full paths are specified.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 1m 44s Maven dependency ordering for branch
          +1 mvninstall 12m 28s trunk passed
          +1 mvnsite 8m 4s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvnsite 7m 53s the patch passed
          -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 unit 9m 40s root in the patch passed.
          +1 asflicense 0m 31s The patch does not generate ASF License warnings.
          41m 19s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844615/HADOOP-13929.001.patch
          Optional Tests asflicense mvnsite unit xml
          uname Linux 6b0efa1ba22d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c721f78
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11319/artifact/patchprocess/whitespace-eol.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11319/testReport/
          modules C: hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11319/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 44s Maven dependency ordering for branch +1 mvninstall 12m 28s trunk passed +1 mvnsite 8m 4s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvnsite 7m 53s the patch passed -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 0s The patch has no ill-formed XML file. +1 unit 9m 40s root in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 41m 19s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844615/HADOOP-13929.001.patch Optional Tests asflicense mvnsite unit xml uname Linux 6b0efa1ba22d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c721f78 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11319/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11319/testReport/ modules C: hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11319/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jzhuge John Zhuge added a comment -

          Patch 002

          • Fix whilespaces
          Show
          jzhuge John Zhuge added a comment - Patch 002 Fix whilespaces
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 12m 30s trunk passed
          +1 mvnsite 8m 1s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvnsite 7m 58s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 unit 9m 44s root in the patch passed.
          +1 asflicense 0m 30s The patch does not generate ASF License warnings.
          39m 50s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844616/HADOOP-13929.002.patch
          Optional Tests asflicense mvnsite unit xml
          uname Linux dcfb070ab558 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c721f78
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11320/testReport/
          modules C: hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11320/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 12m 30s trunk passed +1 mvnsite 8m 1s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvnsite 7m 58s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 unit 9m 44s root in the patch passed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 39m 50s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844616/HADOOP-13929.002.patch Optional Tests asflicense mvnsite unit xml uname Linux dcfb070ab558 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c721f78 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11320/testReport/ modules C: hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11320/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Hello John Zhuge. Thank you for the patch.

          I would prefer to see this match the layout we've established in places like hadoop-azure and hadoop-aws. We could keep the contract-test-options.xml file as a placeholder, which then uses XInclude to include a credential file that is left out of source control by .gitignore. The placeholder file is a spot to leave commented out samples of how to configure it.

          Show
          cnauroth Chris Nauroth added a comment - Hello John Zhuge . Thank you for the patch. I would prefer to see this match the layout we've established in places like hadoop-azure and hadoop-aws. We could keep the contract-test-options.xml file as a placeholder, which then uses XInclude to include a credential file that is left out of source control by .gitignore. The placeholder file is a spot to leave commented out samples of how to configure it.
          Hide
          jzhuge John Zhuge added a comment -

          Sounds good Chris Nauroth. It is better to be consistent across components.

          Show
          jzhuge John Zhuge added a comment - Sounds good Chris Nauroth . It is better to be consistent across components.
          Hide
          jzhuge John Zhuge added a comment -

          Patch 003

          • Make AdlStorageContract extend AbstractBondedFSContract in order to reuse its implementation
          • Use auth-keys.xml included by adls.xml instead of contract-test-optoins.xml
          • Use non-empty property fs.contract.test.fs.adl to enable live tests. Obsolete property dfs.adl.test.contract.enable and test.fs.adl.name.
          • Git ignore contract-test-options.xml, auth-keys.xml, and azure-auth-keys.xml in the entire source tree
          Show
          jzhuge John Zhuge added a comment - Patch 003 Make AdlStorageContract extend AbstractBondedFSContract in order to reuse its implementation Use auth-keys.xml included by adls.xml instead of contract-test-optoins.xml Use non-empty property fs.contract.test.fs.adl to enable live tests. Obsolete property dfs.adl.test.contract.enable and test.fs.adl.name . Git ignore contract-test-options.xml , auth-keys.xml , and azure-auth-keys.xml in the entire source tree
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          0 mvndep 1m 53s Maven dependency ordering for branch
          +1 mvninstall 12m 28s trunk passed
          +1 compile 9m 32s trunk passed
          +1 checkstyle 1m 30s trunk passed
          +1 mvnsite 8m 8s trunk passed
          +1 mvneclipse 1m 14s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 25s trunk passed
          +1 javadoc 4m 19s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 12m 45s the patch passed
          +1 compile 9m 16s the patch passed
          +1 javac 9m 16s the patch passed
          -0 checkstyle 1m 33s root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
          +1 mvnsite 8m 22s the patch passed
          +1 mvneclipse 1m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 34s the patch passed
          +1 javadoc 4m 28s the patch passed
          -1 unit 111m 5s root in the patch failed.
          -1 asflicense 0m 35s The patch generated 2 ASF License warnings.
          212m 14s



          Reason Tests
          Failed junit tests hadoop.hdfs.web.TestWebHDFSXAttr
            hadoop.hdfs.web.TestWebHDFS
            hadoop.hdfs.web.TestWebHdfsTokens
            hadoop.hdfs.TestDFSClientRetries
            hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
            hadoop.hdfs.server.namenode.TestAuditLogs
            hadoop.yarn.server.timeline.webapp.TestTimelineWebServices
          Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845269/HADOOP-13929.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 76e349bc3551 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6938b67
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. 0 mvndep 1m 53s Maven dependency ordering for branch +1 mvninstall 12m 28s trunk passed +1 compile 9m 32s trunk passed +1 checkstyle 1m 30s trunk passed +1 mvnsite 8m 8s trunk passed +1 mvneclipse 1m 14s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 25s trunk passed +1 javadoc 4m 19s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 12m 45s the patch passed +1 compile 9m 16s the patch passed +1 javac 9m 16s the patch passed -0 checkstyle 1m 33s root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) +1 mvnsite 8m 22s the patch passed +1 mvneclipse 1m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 34s the patch passed +1 javadoc 4m 28s the patch passed -1 unit 111m 5s root in the patch failed. -1 asflicense 0m 35s The patch generated 2 ASF License warnings. 212m 14s Reason Tests Failed junit tests hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.TestDFSClientRetries   hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter   hadoop.hdfs.server.namenode.TestAuditLogs   hadoop.yarn.server.timeline.webapp.TestTimelineWebServices Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845269/HADOOP-13929.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 76e349bc3551 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6938b67 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11336/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jzhuge John Zhuge added a comment -

          Patch 004

          • Fix checkstyle
          Show
          jzhuge John Zhuge added a comment - Patch 004 Fix checkstyle
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          0 mvndep 1m 45s Maven dependency ordering for branch
          +1 mvninstall 14m 36s trunk passed
          +1 compile 10m 49s trunk passed
          +1 checkstyle 1m 41s trunk passed
          +1 mvnsite 8m 30s trunk passed
          +1 mvneclipse 1m 13s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 25s trunk passed
          +1 javadoc 4m 23s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 12m 43s the patch passed
          +1 compile 9m 15s the patch passed
          +1 javac 9m 15s the patch passed
          -0 checkstyle 1m 35s root: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 mvnsite 8m 8s the patch passed
          +1 mvneclipse 1m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 34s the patch passed
          +1 javadoc 4m 23s the patch passed
          -1 unit 102m 31s root in the patch failed.
          -1 asflicense 0m 39s The patch generated 2 ASF License warnings.
          207m 20s



          Reason Tests
          Failed junit tests hadoop.hdfs.web.TestWebHDFSXAttr
            hadoop.hdfs.web.TestWebHDFS
            hadoop.hdfs.web.TestWebHdfsTokens
            hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
            hadoop.hdfs.server.namenode.TestAuditLogs
            hadoop.yarn.server.timeline.webapp.TestTimelineWebServices



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845285/HADOOP-13929.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux afb4393becad 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6938b67
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. 0 mvndep 1m 45s Maven dependency ordering for branch +1 mvninstall 14m 36s trunk passed +1 compile 10m 49s trunk passed +1 checkstyle 1m 41s trunk passed +1 mvnsite 8m 30s trunk passed +1 mvneclipse 1m 13s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 25s trunk passed +1 javadoc 4m 23s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 12m 43s the patch passed +1 compile 9m 15s the patch passed +1 javac 9m 15s the patch passed -0 checkstyle 1m 35s root: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 mvnsite 8m 8s the patch passed +1 mvneclipse 1m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 34s the patch passed +1 javadoc 4m 23s the patch passed -1 unit 102m 31s root in the patch failed. -1 asflicense 0m 39s The patch generated 2 ASF License warnings. 207m 20s Reason Tests Failed junit tests hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter   hadoop.hdfs.server.namenode.TestAuditLogs   hadoop.yarn.server.timeline.webapp.TestTimelineWebServices Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845285/HADOOP-13929.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux afb4393becad 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6938b67 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11338/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jzhuge John Zhuge added a comment -

          TestWebHDFSXAttr, TestWebHDFS, TestWebHdfsTokens, TestWebHdfsWithRestCsrfPreventionFilter, and TestAuditLogs failures may be caused by HDFS-11280.

          TestTimelineWebServices failure is unrelated.

          Show
          jzhuge John Zhuge added a comment - TestWebHDFSXAttr, TestWebHDFS, TestWebHdfsTokens, TestWebHdfsWithRestCsrfPreventionFilter, and TestAuditLogs failures may be caused by HDFS-11280 . TestTimelineWebServices failure is unrelated.
          Hide
          jzhuge John Zhuge added a comment -

          Patch 005

          • Fix checkstyle
          Show
          jzhuge John Zhuge added a comment - Patch 005 Fix checkstyle
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 12m 25s trunk passed
          +1 compile 9m 32s trunk passed
          +1 checkstyle 1m 30s trunk passed
          +1 mvnsite 8m 9s trunk passed
          +1 mvneclipse 1m 6s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 23s trunk passed
          +1 javadoc 4m 23s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 12m 56s the patch passed
          +1 compile 9m 15s the patch passed
          +1 javac 9m 15s the patch passed
          +1 checkstyle 1m 34s the patch passed
          +1 mvnsite 8m 9s the patch passed
          +1 mvneclipse 1m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 35s the patch passed
          +1 javadoc 4m 52s the patch passed
          -1 unit 101m 28s root in the patch failed.
          -1 asflicense 0m 38s The patch generated 2 ASF License warnings.
          201m 22s



          Reason Tests
          Failed junit tests hadoop.hdfs.web.TestWebHDFSXAttr
            hadoop.hdfs.web.TestWebHDFS
            hadoop.hdfs.web.TestWebHdfsTokens
            hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
            hadoop.hdfs.server.namenode.TestAuditLogs
            hadoop.yarn.server.timeline.webapp.TestTimelineWebServices



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845311/HADOOP-13929.005.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 2f8c2f3c1410 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6938b67
          Default Java 1.8.0_111
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11339/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11339/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11339/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11339/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 12m 25s trunk passed +1 compile 9m 32s trunk passed +1 checkstyle 1m 30s trunk passed +1 mvnsite 8m 9s trunk passed +1 mvneclipse 1m 6s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 23s trunk passed +1 javadoc 4m 23s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 12m 56s the patch passed +1 compile 9m 15s the patch passed +1 javac 9m 15s the patch passed +1 checkstyle 1m 34s the patch passed +1 mvnsite 8m 9s the patch passed +1 mvneclipse 1m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 35s the patch passed +1 javadoc 4m 52s the patch passed -1 unit 101m 28s root in the patch failed. -1 asflicense 0m 38s The patch generated 2 ASF License warnings. 201m 22s Reason Tests Failed junit tests hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter   hadoop.hdfs.server.namenode.TestAuditLogs   hadoop.yarn.server.timeline.webapp.TestTimelineWebServices Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845311/HADOOP-13929.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 2f8c2f3c1410 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6938b67 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11339/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11339/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11339/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11339/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          good q. about globalness. I think for auth-keys.xml we should do that; +1 there.

          For the contract-test-options, not sure? Certainly for adl; but globally? It would help for future work, including any movement of modules around.

          Chris Nauroth —thoughts?

          Show
          stevel@apache.org Steve Loughran added a comment - good q. about globalness. I think for auth-keys.xml we should do that; +1 there. For the contract-test-options, not sure? Certainly for adl; but globally? It would help for future work, including any movement of modules around. Chris Nauroth —thoughts?
          Hide
          jzhuge John Zhuge added a comment -

          Another question: can we add another property fs.contract.test.enabled (default true to be backwards compatible)?

          It is awkward to use fs.contract.test.fs.%s being empty to decide whether live/integration tests are enabled because 1) not clear, the property essentially serves 2 purposes; 2) not easy to disable then re-enable.

          Show
          jzhuge John Zhuge added a comment - Another question: can we add another property fs.contract.test.enabled (default true to be backwards compatible)? It is awkward to use fs.contract.test.fs.%s being empty to decide whether live/integration tests are enabled because 1) not clear, the property essentially serves 2 purposes; 2) not easy to disable then re-enable.
          Hide
          jzhuge John Zhuge added a comment -

          Making a few more changes

          Show
          jzhuge John Zhuge added a comment - Making a few more changes
          Hide
          cnauroth Chris Nauroth added a comment -

          I know John is making more changes, but here is my reply to the most recent comments.

          Patch 005 deleted contract-test-options.xml, but kept it listed in .gitignore. I don't think it needs to remain in .gitignore, because there is no other file anywhere in the source tree named contract-test-options.xml, besides the ADL one that the patch deletes.

          In general though, trying to achieve some commonality across these file names and using .gitignore entries that can cover all sub-modules globally sound like a great move for future-proofing this.

          Another question: can we add another property fs.contract.test.enabled (default true to be backwards compatible)?

          I'm not sure I completely understand, but does the lack of fs in this new property mean that it would be global, not tied to a specific file system's tests? If it's global, then a weakness of this approach is that for developers running mvn verify at the root of the source tree or at the root of hadoop-cloud-storage-project, it would be all or nothing. If a developer had credentials to AWS and Azure but not OpenStack, then they'd need to do something different to run just for the modules where they have credentials.

          Show
          cnauroth Chris Nauroth added a comment - I know John is making more changes, but here is my reply to the most recent comments. Patch 005 deleted contract-test-options.xml, but kept it listed in .gitignore. I don't think it needs to remain in .gitignore, because there is no other file anywhere in the source tree named contract-test-options.xml, besides the ADL one that the patch deletes. In general though, trying to achieve some commonality across these file names and using .gitignore entries that can cover all sub-modules globally sound like a great move for future-proofing this. Another question: can we add another property fs.contract.test.enabled (default true to be backwards compatible)? I'm not sure I completely understand, but does the lack of fs in this new property mean that it would be global, not tied to a specific file system's tests? If it's global, then a weakness of this approach is that for developers running mvn verify at the root of the source tree or at the root of hadoop-cloud-storage-project, it would be all or nothing. If a developer had credentials to AWS and Azure but not OpenStack, then they'd need to do something different to run just for the modules where they have credentials.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I understand the enable/disable thing. Essentially we have an implicit enable/disable flag: if the path is there, they run.

          regarding a global flag, no, -1 to that. I often run the AWS s3? tests with s3a defined, s3n and s3 undefined. That way: less tests, faster failures.

          Per filesystem? Possibly

          Show
          stevel@apache.org Steve Loughran added a comment - I understand the enable/disable thing. Essentially we have an implicit enable/disable flag: if the path is there, they run. regarding a global flag, no, -1 to that. I often run the AWS s3? tests with s3a defined, s3n and s3 undefined. That way: less tests, faster failures. Per filesystem? Possibly
          Hide
          jzhuge John Zhuge added a comment -

          Thanks both of you for the input:

          • No go on fs.contract.test.enabled
          • Globally git ignore auth-keys.xml and azure-auth-keys.xml
          Show
          jzhuge John Zhuge added a comment - Thanks both of you for the input: No go on fs.contract.test.enabled Globally git ignore auth-keys.xml and azure-auth-keys.xml
          Hide
          jzhuge John Zhuge added a comment -

          Patch 006

          • Only globally git ignore auth-keys.xml and azure-auth-keys.xml
          • Enhance AdlStorageConfiguration#createStorageConnector

          Testing

          • Live unit tests
          Show
          jzhuge John Zhuge added a comment - Patch 006 Only globally git ignore auth-keys.xml and azure-auth-keys.xml Enhance AdlStorageConfiguration#createStorageConnector Testing Live unit tests
          Hide
          jzhuge John Zhuge added a comment -

          Patch 007

          • Add ADLS properties to core-default.xml
          • Update index.md

          Testing

          • Live unit tests
          Show
          jzhuge John Zhuge added a comment - Patch 007 Add ADLS properties to core-default.xml Update index.md Testing Live unit tests
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          0 mvndep 0m 20s Maven dependency ordering for branch
          +1 mvninstall 13m 5s trunk passed
          +1 compile 9m 56s trunk passed
          +1 checkstyle 1m 34s trunk passed
          +1 mvnsite 9m 5s trunk passed
          +1 mvneclipse 1m 11s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 1m 52s trunk passed
          +1 javadoc 4m 35s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 13m 59s the patch passed
          +1 compile 9m 57s the patch passed
          +1 javac 9m 57s the patch passed
          +1 checkstyle 1m 37s the patch passed
          +1 mvnsite 9m 7s the patch passed
          +1 mvneclipse 1m 13s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 2m 11s the patch passed
          +1 javadoc 4m 41s the patch passed
          -1 unit 14m 26s root in the patch failed.
          -1 asflicense 0m 35s The patch generated 2 ASF License warnings.
          122m 31s



          Reason Tests
          Failed junit tests hadoop.conf.TestCommonConfigurationFields



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846201/HADOOP-13929.007.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux bcadb13f057a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ac16400
          Default Java 1.8.0_111
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. 0 mvndep 0m 20s Maven dependency ordering for branch +1 mvninstall 13m 5s trunk passed +1 compile 9m 56s trunk passed +1 checkstyle 1m 34s trunk passed +1 mvnsite 9m 5s trunk passed +1 mvneclipse 1m 11s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 1m 52s trunk passed +1 javadoc 4m 35s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 13m 59s the patch passed +1 compile 9m 57s the patch passed +1 javac 9m 57s the patch passed +1 checkstyle 1m 37s the patch passed +1 mvnsite 9m 7s the patch passed +1 mvneclipse 1m 13s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 2m 11s the patch passed +1 javadoc 4m 41s the patch passed -1 unit 14m 26s root in the patch failed. -1 asflicense 0m 35s The patch generated 2 ASF License warnings. 122m 31s Reason Tests Failed junit tests hadoop.conf.TestCommonConfigurationFields Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846201/HADOOP-13929.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux bcadb13f057a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ac16400 Default Java 1.8.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11392/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jzhuge John Zhuge added a comment -

          Patch 008

          • Fix TestCommonConfigurationFields failure
          • Fix checkstyle
          Show
          jzhuge John Zhuge added a comment - Patch 008 Fix TestCommonConfigurationFields failure Fix checkstyle
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 0m 43s Maven dependency ordering for branch
          +1 mvninstall 12m 56s trunk passed
          +1 compile 9m 58s trunk passed
          +1 checkstyle 1m 36s trunk passed
          +1 mvnsite 9m 5s trunk passed
          +1 mvneclipse 1m 9s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 1m 51s trunk passed
          +1 javadoc 4m 37s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 14m 4s the patch passed
          +1 compile 9m 38s the patch passed
          +1 javac 9m 38s the patch passed
          +1 checkstyle 1m 35s the patch passed
          +1 mvnsite 9m 6s the patch passed
          +1 mvneclipse 1m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 2m 12s the patch passed
          +1 javadoc 4m 39s the patch passed
          -1 unit 108m 34s root in the patch failed.
          -1 asflicense 0m 38s The patch generated 2 ASF License warnings.
          216m 54s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.balancer.TestBalancer
            hadoop.yarn.server.timeline.webapp.TestTimelineWebServices
          Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846204/HADOOP-13929.008.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 1e97669bf727 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ac16400
          Default Java 1.8.0_111
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11393/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11393/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11393/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11393/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 0m 43s Maven dependency ordering for branch +1 mvninstall 12m 56s trunk passed +1 compile 9m 58s trunk passed +1 checkstyle 1m 36s trunk passed +1 mvnsite 9m 5s trunk passed +1 mvneclipse 1m 9s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 1m 51s trunk passed +1 javadoc 4m 37s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 14m 4s the patch passed +1 compile 9m 38s the patch passed +1 javac 9m 38s the patch passed +1 checkstyle 1m 35s the patch passed +1 mvnsite 9m 6s the patch passed +1 mvneclipse 1m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 2m 12s the patch passed +1 javadoc 4m 39s the patch passed -1 unit 108m 34s root in the patch failed. -1 asflicense 0m 38s The patch generated 2 ASF License warnings. 216m 54s Reason Tests Failed junit tests hadoop.hdfs.server.balancer.TestBalancer   hadoop.yarn.server.timeline.webapp.TestTimelineWebServices Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846204/HADOOP-13929.008.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 1e97669bf727 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ac16400 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11393/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11393/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11393/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11393/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jzhuge John Zhuge added a comment -

          TestTimelineWebServices.testPrimaryFilterNumericString failure is tracked by YARN-5934.
          TestBalancer and TestDataNodeVolumeFailure pass locally.

          Show
          jzhuge John Zhuge added a comment - TestTimelineWebServices.testPrimaryFilterNumericString failure is tracked by YARN-5934 . TestBalancer and TestDataNodeVolumeFailure pass locally.
          Hide
          jzhuge John Zhuge added a comment -

          Vishwajeet Dusane or Atul Sikaria, Could you please take a look at the change?

          Show
          jzhuge John Zhuge added a comment - Vishwajeet Dusane or Atul Sikaria , Could you please take a look at the change?
          Hide
          cnauroth Chris Nauroth added a comment -

          John Zhuge, there are a lot of nice clean-ups here. Thank you! Unfortunately, I've lost my configurations for live testing against ADL, so I'm going to need to recreate that. Meanwhile, Vishwajeet Dusane and Atul Sikaria, if you get a test run completed with this patch before me, please comment and let me know.

          Show
          cnauroth Chris Nauroth added a comment - John Zhuge , there are a lot of nice clean-ups here. Thank you! Unfortunately, I've lost my configurations for live testing against ADL, so I'm going to need to recreate that. Meanwhile, Vishwajeet Dusane and Atul Sikaria , if you get a test run completed with this patch before me, please comment and let me know.
          Hide
          vishwajeet.dusane Vishwajeet Dusane added a comment -

          John Zhuge and Chris Nauroth - Thanks a lot for the clean up, i have not got a chance to go through the patch yet. I am on vacation and would be back on 16th Jan.

          CC: Engaging folks, who can also help Shradha Revankar, Shrikant Naidu, Atul Sikaria and Chris Douglas

          Show
          vishwajeet.dusane Vishwajeet Dusane added a comment - John Zhuge and Chris Nauroth - Thanks a lot for the clean up, i have not got a chance to go through the patch yet. I am on vacation and would be back on 16th Jan. CC: Engaging folks, who can also help Shradha Revankar , Shrikant Naidu , Atul Sikaria and Chris Douglas
          Hide
          jzhuge John Zhuge added a comment -

          The patches suffered feature creep. Let me think about how to proceed.

          Show
          jzhuge John Zhuge added a comment - The patches suffered feature creep. Let me think about how to proceed.
          Hide
          jzhuge John Zhuge added a comment -

          Patch 009

          • Remove unrelated changes from Patch 008

          Testing done

          • Passed unit tests
          • Passed live unit tests except the known TestAdlContractRootDirLive
            testRmNonEmptyRootDirNonRecursive failure in HADOOP-13927
          • Passed TestCommonConfigurationFields
          Show
          jzhuge John Zhuge added a comment - Patch 009 Remove unrelated changes from Patch 008 Testing done Passed unit tests Passed live unit tests except the known TestAdlContractRootDirLive testRmNonEmptyRootDirNonRecursive failure in HADOOP-13927 Passed TestCommonConfigurationFields
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 2m 9s Maven dependency ordering for branch
          +1 mvninstall 15m 43s trunk passed
          +1 compile 15m 38s trunk passed
          +1 checkstyle 1m 52s trunk passed
          +1 mvnsite 10m 58s trunk passed
          +1 mvneclipse 1m 29s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 2m 16s trunk passed
          +1 javadoc 6m 10s trunk passed
          0 mvndep 0m 21s Maven dependency ordering for patch
          +1 mvninstall 17m 6s the patch passed
          +1 compile 10m 47s the patch passed
          +1 javac 10m 47s the patch passed
          +1 checkstyle 1m 34s the patch passed
          +1 mvnsite 8m 50s the patch passed
          +1 mvneclipse 1m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 2m 9s the patch passed
          +1 javadoc 4m 45s the patch passed
          -1 unit 146m 53s root in the patch failed.
          -1 asflicense 0m 56s The patch generated 2 ASF License warnings.
          274m 41s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestGetFileChecksum
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.yarn.server.timeline.webapp.TestTimelineWebServices



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848996/HADOOP-13929.009.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 390623bd1d1c 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ce83205
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11499/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11499/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11499/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11499/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 2m 9s Maven dependency ordering for branch +1 mvninstall 15m 43s trunk passed +1 compile 15m 38s trunk passed +1 checkstyle 1m 52s trunk passed +1 mvnsite 10m 58s trunk passed +1 mvneclipse 1m 29s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 2m 16s trunk passed +1 javadoc 6m 10s trunk passed 0 mvndep 0m 21s Maven dependency ordering for patch +1 mvninstall 17m 6s the patch passed +1 compile 10m 47s the patch passed +1 javac 10m 47s the patch passed +1 checkstyle 1m 34s the patch passed +1 mvnsite 8m 50s the patch passed +1 mvneclipse 1m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 2m 9s the patch passed +1 javadoc 4m 45s the patch passed -1 unit 146m 53s root in the patch failed. -1 asflicense 0m 56s The patch generated 2 ASF License warnings. 274m 41s Reason Tests Failed junit tests hadoop.hdfs.TestGetFileChecksum   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.yarn.server.timeline.webapp.TestTimelineWebServices Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848996/HADOOP-13929.009.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 390623bd1d1c 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ce83205 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11499/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11499/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11499/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11499/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jzhuge John Zhuge added a comment -

          TestGetFileChecksum,TestDataNodeVolumeFailure,TestRollingUpgrade,TestDataNodeVolumeFailureReporting passed locally.

          TestTimelineWebServices failure is tracked by YARN-5934.

          Show
          jzhuge John Zhuge added a comment - TestGetFileChecksum,TestDataNodeVolumeFailure,TestRollingUpgrade,TestDataNodeVolumeFailureReporting passed locally. TestTimelineWebServices failure is tracked by YARN-5934 .
          Hide
          vishwajeet.dusane Vishwajeet Dusane added a comment -

          Thanks you John Zhuge for trimming to patch 9. It is much easier to review. I ran adl live test with Patch 9 and all test passed. +1 on the overall Patch 9.

          Regarding semantics to follow across file system,

          • As per comments, to use test.fs.adl.name similar to test.fs.s3n.name and test.fs.s3a.name.

          Would leave it to Chris Nauroth and Steve Loughran to comment on the guideline.

          Show
          vishwajeet.dusane Vishwajeet Dusane added a comment - Thanks you John Zhuge for trimming to patch 9. It is much easier to review. I ran adl live test with Patch 9 and all test passed. +1 on the overall Patch 9. Regarding semantics to follow across file system, As per comments , to use test.fs.adl.name similar to test.fs.s3n.name and test.fs.s3a.name . Would leave it to Chris Nauroth and Steve Loughran to comment on the guideline.
          Hide
          jzhuge John Zhuge added a comment - - edited

          From hadoop-aws/index.md, test.fs.s3a.name is for integration tests, while fs.contract.test.fs.s3a is for filesystem contract tests. Should ADLS also use 2 properties to differentiate these 2 types of tests in live directory?

          BTW, is hadoop-aws/index.md accurate about the use of contract-test-options.xml?

          Show
          jzhuge John Zhuge added a comment - - edited From hadoop-aws/index.md, test.fs.s3a.name is for integration tests, while fs.contract.test.fs.s3a is for filesystem contract tests. Should ADLS also use 2 properties to differentiate these 2 types of tests in live directory? BTW, is hadoop-aws/index.md accurate about the use of contract-test-options.xml ?
          Hide
          jzhuge John Zhuge added a comment -

          Leave the decision to differentiate the integration tests and contract tests to a follow-up JIRA, thus can further simplify this patch.

          Show
          jzhuge John Zhuge added a comment - Leave the decision to differentiate the integration tests and contract tests to a follow-up JIRA, thus can further simplify this patch.
          Hide
          vishwajeet.dusane Vishwajeet Dusane added a comment -

          John Zhuge - I agree with you to leave the decision to differentiate integration test and contract test.

          Show
          vishwajeet.dusane Vishwajeet Dusane added a comment - John Zhuge - I agree with you to leave the decision to differentiate integration test and contract test.
          Hide
          jzhuge John Zhuge added a comment -

          Patch 010

          • Keep property test.fs.adl.name and dfs.adl.test.contract.enable thus make the patch simpler than patch 009

          Testing done

          • Passed unit tests
          • Passed live unit tests
          • Passed TestCommonConfigurationFields
          Show
          jzhuge John Zhuge added a comment - Patch 010 Keep property test.fs.adl.name and dfs.adl.test.contract.enable thus make the patch simpler than patch 009 Testing done Passed unit tests Passed live unit tests Passed TestCommonConfigurationFields
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 42s Maven dependency ordering for branch
          +1 mvninstall 12m 36s trunk passed
          +1 compile 12m 46s trunk passed
          +1 checkstyle 1m 31s trunk passed
          +1 mvnsite 8m 45s trunk passed
          +1 mvneclipse 1m 5s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 25s trunk passed
          +1 javadoc 4m 28s trunk passed
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 12m 48s the patch passed
          +1 compile 10m 46s the patch passed
          +1 javac 10m 46s the patch passed
          -0 checkstyle 1m 33s root: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 mvnsite 8m 45s the patch passed
          +1 mvneclipse 1m 7s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 34s the patch passed
          +1 javadoc 4m 29s the patch passed
          -1 unit 15m 54s root in the patch failed.
          -1 asflicense 0m 36s The patch generated 2 ASF License warnings.
          121m 57s



          Reason Tests
          Failed junit tests hadoop.security.TestKDiag



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849909/HADOOP-13929.010.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 3b5464749fcc 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 312b36d
          Default Java 1.8.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 42s Maven dependency ordering for branch +1 mvninstall 12m 36s trunk passed +1 compile 12m 46s trunk passed +1 checkstyle 1m 31s trunk passed +1 mvnsite 8m 45s trunk passed +1 mvneclipse 1m 5s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 25s trunk passed +1 javadoc 4m 28s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 12m 48s the patch passed +1 compile 10m 46s the patch passed +1 javac 10m 46s the patch passed -0 checkstyle 1m 33s root: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 mvnsite 8m 45s the patch passed +1 mvneclipse 1m 7s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 34s the patch passed +1 javadoc 4m 29s the patch passed -1 unit 15m 54s root in the patch failed. -1 asflicense 0m 36s The patch generated 2 ASF License warnings. 121m 57s Reason Tests Failed junit tests hadoop.security.TestKDiag Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849909/HADOOP-13929.010.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 3b5464749fcc 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 312b36d Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11528/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jzhuge John Zhuge added a comment -

          Patch 011

          • Fix checkstyle
          Show
          jzhuge John Zhuge added a comment - Patch 011 Fix checkstyle
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 12m 28s trunk passed
          +1 compile 14m 4s trunk passed
          +1 checkstyle 1m 40s trunk passed
          +1 mvnsite 9m 46s trunk passed
          +1 mvneclipse 1m 17s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 24s trunk passed
          +1 javadoc 4m 55s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 15m 2s the patch passed
          +1 compile 13m 25s the patch passed
          +1 javac 13m 25s the patch passed
          +1 checkstyle 1m 50s the patch passed
          +1 mvnsite 8m 46s the patch passed
          +1 mvneclipse 1m 7s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: .
          +1 findbugs 0m 36s the patch passed
          +1 javadoc 4m 32s the patch passed
          -1 unit 17m 12s root in the patch failed.
          -1 asflicense 0m 37s The patch generated 2 ASF License warnings.
          131m 0s



          Reason Tests
          Failed junit tests hadoop.fs.sftp.TestSFTPFileSystem
            hadoop.fs.viewfs.TestViewFsTrash
            hadoop.security.TestKDiag



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13929
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849973/HADOOP-13929.011.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux ac33b04e202a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 312b36d
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11529/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11529/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11529/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-tools/hadoop-azure-datalake . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11529/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 12m 28s trunk passed +1 compile 14m 4s trunk passed +1 checkstyle 1m 40s trunk passed +1 mvnsite 9m 46s trunk passed +1 mvneclipse 1m 17s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 24s trunk passed +1 javadoc 4m 55s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 15m 2s the patch passed +1 compile 13m 25s the patch passed +1 javac 13m 25s the patch passed +1 checkstyle 1m 50s the patch passed +1 mvnsite 8m 46s the patch passed +1 mvneclipse 1m 7s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: . +1 findbugs 0m 36s the patch passed +1 javadoc 4m 32s the patch passed -1 unit 17m 12s root in the patch failed. -1 asflicense 0m 37s The patch generated 2 ASF License warnings. 131m 0s Reason Tests Failed junit tests hadoop.fs.sftp.TestSFTPFileSystem   hadoop.fs.viewfs.TestViewFsTrash   hadoop.security.TestKDiag Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13929 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849973/HADOOP-13929.011.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux ac33b04e202a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 312b36d Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11529/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11529/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/11529/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-tools/hadoop-azure-datalake . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11529/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jzhuge John Zhuge added a comment -

          Unit test failures are unrelated.

          Show
          jzhuge John Zhuge added a comment - Unit test failures are unrelated.
          Hide
          vishwajeet.dusane Vishwajeet Dusane added a comment -

          Thanks John Zhuge for the updated Patch11.

          Minor comment on azure-auth-keys.xml in .gitignore seems to be unused, do you consider it removing from the patch? Otherwise +1 on the overall patch.

          Show
          vishwajeet.dusane Vishwajeet Dusane added a comment - Thanks John Zhuge for the updated Patch11. Minor comment on azure-auth-keys.xml in .gitignore seems to be unused, do you consider it removing from the patch? Otherwise +1 on the overall patch.
          Hide
          jzhuge John Zhuge added a comment -

          azure-auth-keys.xml is used by wasb.xml and azure-test.xml, also documented by index.md for hadoop-azure.

          Show
          jzhuge John Zhuge added a comment - azure-auth-keys.xml is used by wasb.xml and azure-test.xml , also documented by index.md for hadoop-azure.
          Hide
          vishwajeet.dusane Vishwajeet Dusane added a comment -

          Thanks John Zhuge, We should have specific key name for data lake as azure-data-lake-auth-keys.xml than generic auth-keys.xml. It would be best to create separate dependent task on this JIRA than drag this thread further more?

          Show
          vishwajeet.dusane Vishwajeet Dusane added a comment - Thanks John Zhuge , We should have specific key name for data lake as azure-data-lake-auth-keys.xml than generic auth-keys.xml . It would be best to create separate dependent task on this JIRA than drag this thread further more?
          Hide
          jzhuge John Zhuge added a comment -

          Is there any use case that calls for different 'auth-keys.xml' file names? After all, the files reside in their separate component src/test/resources directories.

          Even if it is shared by multiple components, multiple credentials can be placed into it because property names are different.

          Show
          jzhuge John Zhuge added a comment - Is there any use case that calls for different 'auth-keys.xml' file names? After all, the files reside in their separate component src/test/resources directories. Even if it is shared by multiple components, multiple credentials can be placed into it because property names are different.
          Hide
          vishwajeet.dusane Vishwajeet Dusane added a comment -

          Thanks John Zhuge for clarification.

          Show
          vishwajeet.dusane Vishwajeet Dusane added a comment - Thanks John Zhuge for clarification.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Hi, John Zhuge

          Should we also put contract-test-options.xml into .gitignore?

          Show
          eddyxu Lei (Eddy) Xu added a comment - Hi, John Zhuge Should we also put contract-test-options.xml into .gitignore ?
          Hide
          jzhuge John Zhuge added a comment -

          Chris Nauroth recommended to remove it from .gitignore:

          Patch 005 deleted contract-test-options.xml, but kept it listed in .gitignore. I don't think it needs to remain in .gitignore, because there is no other file anywhere in the source tree named contract-test-options.xml, besides the ADL one that the patch deletes.

          Show
          jzhuge John Zhuge added a comment - Chris Nauroth recommended to remove it from .gitignore : Patch 005 deleted contract-test-options.xml, but kept it listed in .gitignore. I don't think it needs to remain in .gitignore, because there is no other file anywhere in the source tree named contract-test-options.xml, besides the ADL one that the patch deletes.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Ok, thanks for the explanation, John Zhuge. It makes sense, thus +1.

          Committed to trunk.

          Thanks for the patch, John Zhuge, and thanks for the reviews, Chris Nauroth!

          Show
          eddyxu Lei (Eddy) Xu added a comment - Ok, thanks for the explanation, John Zhuge . It makes sense, thus +1. Committed to trunk. Thanks for the patch, John Zhuge , and thanks for the reviews, Chris Nauroth !
          Hide
          jzhuge John Zhuge added a comment -

          Thanks Lei (Eddy) Xu for the review and commit! Appreciate the review from Steve Loughran, Chris Nauroth, and Vishwajeet Dusane.

          Show
          jzhuge John Zhuge added a comment - Thanks Lei (Eddy) Xu for the review and commit! Appreciate the review from Steve Loughran , Chris Nauroth , and Vishwajeet Dusane .
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Jenkins build Hadoop-trunk-Commit #11241 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11241/)
          HADOOP-13929. ADLS connector should not check in (lei: rev 71c23c9fc94cfdf58de80effbc3f51c0925d0cfe)

          • (edit) hadoop-tools/hadoop-azure-datalake/src/main/java/org/apache/hadoop/fs/adl/AdlFileSystem.java
          • (delete) hadoop-tools/hadoop-azure-datalake/src/test/resources/contract-test-options.xml
          • (edit) .gitignore
          • (edit) hadoop-tools/hadoop-azure-datalake/src/site/markdown/index.md
          • (edit) hadoop-tools/hadoop-azure-datalake/src/test/java/org/apache/hadoop/fs/adl/live/AdlStorageConfiguration.java
          • (edit) hadoop-tools/hadoop-azure-datalake/src/test/resources/adls.xml
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Jenkins build Hadoop-trunk-Commit #11241 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11241/ ) HADOOP-13929 . ADLS connector should not check in (lei: rev 71c23c9fc94cfdf58de80effbc3f51c0925d0cfe) (edit) hadoop-tools/hadoop-azure-datalake/src/main/java/org/apache/hadoop/fs/adl/AdlFileSystem.java (delete) hadoop-tools/hadoop-azure-datalake/src/test/resources/contract-test-options.xml (edit) .gitignore (edit) hadoop-tools/hadoop-azure-datalake/src/site/markdown/index.md (edit) hadoop-tools/hadoop-azure-datalake/src/test/java/org/apache/hadoop/fs/adl/live/AdlStorageConfiguration.java (edit) hadoop-tools/hadoop-azure-datalake/src/test/resources/adls.xml
          Hide
          vishwajeet.dusane Vishwajeet Dusane added a comment -

          Thanks John Zhuge - This is a good optimization. And Lei (Eddy) Xu thanks for the commit. Thanks to Steve Loughran and Chris Nauroth to ensure consistency across.

          Show
          vishwajeet.dusane Vishwajeet Dusane added a comment - Thanks John Zhuge - This is a good optimization. And Lei (Eddy) Xu thanks for the commit. Thanks to Steve Loughran and Chris Nauroth to ensure consistency across.

            People

            • Assignee:
              jzhuge John Zhuge
              Reporter:
              jzhuge John Zhuge
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development