Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12994

Specify PositionedReadable, add contract tests, fix problems

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Some work on S3a has shown up that there aren't tests catching regressions in readFully, reviewing the documentation shows that its specification could be improved.

      1. review the spec
      2. review the implementations
      3. add tests (proposed: to the seek contract; streams which support seek should support positioned readable)
      4. fix code, where it differs significantly from HDFS or LocalFS
      1. HADOOP-12994-001.patch
        62 kB
        Steve Loughran
      2. HADOOP-12994-002.patch
        63 kB
        Steve Loughran
      3. HADOOP-12994-003.patch
        68 kB
        Steve Loughran
      4. HADOOP-12994-004.patch
        68 kB
        Steve Loughran
      5. HADOOP-12994-005.patch
        68 kB
        Steve Loughran
      6. HADOOP-12994-006.patch
        68 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          stevel@apache.org Steve Loughran added a comment -

          new tests show that HDFS client doesn't validate is arguments completely; while still failing, it fails in "odd" ways

          Show
          stevel@apache.org Steve Loughran added a comment - new tests show that HDFS client doesn't validate is arguments completely; while still failing, it fails in "odd" ways
          Hide
          stevel@apache.org Steve Loughran added a comment -

          One issue here: currently DfsInputStream's internal pread method returns -1 on a negative position, which is then uprated to an EOFException in read(). Other implementations view a negative offset as an illegal argument.

          I'm going to make EOFException the default, as it is consistent with a negative seek(). Implementations can fail-fast on this state

          Show
          stevel@apache.org Steve Loughran added a comment - One issue here: currently DfsInputStream's internal pread method returns -1 on a negative position, which is then uprated to an EOFException in read(). Other implementations view a negative offset as an illegal argument. I'm going to make EOFException the default, as it is consistent with a negative seek(). Implementations can fail-fast on this state
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 001, including

          1. fs specification doc to define readFully properly
          2. tests in Seek contract test (the operations go with seek; the default implementation uses it)
          3. argument validation across hadoop-common filesystems
          4. argument validation for DFS client
          5. s3a to downgrade an EOF exception in a read() to a -1. That is what HDFS does, and allows PositionedReadable.read() to work

          I also unrolled s3a's readFully to avoid so many expensive seek calls; we may want to think about pulling that into a separate patch as its an optimisation and not needed.

          tested on: local, hdfs, s3a against amazon ireland.
          not tested on: swift, azure

          Show
          stevel@apache.org Steve Loughran added a comment - patch 001, including fs specification doc to define readFully properly tests in Seek contract test (the operations go with seek; the default implementation uses it) argument validation across hadoop-common filesystems argument validation for DFS client s3a to downgrade an EOF exception in a read() to a -1. That is what HDFS does, and allows PositionedReadable.read() to work I also unrolled s3a's readFully to avoid so many expensive seek calls; we may want to think about pulling that into a separate patch as its an optimisation and not needed. tested on: local, hdfs, s3a against amazon ireland. not tested on: swift, azure
          Hide
          stevel@apache.org Steve Loughran added a comment -

          ..oh, and took the opportunity to clean up some bits of the tests and add javadocs for PositionedReadable.

          Show
          stevel@apache.org Steve Loughran added a comment - ..oh, and took the opportunity to clean up some bits of the tests and add javadocs for PositionedReadable.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Chris Nauroth Andrew Wangjay vyas —any of you feeling like reviewing a bit more of the FS specification. Hardening all implementations, getting them to throw the same EOF on -ve positions that DFSClient would do, then adding the tests.

          Show
          stevel@apache.org Steve Loughran added a comment - Chris Nauroth Andrew Wang jay vyas —any of you feeling like reviewing a bit more of the FS specification. Hardening all implementations, getting them to throw the same EOF on -ve positions that DFSClient would do, then adding the tests.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 12 new or modified test files.
          0 mvndep 0m 13s Maven dependency ordering for branch
          +1 mvninstall 6m 25s trunk passed
          +1 compile 5m 35s trunk passed with JDK v1.8.0_77
          +1 compile 6m 32s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 6s trunk passed
          +1 mvnsite 1m 47s trunk passed
          +1 mvneclipse 0m 40s trunk passed
          +1 findbugs 3m 40s trunk passed
          +1 javadoc 1m 24s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 42s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 25s the patch passed
          +1 compile 5m 32s the patch passed with JDK v1.8.0_77
          +1 javac 5m 32s the patch passed
          +1 compile 6m 32s the patch passed with JDK v1.7.0_95
          +1 javac 6m 32s the patch passed
          -1 checkstyle 1m 6s root: patch generated 4 new + 263 unchanged - 10 fixed = 267 total (was 273)
          +1 mvnsite 1m 48s the patch passed
          +1 mvneclipse 0m 40s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 findbugs 1m 49s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          -1 javadoc 2m 24s hadoop-common-project_hadoop-common-jdk1.8.0_77 with JDK v1.8.0_77 generated 17 new + 1 unchanged - 0 fixed = 18 total (was 1)
          +1 javadoc 1m 24s the patch passed with JDK v1.8.0_77
          -1 javadoc 4m 27s hadoop-common-project_hadoop-common-jdk1.7.0_95 with JDK v1.7.0_95 generated 17 new + 13 unchanged - 0 fixed = 30 total (was 13)
          +1 javadoc 1m 42s the patch passed with JDK v1.7.0_95
          -1 unit 16m 48s hadoop-common in the patch failed with JDK v1.8.0_77.
          +1 unit 0m 49s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_77.
          -1 unit 7m 23s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 0m 54s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          82m 13s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            Inconsistent synchronization of org.apache.hadoop.fs.HarFileSystem$HarFSDataInputStream$HarFsInputStream.position; locked 91% of time Unsynchronized access at HarFileSystem.java:91% of time Unsynchronized access at HarFileSystem.java:[line 1057]
          JDK v1.8.0_77 Failed junit tests hadoop.crypto.TestCryptoStreamsWithJceAesCtrCryptoCodec
            hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec
            hadoop.crypto.TestCryptoStreamsForLocalFS
            hadoop.crypto.TestCryptoStreams
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle
          JDK v1.7.0_95 Failed junit tests hadoop.crypto.TestCryptoStreamsWithJceAesCtrCryptoCodec
            hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec
            hadoop.crypto.TestCryptoStreamsForLocalFS
            hadoop.crypto.TestCryptoStreams



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12796756/HADOOP-12994-001.patch
          JIRA Issue HADOOP-12994
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 48021d09a85b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0ecdd4c
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/diff-checkstyle-root.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          javadoc hadoop-common-project_hadoop-common-jdk1.8.0_77: https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          javadoc hadoop-common-project_hadoop-common-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 12 new or modified test files. 0 mvndep 0m 13s Maven dependency ordering for branch +1 mvninstall 6m 25s trunk passed +1 compile 5m 35s trunk passed with JDK v1.8.0_77 +1 compile 6m 32s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 6s trunk passed +1 mvnsite 1m 47s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 3m 40s trunk passed +1 javadoc 1m 24s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 42s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 25s the patch passed +1 compile 5m 32s the patch passed with JDK v1.8.0_77 +1 javac 5m 32s the patch passed +1 compile 6m 32s the patch passed with JDK v1.7.0_95 +1 javac 6m 32s the patch passed -1 checkstyle 1m 6s root: patch generated 4 new + 263 unchanged - 10 fixed = 267 total (was 273) +1 mvnsite 1m 48s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 1m 49s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) -1 javadoc 2m 24s hadoop-common-project_hadoop-common-jdk1.8.0_77 with JDK v1.8.0_77 generated 17 new + 1 unchanged - 0 fixed = 18 total (was 1) +1 javadoc 1m 24s the patch passed with JDK v1.8.0_77 -1 javadoc 4m 27s hadoop-common-project_hadoop-common-jdk1.7.0_95 with JDK v1.7.0_95 generated 17 new + 13 unchanged - 0 fixed = 30 total (was 13) +1 javadoc 1m 42s the patch passed with JDK v1.7.0_95 -1 unit 16m 48s hadoop-common in the patch failed with JDK v1.8.0_77. +1 unit 0m 49s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_77. -1 unit 7m 23s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 0m 54s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 82m 13s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   Inconsistent synchronization of org.apache.hadoop.fs.HarFileSystem$HarFSDataInputStream$HarFsInputStream.position; locked 91% of time Unsynchronized access at HarFileSystem.java:91% of time Unsynchronized access at HarFileSystem.java: [line 1057] JDK v1.8.0_77 Failed junit tests hadoop.crypto.TestCryptoStreamsWithJceAesCtrCryptoCodec   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec   hadoop.crypto.TestCryptoStreamsForLocalFS   hadoop.crypto.TestCryptoStreams JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle JDK v1.7.0_95 Failed junit tests hadoop.crypto.TestCryptoStreamsWithJceAesCtrCryptoCodec   hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec   hadoop.crypto.TestCryptoStreamsForLocalFS   hadoop.crypto.TestCryptoStreams Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12796756/HADOOP-12994-001.patch JIRA Issue HADOOP-12994 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 48021d09a85b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0ecdd4c Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html javadoc hadoop-common-project_hadoop-common-jdk1.8.0_77: https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt javadoc hadoop-common-project_hadoop-common-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9013/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          warnings & javadocs minor; test failure in Test crypto streams, as -ve position exception is now EOFException —the same as HDFS. Test was looking for IllegalArgumentException.

          Show
          stevel@apache.org Steve Loughran added a comment - warnings & javadocs minor; test failure in Test crypto streams, as -ve position exception is now EOFException —the same as HDFS. Test was looking for IllegalArgumentException.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 002. Addresses findbugs, changes tests in CryptoTestBase to look for EOF exceptions for seeks outside limits of file

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 002. Addresses findbugs, changes tests in CryptoTestBase to look for EOF exceptions for seeks outside limits of file
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 13 new or modified test files.
          0 mvndep 1m 11s Maven dependency ordering for branch
          +1 mvninstall 8m 2s trunk passed
          +1 compile 9m 0s trunk passed with JDK v1.8.0_74
          +1 compile 8m 56s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 20s trunk passed
          +1 mvnsite 2m 19s trunk passed
          +1 mvneclipse 0m 50s trunk passed
          +1 findbugs 4m 40s trunk passed
          +1 javadoc 2m 17s trunk passed with JDK v1.8.0_74
          +1 javadoc 2m 19s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 20s Maven dependency ordering for patch
          +1 mvninstall 1m 59s the patch passed
          +1 compile 11m 17s the patch passed with JDK v1.8.0_74
          +1 javac 11m 17s the patch passed
          +1 compile 9m 39s the patch passed with JDK v1.7.0_95
          +1 javac 9m 39s the patch passed
          +1 checkstyle 1m 24s root: patch generated 0 new + 263 unchanged - 10 fixed = 263 total (was 273)
          +1 mvnsite 2m 26s the patch passed
          +1 mvneclipse 0m 56s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 5m 41s the patch passed
          +1 javadoc 2m 18s the patch passed with JDK v1.8.0_74
          +1 javadoc 2m 17s the patch passed with JDK v1.7.0_95
          -1 unit 11m 17s hadoop-common in the patch failed with JDK v1.8.0_74.
          +1 unit 1m 21s hadoop-hdfs-client in the patch passed with JDK v1.8.0_74.
          +1 unit 0m 20s hadoop-aws in the patch passed with JDK v1.8.0_74.
          +1 unit 11m 1s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 1m 18s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 17s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 31s Patch does not generate ASF License warnings.
          107m 38s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.ipc.TestIPC



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12796838/HADOOP-12994-002.patch
          JIRA Issue HADOOP-12994
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f2aeeeb03c10 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 5092c94
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9020/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9020/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9020/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9020/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 13 new or modified test files. 0 mvndep 1m 11s Maven dependency ordering for branch +1 mvninstall 8m 2s trunk passed +1 compile 9m 0s trunk passed with JDK v1.8.0_74 +1 compile 8m 56s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 20s trunk passed +1 mvnsite 2m 19s trunk passed +1 mvneclipse 0m 50s trunk passed +1 findbugs 4m 40s trunk passed +1 javadoc 2m 17s trunk passed with JDK v1.8.0_74 +1 javadoc 2m 19s trunk passed with JDK v1.7.0_95 0 mvndep 0m 20s Maven dependency ordering for patch +1 mvninstall 1m 59s the patch passed +1 compile 11m 17s the patch passed with JDK v1.8.0_74 +1 javac 11m 17s the patch passed +1 compile 9m 39s the patch passed with JDK v1.7.0_95 +1 javac 9m 39s the patch passed +1 checkstyle 1m 24s root: patch generated 0 new + 263 unchanged - 10 fixed = 263 total (was 273) +1 mvnsite 2m 26s the patch passed +1 mvneclipse 0m 56s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 41s the patch passed +1 javadoc 2m 18s the patch passed with JDK v1.8.0_74 +1 javadoc 2m 17s the patch passed with JDK v1.7.0_95 -1 unit 11m 17s hadoop-common in the patch failed with JDK v1.8.0_74. +1 unit 1m 21s hadoop-hdfs-client in the patch passed with JDK v1.8.0_74. +1 unit 0m 20s hadoop-aws in the patch passed with JDK v1.8.0_74. +1 unit 11m 1s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 1m 18s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. +1 unit 0m 17s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 31s Patch does not generate ASF License warnings. 107m 38s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.ipc.TestIPC Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12796838/HADOOP-12994-002.patch JIRA Issue HADOOP-12994 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f2aeeeb03c10 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5092c94 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9020/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9020/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9020/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9020/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Hello Steve Loughran. This looks good overall. Thank you for expanding the test coverage.

          I ran live Azure testing with the new tests, and one of them failed, as described below. I'm in the middle of a full test run now, which might take up to an hour to complete. I'm not expecting any other impacts, but I'll confirm later after the test run finishes.

          Here are a few comments.

          1. Regarding this ChecksumFileSystem change:
            -  public ChecksumFileSystem(FileSystem fs) {
            +  protected ChecksumFileSystem(FileSystem fs) {
                 super(fs);
               }
            

            I think this is backwards-compatible, but it took me some time to convince myself. I think the reasoning is that since the class is abstract, it cannot be instantiated, so therefore a public constructor doesn't make sense. Anonymous inner classes are not a concern, because they rely on the default no-args constructor. Do I have it right? I'm not asking for any changes here, just trying to confirm my own thinking and calling it out in case others want to comment.

          2. I don't see anything that checks if the supports-positioned-readable contract option is enabled. There should be corresponding skipIfUnsupported calls, right?
          3. FSDataInputStream has a typo in the JavaDocs:
               * @param buffer    buffer into which data is readT
            
          4. After I applied the patch, TestAzureNativeContractSeek#testReadSmallFile failed with an unexpected EOFException. I think the problem is that it doesn't override NativeAzureFileSystem#NativeAzureFsInputStream#read to map EOFException to -1. I tried applying a similar change to what you did for S3AInputStream, and then the test passed.
          Show
          cnauroth Chris Nauroth added a comment - Hello Steve Loughran . This looks good overall. Thank you for expanding the test coverage. I ran live Azure testing with the new tests, and one of them failed, as described below. I'm in the middle of a full test run now, which might take up to an hour to complete. I'm not expecting any other impacts, but I'll confirm later after the test run finishes. Here are a few comments. Regarding this ChecksumFileSystem change: - public ChecksumFileSystem(FileSystem fs) { + protected ChecksumFileSystem(FileSystem fs) { super (fs); } I think this is backwards-compatible, but it took me some time to convince myself. I think the reasoning is that since the class is abstract, it cannot be instantiated, so therefore a public constructor doesn't make sense. Anonymous inner classes are not a concern, because they rely on the default no-args constructor. Do I have it right? I'm not asking for any changes here, just trying to confirm my own thinking and calling it out in case others want to comment. I don't see anything that checks if the supports-positioned-readable contract option is enabled. There should be corresponding skipIfUnsupported calls, right? FSDataInputStream has a typo in the JavaDocs: * @param buffer buffer into which data is readT After I applied the patch, TestAzureNativeContractSeek#testReadSmallFile failed with an unexpected EOFException . I think the problem is that it doesn't override NativeAzureFileSystem#NativeAzureFsInputStream#read to map EOFException to -1. I tried applying a similar change to what you did for S3AInputStream , and then the test passed.
          Hide
          cnauroth Chris Nauroth added a comment -

          My full Azure test run completed successfully, with the exception of what I described in my last comment.

          Show
          cnauroth Chris Nauroth added a comment - My full Azure test run completed successfully, with the exception of what I described in my last comment.
          Hide
          cnauroth Chris Nauroth added a comment -

          Steve Loughran, here are a few additional minor notes:

          1. Please add describe calls to testReadNullBuffer and testReadAtEOFOperations.
          2. Please check the indentation in testReadNullBuffer.
          Show
          cnauroth Chris Nauroth added a comment - Steve Loughran , here are a few additional minor notes: Please add describe calls to testReadNullBuffer and testReadAtEOFOperations . Please check the indentation in testReadNullBuffer .
          Hide
          stevel@apache.org Steve Loughran added a comment -
          1. the protected thing was just an IDE hint; I'll put it as a distraction
          2. I ended up not using that supports-positioned-readable flag, as it comes for free if you handle seek(), and that is implicitly flagged if you implement a subclass of the seek tests.
          3. I'll look at that azure test
          Show
          stevel@apache.org Steve Loughran added a comment - the protected thing was just an IDE hint; I'll put it as a distraction I ended up not using that supports-positioned-readable flag, as it comes for free if you handle seek() , and that is implicitly flagged if you implement a subclass of the seek tests. I'll look at that azure test
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Actually, I will do supports-readfully

          there's an implicit requirement there —how seeks past EOF are handled—, where even if the FS doesn't support that, readFully is expected to catch and downgrade the exception.

          Show
          stevel@apache.org Steve Loughran added a comment - Actually, I will do supports-readfully there's an implicit requirement there —how seeks past EOF are handled—, where even if the FS doesn't support that, readFully is expected to catch and downgrade the exception.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          HADOOP-12994 patch 003
          -address chris's concerns
          -add an option for supports positioned readable; default is driven by "supports seek"
          -split up test operations into separate test cases
          -tested on local, hdfs, s3 filesystems

          I've caught the EOF in Azure and downgraded to a -1, but not tested it...my ASF MSDN sub hasn't renewed; I'm chasing that up, and am currently azureless

          Show
          stevel@apache.org Steve Loughran added a comment - HADOOP-12994 patch 003 -address chris's concerns -add an option for supports positioned readable; default is driven by "supports seek" -split up test operations into separate test cases -tested on local, hdfs, s3 filesystems I've caught the EOF in Azure and downgraded to a -1, but not tested it...my ASF MSDN sub hasn't renewed; I'm chasing that up, and am currently azureless
          Hide
          stevel@apache.org Steve Loughran added a comment -

          As an aside, I was thinking we could just modify FSInputStreamread(long position, byte[] buffer, int offset, int length)) to catch an EOF exception and downgrade.

          With azure, there's now two classes that handle EOF specially; I think swift will need it too. Put it all in one place, and there's one code path that everything picks up.

          I can't see how, on any FS, catching an EOF exception on the seek or read and downgrading to a -1 would be a mistake

          Remember: the param validation of seeking < 0 has already taken place, so the only way an EOF could be raised is if they went off the end and the FS raises EOFs here. For which catch and downgrade is the required action

          Show
          stevel@apache.org Steve Loughran added a comment - As an aside, I was thinking we could just modify FSInputStreamread(long position, byte[] buffer, int offset, int length)) to catch an EOF exception and downgrade. With azure, there's now two classes that handle EOF specially; I think swift will need it too. Put it all in one place, and there's one code path that everything picks up. I can't see how, on any FS, catching an EOF exception on the seek or read and downgrading to a -1 would be a mistake Remember: the param validation of seeking < 0 has already taken place, so the only way an EOF could be raised is if they went off the end and the FS raises EOFs here. For which catch and downgrade is the required action
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 004; as 003 but fixes a statistics counting bug in azure; a -1 on a read was being added to the byte count, so actually lowering it

          Show
          stevel@apache.org Steve Loughran added a comment - patch 004; as 003 but fixes a statistics counting bug in azure; a -1 on a read was being added to the byte count, so actually lowering it
          Hide
          cnauroth Chris Nauroth added a comment -

          Steve Loughran, thank you for patch v004. This looks good to me. Nice catch on that stats counting bug too. Let's see what pre-commit thinks. I'll kick off an Azure test run too.

          Show
          cnauroth Chris Nauroth added a comment - Steve Loughran , thank you for patch v004. This looks good to me. Nice catch on that stats counting bug too. Let's see what pre-commit thinks. I'll kick off an Azure test run too.
          Hide
          cnauroth Chris Nauroth added a comment -

          Steve Loughran, unfortunately the Azure test still failed. Sorry, I really should have given you the full stack trace the first time. See below.

          Your latest patch changed the single-byte read, but this test failure actually happens within the bulk positional read. It's the very last assertion of testReadSmallFile.

          NativeAzureFsInputStream does not override the positional read method, so it inherits the seek/bulk read/seek back implementation from FSInputStream. NativeAzureFsInputStream#seek is coded to raise the EOF eagerly, so this is where it fails. When I fixed the test locally, I did so by copy-pasting your override of the positional read method from S3AInputStream.

          Your comment about covering this logic in the base class seems apt considering the copy-pasting to different subclasses.

          testReadSmallFile(org.apache.hadoop.fs.azure.contract.TestAzureNativeContractSeek)  Time elapsed: 4.038 sec  <<< ERROR!
          java.io.EOFException: Attempted to seek or read past the end of the file
          	at org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsInputStream.seek(NativeAzureFileSystem.java:833)
          	at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:70)
          	at org.apache.hadoop.fs.BufferedFSInputStream.read(BufferedFSInputStream.java:108)
          	at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:92)
          	at org.apache.hadoop.fs.contract.AbstractContractSeekTest.testReadSmallFile(AbstractContractSeekTest.java:568)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:606)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
          	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          
          Show
          cnauroth Chris Nauroth added a comment - Steve Loughran , unfortunately the Azure test still failed. Sorry, I really should have given you the full stack trace the first time. See below. Your latest patch changed the single-byte read, but this test failure actually happens within the bulk positional read. It's the very last assertion of testReadSmallFile . NativeAzureFsInputStream does not override the positional read method, so it inherits the seek/bulk read/seek back implementation from FSInputStream . NativeAzureFsInputStream#seek is coded to raise the EOF eagerly, so this is where it fails. When I fixed the test locally, I did so by copy-pasting your override of the positional read method from S3AInputStream . Your comment about covering this logic in the base class seems apt considering the copy-pasting to different subclasses. testReadSmallFile(org.apache.hadoop.fs.azure.contract.TestAzureNativeContractSeek) Time elapsed: 4.038 sec <<< ERROR! java.io.EOFException: Attempted to seek or read past the end of the file at org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsInputStream.seek(NativeAzureFileSystem.java:833) at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:70) at org.apache.hadoop.fs.BufferedFSInputStream.read(BufferedFSInputStream.java:108) at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:92) at org.apache.hadoop.fs.contract.AbstractContractSeekTest.testReadSmallFile(AbstractContractSeekTest.java:568) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
          0 mvndep 1m 12s Maven dependency ordering for branch
          +1 mvninstall 6m 43s trunk passed
          +1 compile 5m 54s trunk passed with JDK v1.8.0_77
          +1 compile 6m 41s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 8s trunk passed
          +1 mvnsite 2m 10s trunk passed
          +1 mvneclipse 0m 53s trunk passed
          +1 findbugs 4m 10s trunk passed
          +1 javadoc 1m 35s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 55s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 42s the patch passed
          +1 compile 5m 33s the patch passed with JDK v1.8.0_77
          +1 javac 5m 33s the patch passed
          +1 compile 6m 57s the patch passed with JDK v1.7.0_95
          +1 javac 6m 57s the patch passed
          -1 checkstyle 1m 9s root: patch generated 1 new + 279 unchanged - 10 fixed = 280 total (was 289)
          +1 mvnsite 2m 14s the patch passed
          +1 mvneclipse 0m 47s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 5m 21s the patch passed
          +1 javadoc 2m 2s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 56s the patch passed with JDK v1.7.0_95
          +1 unit 7m 9s hadoop-common in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 50s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_77.
          +1 unit 1m 13s hadoop-azure in the patch passed with JDK v1.8.0_77.
          +1 unit 7m 59s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 59s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 13s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 unit 1m 31s hadoop-azure in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          82m 39s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797599/HADOOP-12994-004.patch
          JIRA Issue HADOOP-12994
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux b4390e4fd5ee 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a62637a
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9041/artifact/patchprocess/diff-checkstyle-root.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9041/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9041/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 1m 12s Maven dependency ordering for branch +1 mvninstall 6m 43s trunk passed +1 compile 5m 54s trunk passed with JDK v1.8.0_77 +1 compile 6m 41s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 8s trunk passed +1 mvnsite 2m 10s trunk passed +1 mvneclipse 0m 53s trunk passed +1 findbugs 4m 10s trunk passed +1 javadoc 1m 35s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 55s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 42s the patch passed +1 compile 5m 33s the patch passed with JDK v1.8.0_77 +1 javac 5m 33s the patch passed +1 compile 6m 57s the patch passed with JDK v1.7.0_95 +1 javac 6m 57s the patch passed -1 checkstyle 1m 9s root: patch generated 1 new + 279 unchanged - 10 fixed = 280 total (was 289) +1 mvnsite 2m 14s the patch passed +1 mvneclipse 0m 47s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 5m 21s the patch passed +1 javadoc 2m 2s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 56s the patch passed with JDK v1.7.0_95 +1 unit 7m 9s hadoop-common in the patch passed with JDK v1.8.0_77. +1 unit 0m 50s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_77. +1 unit 1m 13s hadoop-azure in the patch passed with JDK v1.8.0_77. +1 unit 7m 59s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 0m 59s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. +1 unit 0m 13s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 unit 1m 31s hadoop-azure in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 82m 39s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797599/HADOOP-12994-004.patch JIRA Issue HADOOP-12994 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux b4390e4fd5ee 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a62637a Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9041/artifact/patchprocess/diff-checkstyle-root.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9041/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9041/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 005: pull up EOF handling into FSInputStream, adding some logging @ debug. This exception will only get raised if seek past EOF is an error, so I don't see the codepath causing problems on other filesystems (the check for pos <0 has already taken place by that point).

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 005: pull up EOF handling into FSInputStream , adding some logging @ debug. This exception will only get raised if seek past EOF is an error, so I don't see the codepath causing problems on other filesystems (the check for pos <0 has already taken place by that point).
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 5 passes azure native tests, though it's pretty slow in the seek one

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractAppend
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 13.093 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractAppend
          Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractCreate
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.775 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractCreate
          Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractDelete
          Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.508 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractDelete
          Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractGetFileStatus
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.224 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractGetFileStatus
          Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractMkdir
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.084 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractMkdir
          Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractOpen
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.864 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractOpen
          Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractRename
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.583 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractRename
          Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractSeek
          Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.391 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractSeek
          
          Results :
          
          Tests run: 56, Failures: 0, Errors: 0, Skipped: 1
          
          Show
          stevel@apache.org Steve Loughran added a comment - Patch 5 passes azure native tests, though it's pretty slow in the seek one ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractAppend Tests run: 5, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 13.093 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractAppend Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractCreate Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.775 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractCreate Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractDelete Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.508 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractDelete Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractGetFileStatus Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.224 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractGetFileStatus Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractMkdir Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.084 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractMkdir Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractOpen Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.864 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractOpen Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractRename Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.583 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractRename Running org.apache.hadoop.fs.azure.contract.TestAzureNativeContractSeek Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.391 sec - in org.apache.hadoop.fs.azure.contract.TestAzureNativeContractSeek Results : Tests run: 56, Failures: 0, Errors: 0, Skipped: 1
          Hide
          stevel@apache.org Steve Loughran added a comment -

          And against an openstack implementation, which is even slower, but there were transatlantic cables in the way

           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.fs.swift.contract.TestSwiftContractCreate
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 80.309 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractCreate
          Running org.apache.hadoop.fs.swift.contract.TestSwiftContractDelete
          Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 110.419 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractDelete
          Running org.apache.hadoop.fs.swift.contract.TestSwiftContractMkdir
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.103 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractMkdir
          Running org.apache.hadoop.fs.swift.contract.TestSwiftContractOpen
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 47.185 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractOpen
          Running org.apache.hadoop.fs.swift.contract.TestSwiftContractRename
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.541 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractRename
          Running org.apache.hadoop.fs.swift.contract.TestSwiftContractRootDir
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 54.093 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractRootDir
          Running org.apache.hadoop.fs.swift.contract.TestSwiftContractSeek
          Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 196.56 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractSeek
          
          Results :
          
          Tests run: 55, Failures: 0, Errors: 0, Skipped: 5
          
          Show
          stevel@apache.org Steve Loughran added a comment - And against an openstack implementation, which is even slower, but there were transatlantic cables in the way T E S T S ------------------------------------------------------- Running org.apache.hadoop.fs.swift.contract.TestSwiftContractCreate Tests run: 6, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 80.309 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractCreate Running org.apache.hadoop.fs.swift.contract.TestSwiftContractDelete Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 110.419 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractDelete Running org.apache.hadoop.fs.swift.contract.TestSwiftContractMkdir Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.103 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractMkdir Running org.apache.hadoop.fs.swift.contract.TestSwiftContractOpen Tests run: 6, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 47.185 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractOpen Running org.apache.hadoop.fs.swift.contract.TestSwiftContractRename Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.541 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractRename Running org.apache.hadoop.fs.swift.contract.TestSwiftContractRootDir Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 54.093 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractRootDir Running org.apache.hadoop.fs.swift.contract.TestSwiftContractSeek Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 196.56 sec - in org.apache.hadoop.fs.swift.contract.TestSwiftContractSeek Results : Tests run: 55, Failures: 0, Errors: 0, Skipped: 5
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 6m 48s trunk passed
          +1 compile 6m 24s trunk passed with JDK v1.8.0_77
          +1 compile 7m 7s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 7s trunk passed
          +1 mvnsite 2m 17s trunk passed
          +1 mvneclipse 0m 50s trunk passed
          +1 findbugs 4m 12s trunk passed
          +1 javadoc 1m 38s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 58s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 1m 43s the patch passed
          +1 compile 6m 13s the patch passed with JDK v1.8.0_77
          +1 javac 6m 13s the patch passed
          +1 compile 6m 31s the patch passed with JDK v1.7.0_95
          +1 javac 6m 31s the patch passed
          +1 checkstyle 1m 6s root: patch generated 0 new + 278 unchanged - 10 fixed = 278 total (was 288)
          +1 mvnsite 2m 10s the patch passed
          +1 mvneclipse 0m 50s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 5m 6s the patch passed
          +1 javadoc 1m 42s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 57s the patch passed with JDK v1.7.0_95
          -1 unit 7m 23s hadoop-common in the patch failed with JDK v1.8.0_77.
          +1 unit 0m 50s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_77.
          +1 unit 1m 12s hadoop-azure in the patch passed with JDK v1.8.0_77.
          -1 unit 7m 32s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 0m 55s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 15s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 unit 1m 29s hadoop-azure in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          82m 14s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.net.TestDNS
          JDK v1.7.0_95 Failed junit tests hadoop.net.TestDNS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797719/HADOOP-12994-005.patch
          JIRA Issue HADOOP-12994
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 88b5684e9c18 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6791221
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 48s trunk passed +1 compile 6m 24s trunk passed with JDK v1.8.0_77 +1 compile 7m 7s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 7s trunk passed +1 mvnsite 2m 17s trunk passed +1 mvneclipse 0m 50s trunk passed +1 findbugs 4m 12s trunk passed +1 javadoc 1m 38s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 58s trunk passed with JDK v1.7.0_95 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 43s the patch passed +1 compile 6m 13s the patch passed with JDK v1.8.0_77 +1 javac 6m 13s the patch passed +1 compile 6m 31s the patch passed with JDK v1.7.0_95 +1 javac 6m 31s the patch passed +1 checkstyle 1m 6s root: patch generated 0 new + 278 unchanged - 10 fixed = 278 total (was 288) +1 mvnsite 2m 10s the patch passed +1 mvneclipse 0m 50s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 5m 6s the patch passed +1 javadoc 1m 42s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 57s the patch passed with JDK v1.7.0_95 -1 unit 7m 23s hadoop-common in the patch failed with JDK v1.8.0_77. +1 unit 0m 50s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. +1 unit 0m 12s hadoop-aws in the patch passed with JDK v1.8.0_77. +1 unit 1m 12s hadoop-azure in the patch passed with JDK v1.8.0_77. -1 unit 7m 32s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 0m 55s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. +1 unit 0m 15s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 unit 1m 29s hadoop-azure in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 82m 14s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.net.TestDNS JDK v1.7.0_95 Failed junit tests hadoop.net.TestDNS Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797719/HADOOP-12994-005.patch JIRA Issue HADOOP-12994 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 88b5684e9c18 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6791221 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9046/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 006.

          This adds one more assert, that if you seek(0) on a zero byte file, then a read(buffer, offset, 0) (that is: 0 bytes of data read), is not an error.

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 006. This adds one more assert, that if you seek(0) on a zero byte file, then a read(buffer, offset, 0) (that is: 0 bytes of data read), is not an error.
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 for patch v006 pending pre-commit. I confirmed that all live hadoop-azure tests pass. Steve Loughran, thank you for the patch. Very helpful.

          This wasn't in patch available state, so I just clicked the button and triggered a pre-commit run manually:

          https://builds.apache.org/view/PreCommit%20Builds/job/PreCommit-HADOOP-Build/9047/

          Show
          cnauroth Chris Nauroth added a comment - +1 for patch v006 pending pre-commit. I confirmed that all live hadoop-azure tests pass. Steve Loughran , thank you for the patch. Very helpful. This wasn't in patch available state, so I just clicked the button and triggered a pre-commit run manually: https://builds.apache.org/view/PreCommit%20Builds/job/PreCommit-HADOOP-Build/9047/
          Hide
          stevel@apache.org Steve Loughran added a comment -

          thanks. Chris: I'm going offline ... would you be able to commit it when you're happy?

          Show
          stevel@apache.org Steve Loughran added a comment - thanks. Chris: I'm going offline ... would you be able to commit it when you're happy?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 6m 37s trunk passed
          +1 compile 6m 33s trunk passed with JDK v1.8.0_77
          +1 compile 7m 8s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 9s trunk passed
          +1 mvnsite 2m 12s trunk passed
          +1 mvneclipse 0m 52s trunk passed
          +1 findbugs 4m 14s trunk passed
          +1 javadoc 1m 42s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 59s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 47s the patch passed
          +1 compile 6m 31s the patch passed with JDK v1.8.0_77
          +1 javac 6m 31s the patch passed
          +1 compile 7m 9s the patch passed with JDK v1.7.0_95
          +1 javac 7m 9s the patch passed
          +1 checkstyle 1m 8s root: patch generated 0 new + 279 unchanged - 10 fixed = 279 total (was 289)
          +1 mvnsite 2m 11s the patch passed
          +1 mvneclipse 0m 52s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 5m 4s the patch passed
          +1 javadoc 1m 41s the patch passed with JDK v1.8.0_77
          +1 javadoc 2m 1s the patch passed with JDK v1.7.0_95
          -1 unit 7m 21s hadoop-common in the patch failed with JDK v1.8.0_77.
          +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 13s hadoop-aws in the patch passed with JDK v1.8.0_77.
          +1 unit 1m 24s hadoop-azure in the patch passed with JDK v1.8.0_77.
          +1 unit 7m 46s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 15s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 unit 1m 33s hadoop-azure in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          84m 8s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.net.TestClusterTopology



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797734/HADOOP-12994-006.patch
          JIRA Issue HADOOP-12994
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux e3430a01f0ac 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ce5b481
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9047/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9047/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9047/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9047/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 37s trunk passed +1 compile 6m 33s trunk passed with JDK v1.8.0_77 +1 compile 7m 8s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 9s trunk passed +1 mvnsite 2m 12s trunk passed +1 mvneclipse 0m 52s trunk passed +1 findbugs 4m 14s trunk passed +1 javadoc 1m 42s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 59s trunk passed with JDK v1.7.0_95 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 47s the patch passed +1 compile 6m 31s the patch passed with JDK v1.8.0_77 +1 javac 6m 31s the patch passed +1 compile 7m 9s the patch passed with JDK v1.7.0_95 +1 javac 7m 9s the patch passed +1 checkstyle 1m 8s root: patch generated 0 new + 279 unchanged - 10 fixed = 279 total (was 289) +1 mvnsite 2m 11s the patch passed +1 mvneclipse 0m 52s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 5m 4s the patch passed +1 javadoc 1m 41s the patch passed with JDK v1.8.0_77 +1 javadoc 2m 1s the patch passed with JDK v1.7.0_95 -1 unit 7m 21s hadoop-common in the patch failed with JDK v1.8.0_77. +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. +1 unit 0m 13s hadoop-aws in the patch passed with JDK v1.8.0_77. +1 unit 1m 24s hadoop-azure in the patch passed with JDK v1.8.0_77. +1 unit 7m 46s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. +1 unit 0m 15s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 unit 1m 33s hadoop-azure in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 84m 8s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.net.TestClusterTopology Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797734/HADOOP-12994-006.patch JIRA Issue HADOOP-12994 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux e3430a01f0ac 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ce5b481 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9047/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9047/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9047/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9047/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 7m 8s trunk passed
          +1 compile 6m 32s trunk passed with JDK v1.8.0_77
          +1 compile 6m 53s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 8s trunk passed
          +1 mvnsite 2m 12s trunk passed
          +1 mvneclipse 0m 49s trunk passed
          +1 findbugs 4m 11s trunk passed
          +1 javadoc 1m 42s trunk passed with JDK v1.8.0_77
          +1 javadoc 2m 2s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 49s the patch passed
          +1 compile 6m 35s the patch passed with JDK v1.8.0_77
          +1 javac 6m 35s the patch passed
          +1 compile 6m 53s the patch passed with JDK v1.7.0_95
          +1 javac 6m 53s the patch passed
          +1 checkstyle 1m 8s root: patch generated 0 new + 279 unchanged - 10 fixed = 279 total (was 289)
          +1 mvnsite 2m 8s the patch passed
          +1 mvneclipse 0m 49s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 5m 7s the patch passed
          +1 javadoc 1m 48s the patch passed with JDK v1.8.0_77
          +1 javadoc 2m 9s the patch passed with JDK v1.7.0_95
          +1 unit 7m 51s hadoop-common in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 55s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.8.0_77.
          +1 unit 1m 19s hadoop-azure in the patch passed with JDK v1.8.0_77.
          +1 unit 7m 44s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 57s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95.
          +1 unit 1m 32s hadoop-azure in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          84m 42s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797734/HADOOP-12994-006.patch
          JIRA Issue HADOOP-12994
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 8d3240507f6e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ce5b481
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9048/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9048/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 7m 8s trunk passed +1 compile 6m 32s trunk passed with JDK v1.8.0_77 +1 compile 6m 53s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 8s trunk passed +1 mvnsite 2m 12s trunk passed +1 mvneclipse 0m 49s trunk passed +1 findbugs 4m 11s trunk passed +1 javadoc 1m 42s trunk passed with JDK v1.8.0_77 +1 javadoc 2m 2s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 49s the patch passed +1 compile 6m 35s the patch passed with JDK v1.8.0_77 +1 javac 6m 35s the patch passed +1 compile 6m 53s the patch passed with JDK v1.7.0_95 +1 javac 6m 53s the patch passed +1 checkstyle 1m 8s root: patch generated 0 new + 279 unchanged - 10 fixed = 279 total (was 289) +1 mvnsite 2m 8s the patch passed +1 mvneclipse 0m 49s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 5m 7s the patch passed +1 javadoc 1m 48s the patch passed with JDK v1.8.0_77 +1 javadoc 2m 9s the patch passed with JDK v1.7.0_95 +1 unit 7m 51s hadoop-common in the patch passed with JDK v1.8.0_77. +1 unit 0m 55s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.8.0_77. +1 unit 1m 19s hadoop-azure in the patch passed with JDK v1.8.0_77. +1 unit 7m 44s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 0m 57s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.7.0_95. +1 unit 1m 32s hadoop-azure in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 84m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797734/HADOOP-12994-006.patch JIRA Issue HADOOP-12994 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 8d3240507f6e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ce5b481 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9048/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9048/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          I have committed this to trunk, branch-2 and branch-2.8. For branch-2.8, I needed to resolve a trivial merge conflict. Steve Loughran, thank you for the contribution.

          Show
          cnauroth Chris Nauroth added a comment - I have committed this to trunk, branch-2 and branch-2.8. For branch-2.8, I needed to resolve a trivial merge conflict. Steve Loughran , thank you for the contribution.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9582 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9582/)
          HADOOP-12994. Specify PositionedReadable, add contract tests, fix (cnauroth: rev 843ee8d59d8bacbca0d87ccf0790772e39d16138)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PositionedReadable.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractConcatTest.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractDeleteTest.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractRootDirectoryTest.java
          • hadoop-common-project/hadoop-common/src/test/resources/contract/localfs.xml
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputStream.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractAppendTest.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractOpenTest.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/CryptoStreamsTestBase.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractCreateTest.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSExceptionMessages.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractMkdirTest.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFs.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractOptions.java
          • hadoop-common-project/hadoop-common/src/site/markdown/filesystem/fsdatainputstream.md
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractFSContractTestBase.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractSeekTest.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractRenameTest.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoInputStream.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          • hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9582 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9582/ ) HADOOP-12994 . Specify PositionedReadable, add contract tests, fix (cnauroth: rev 843ee8d59d8bacbca0d87ccf0790772e39d16138) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PositionedReadable.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractConcatTest.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractDeleteTest.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractRootDirectoryTest.java hadoop-common-project/hadoop-common/src/test/resources/contract/localfs.xml hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputStream.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractAppendTest.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractOpenTest.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/CryptoStreamsTestBase.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractCreateTest.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSExceptionMessages.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractMkdirTest.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFs.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractOptions.java hadoop-common-project/hadoop-common/src/site/markdown/filesystem/fsdatainputstream.md hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractFSContractTestBase.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractSeekTest.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractRenameTest.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoInputStream.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Following Test fails after this in...Since jenkins did not run on hdfs project, it was not shown in QA report..

          java.lang.AssertionError: expected:<0> but was:<-1>
          	at org.junit.Assert.fail(Assert.java:88)
          	at org.junit.Assert.failNotEquals(Assert.java:743)
          	at org.junit.Assert.assertEquals(Assert.java:118)
          	at org.junit.Assert.assertEquals(Assert.java:555)
          	at org.junit.Assert.assertEquals(Assert.java:542)
          	at org.apache.hadoop.fs.contract.AbstractContractSeekTest.testReadFullyZeroByteFile(AbstractContractSeekTest.java:373)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
          	at java.lang.reflect.Method.invoke(Unknown Source)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
          	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          
          Show
          brahmareddy Brahma Reddy Battula added a comment - Following Test fails after this in...Since jenkins did not run on hdfs project, it was not shown in QA report.. java.lang.AssertionError: expected:<0> but was:<-1> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.junit.Assert.assertEquals(Assert.java:542) at org.apache.hadoop.fs.contract.AbstractContractSeekTest.testReadFullyZeroByteFile(AbstractContractSeekTest.java:373) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          Hide
          stevel@apache.org Steve Loughran added a comment -

          can't have jenkins unhappy. I thought I'd been checking things locally, but clearly not..

          Show
          stevel@apache.org Steve Loughran added a comment - can't have jenkins unhappy. I thought I'd been checking things locally, but clearly not..
          Hide
          dian.fu Dian Fu added a comment -

          Test TestShortCircuitLocalRead fails after this patch is in:

          java.lang.IndexOutOfBoundsException: Requested more bytes than destination buffer size
          	at org.apache.hadoop.fs.FSInputStream.validatePositionedReadArgs(FSInputStream.java:107)
          	at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:975)
          	at java.io.DataInputStream.read(DataInputStream.java:149)
          	at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.checkFileContent(TestShortCircuitLocalRead.java:157)
          	at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitReadImpl(TestShortCircuitLocalRead.java:286)
          	at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitRead(TestShortCircuitLocalRead.java:241)
          	at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.testSmallFileLocalRead(TestShortCircuitLocalRead.java:308)
          
          Show
          dian.fu Dian Fu added a comment - Test TestShortCircuitLocalRead fails after this patch is in: java.lang.IndexOutOfBoundsException: Requested more bytes than destination buffer size at org.apache.hadoop.fs.FSInputStream.validatePositionedReadArgs(FSInputStream.java:107) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:975) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.checkFileContent(TestShortCircuitLocalRead.java:157) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitReadImpl(TestShortCircuitLocalRead.java:286) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.doTestShortCircuitRead(TestShortCircuitLocalRead.java:241) at org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead.testSmallFileLocalRead(TestShortCircuitLocalRead.java:308)

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development