Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12726

Unsupported FS operations should throw UnsupportedOperationException

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Unsupported FileSystem operations now throw an UnsupportedOperationException rather than an IOException.

      Description

      In the FileSystem implementation classes, unsupported operations throw new IOException("Not supported"), which makes it needlessly difficult to distinguish an actual error from an unsupported operation. They should instead throw new UnsupportedOperationException().

      It's possible that this anti-idiom is used elsewhere in the code base. This JIRA should include finding and cleaning up those instances as well.

      1. HADOOP-12726.001.patch
        7 kB
        Daniel Templeton
      2. HADOOP-12726.002.patch
        6 kB
        Daniel Templeton
      3. HADOOP-12726.003.patch
        8 kB
        Chris Douglas

        Issue Links

          Activity

          Hide
          stevel@apache.org Steve Loughran added a comment -

          in favour for all FS operations. In an ideal world, you'd even be able to tell in advance what was unsupported, but as it may depend on the specific FS instance, that's a tough one, especially for rawfs

          Show
          stevel@apache.org Steve Loughran added a comment - in favour for all FS operations. In an ideal world, you'd even be able to tell in advance what was unsupported, but as it may depend on the specific FS instance, that's a tough one, especially for rawfs
          Hide
          stevel@apache.org Steve Loughran added a comment -

          This wont be incompatible, provided the exception is an IOE. If you use java.lang.UnsupportedOperationException, things would break, so your patch wouldn't get in

          Show
          stevel@apache.org Steve Loughran added a comment - This wont be incompatible, provided the exception is an IOE. If you use java.lang.UnsupportedOperationException, things would break, so your patch wouldn't get in
          Hide
          chris.douglas Chris Douglas added a comment -

          If you use java.lang.UnsupportedOperationException, things would break, so your patch wouldn't get in

          What do you think about fixing this in trunk?

          Show
          chris.douglas Chris Douglas added a comment - If you use java.lang.UnsupportedOperationException, things would break, so your patch wouldn't get in What do you think about fixing this in trunk?
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I'd go for an UnsupportedIOOperationException extends IOException in trunk

          Show
          stevel@apache.org Steve Loughran added a comment - I'd go for an UnsupportedIOOperationException extends IOException in trunk
          Hide
          templedf Daniel Templeton added a comment -

          I really dislike the idea of replicating core functionality like that. I'd also balk at creating Frankenstein's exception.

          Is your concern that some bits of the system are catching IOException with the expectation that they will also catch unsupported operations? I would offer the argument that an unsupported operation is inherently a different thing from an IOException. If some code is catching IOException to deal with both, they're probably doing something wrong. I'd also be willing to bet that in the only case where that approach is valid: log and exit, they're catching Exception instead.

          I think it should be pretty reasonable to go through the code to find all cases of calls to potentially unsupported operations and make sure they handle an UnsupportedOperationException correctly. That leaves client code at risk of breaking, but that's where we started.

          Show
          templedf Daniel Templeton added a comment - I really dislike the idea of replicating core functionality like that. I'd also balk at creating Frankenstein's exception. Is your concern that some bits of the system are catching IOException with the expectation that they will also catch unsupported operations? I would offer the argument that an unsupported operation is inherently a different thing from an IOException . If some code is catching IOException to deal with both, they're probably doing something wrong. I'd also be willing to bet that in the only case where that approach is valid: log and exit, they're catching Exception instead. I think it should be pretty reasonable to go through the code to find all cases of calls to potentially unsupported operations and make sure they handle an UnsupportedOperationException correctly. That leaves client code at risk of breaking, but that's where we started.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I'd also balk at creating Frankenstein's exception.

          see java.io.InterruptedIOException as evidence our strategy is legitmate

          Is your concern that some bits of the system are catching IOException with the expectation that they will also catch unsupported operations?

          Yes, I am confident that some apps will be doing just that. Indeed, I'd use your KMS patch as an example.

          I think it should be pretty reasonable to go through the code to find all cases of calls to potentially unsupported operations

          Which code are we talking about here? FileSystem is a public API, tagged as stable. To see how these things get used, you can't just look at Hadoop-core, you need HBase, accumulo, spark, hive, ... etc in your IDE. I know, I've been there.

          The exception model is part of its semantics.

          Now, looking at what coverage there is of the FS API, it does say that seek can throw UnsupportedOperationException, indeed, it's implied its preferred to an IOE.

          supported(FSDIS, Seekable.seek) else raise [UnsupportedOperationException, IOException]
          

          Which meant when I was rummaging around the implementations of seek(), some must do this -looks like CompressionInputStream is one of them.

          So, if you can show that other implementations of various unsupported operations are actually throwing UnsupportedOperationException then you can make the case that changing the exception is not incompatible, just making things more consistent. You get to be the one who looks through all the methods though

          Show
          stevel@apache.org Steve Loughran added a comment - I'd also balk at creating Frankenstein's exception. see java.io.InterruptedIOException as evidence our strategy is legitmate Is your concern that some bits of the system are catching IOException with the expectation that they will also catch unsupported operations? Yes, I am confident that some apps will be doing just that. Indeed, I'd use your KMS patch as an example. I think it should be pretty reasonable to go through the code to find all cases of calls to potentially unsupported operations Which code are we talking about here? FileSystem is a public API, tagged as stable. To see how these things get used, you can't just look at Hadoop-core, you need HBase, accumulo, spark, hive, ... etc in your IDE. I know, I've been there. The exception model is part of its semantics. Now, looking at what coverage there is of the FS API , it does say that seek can throw UnsupportedOperationException, indeed, it's implied its preferred to an IOE. supported(FSDIS, Seekable.seek) else raise [UnsupportedOperationException, IOException] Which meant when I was rummaging around the implementations of seek(), some must do this -looks like CompressionInputStream is one of them. So, if you can show that other implementations of various unsupported operations are actually throwing UnsupportedOperationException then you can make the case that changing the exception is not incompatible, just making things more consistent. You get to be the one who looks through all the methods though
          Hide
          chris.douglas Chris Douglas added a comment -

          java.io.InterruptedIOException serves a different purpose (albeit incompletely), to allow I/O stacks to reason about which data were acknowledged/believed to be durable when the thread was interrupted. It also has the effect of not littering all APIs with InterruptedException, but it's not purely intended as a tunnel through the type system, as UnsupportedIOOperationException would be.

          FileSystem is a public API, but implementations routinely throw UnsupportedOperationException. Particularly as they're layered over compression libraries, encryption libraries, etc. that are also pluggable. Exceptions are part of the semantics, you are entirely correct on that point, but throwing an IOException for unsupported operations requires justification, not the other way around. Even if we want to label this as incompatible, we can still fix it in trunk.

          Show
          chris.douglas Chris Douglas added a comment - java.io.InterruptedIOException serves a different purpose (albeit incompletely), to allow I/O stacks to reason about which data were acknowledged/believed to be durable when the thread was interrupted. It also has the effect of not littering all APIs with InterruptedException , but it's not purely intended as a tunnel through the type system, as UnsupportedIOOperationException would be. FileSystem is a public API, but implementations routinely throw UnsupportedOperationException . Particularly as they're layered over compression libraries, encryption libraries, etc. that are also pluggable. Exceptions are part of the semantics, you are entirely correct on that point, but throwing an IOException for unsupported operations requires justification, not the other way around. Even if we want to label this as incompatible, we can still fix it in trunk.
          Hide
          templedf Daniel Templeton added a comment -

          Here's a first pass at the change. I didn't see any test code that tests the unsupportedness of any method calls.

          Show
          templedf Daniel Templeton added a comment - Here's a first pass at the change. I didn't see any test code that tests the unsupportedness of any method calls.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 35s Maven dependency ordering for branch
          +1 mvninstall 7m 42s trunk passed
          +1 compile 10m 20s trunk passed with JDK v1.8.0_74
          +1 compile 9m 33s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 20s trunk passed
          +1 mvnsite 1m 36s trunk passed
          +1 mvneclipse 0m 34s trunk passed
          +1 findbugs 2m 44s trunk passed
          +1 javadoc 1m 37s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 29s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 6s the patch passed
          +1 compile 10m 5s the patch passed with JDK v1.8.0_74
          +1 javac 10m 5s the patch passed
          +1 compile 8m 25s the patch passed with JDK v1.7.0_95
          +1 javac 8m 25s the patch passed
          +1 checkstyle 1m 12s the patch passed
          +1 mvnsite 1m 23s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 42s the patch passed
          +1 javadoc 1m 24s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 31s the patch passed with JDK v1.7.0_95
          -1 unit 10m 4s hadoop-common in the patch failed with JDK v1.8.0_74.
          +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.8.0_74.
          -1 unit 9m 28s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 0m 13s hadoop-aws in the patch passed with JDK v1.7.0_95.
          -1 asflicense 0m 25s Patch generated 2 ASF License warnings.
          88m 10s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.net.TestDNS
          JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Failed junit tests hadoop.net.TestDNS
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794616/HADOOP-12726.001.patch
          JIRA Issue HADOOP-12726
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b106de3ba8ce 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e7ed05e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 35s Maven dependency ordering for branch +1 mvninstall 7m 42s trunk passed +1 compile 10m 20s trunk passed with JDK v1.8.0_74 +1 compile 9m 33s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 20s trunk passed +1 mvnsite 1m 36s trunk passed +1 mvneclipse 0m 34s trunk passed +1 findbugs 2m 44s trunk passed +1 javadoc 1m 37s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 29s trunk passed with JDK v1.7.0_95 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 6s the patch passed +1 compile 10m 5s the patch passed with JDK v1.8.0_74 +1 javac 10m 5s the patch passed +1 compile 8m 25s the patch passed with JDK v1.7.0_95 +1 javac 8m 25s the patch passed +1 checkstyle 1m 12s the patch passed +1 mvnsite 1m 23s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 42s the patch passed +1 javadoc 1m 24s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 31s the patch passed with JDK v1.7.0_95 -1 unit 10m 4s hadoop-common in the patch failed with JDK v1.8.0_74. +1 unit 0m 14s hadoop-aws in the patch passed with JDK v1.8.0_74. -1 unit 9m 28s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 0m 13s hadoop-aws in the patch passed with JDK v1.7.0_95. -1 asflicense 0m 25s Patch generated 2 ASF License warnings. 88m 10s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.net.TestDNS JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Failed junit tests hadoop.net.TestDNS JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794616/HADOOP-12726.001.patch JIRA Issue HADOOP-12726 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b106de3ba8ce 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e7ed05e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8887/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          templedf Daniel Templeton added a comment -

          Chris Douglas, Steve Loughran, any feedback on the patch?

          Show
          templedf Daniel Templeton added a comment - Chris Douglas , Steve Loughran , any feedback on the patch?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 13s HADOOP-12726 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794616/HADOOP-12726.001.patch
          JIRA Issue HADOOP-12726
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9993/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 13s HADOOP-12726 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794616/HADOOP-12726.001.patch JIRA Issue HADOOP-12726 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9993/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          templedf Daniel Templeton added a comment -

          Guess I need to rebase.

          Show
          templedf Daniel Templeton added a comment - Guess I need to rebase.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          not been looking at this. Have them all throw UnsupportedException: you know it makes sense

          Show
          stevel@apache.org Steve Loughran added a comment - not been looking at this. Have them all throw UnsupportedException: you know it makes sense
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 17s Maven dependency ordering for branch
          +1 mvninstall 7m 23s trunk passed
          +1 compile 7m 16s trunk passed
          +1 checkstyle 1m 28s trunk passed
          +1 mvnsite 1m 19s trunk passed
          +1 mvneclipse 0m 56s trunk passed
          +1 findbugs 1m 54s trunk passed
          +1 javadoc 1m 4s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 0s the patch passed
          +1 compile 7m 14s the patch passed
          +1 javac 7m 14s the patch passed
          +1 checkstyle 1m 29s the patch passed
          +1 mvnsite 1m 23s the patch passed
          +1 mvneclipse 0m 35s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 18s the patch passed
          +1 javadoc 1m 11s the patch passed
          -1 unit 9m 4s hadoop-common in the patch failed.
          +1 unit 0m 31s hadoop-aws in the patch passed.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          70m 27s



          Reason Tests
          Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818021/HADOOP-12726.002.patch
          JIRA Issue HADOOP-12726
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 07e4a94422b3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6cf0175
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10002/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10002/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10002/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 7m 23s trunk passed +1 compile 7m 16s trunk passed +1 checkstyle 1m 28s trunk passed +1 mvnsite 1m 19s trunk passed +1 mvneclipse 0m 56s trunk passed +1 findbugs 1m 54s trunk passed +1 javadoc 1m 4s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 0s the patch passed +1 compile 7m 14s the patch passed +1 javac 7m 14s the patch passed +1 checkstyle 1m 29s the patch passed +1 mvnsite 1m 23s the patch passed +1 mvneclipse 0m 35s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 18s the patch passed +1 javadoc 1m 11s the patch passed -1 unit 9m 4s hadoop-common in the patch failed. +1 unit 0m 31s hadoop-aws in the patch passed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 70m 27s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818021/HADOOP-12726.002.patch JIRA Issue HADOOP-12726 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 07e4a94422b3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6cf0175 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10002/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10002/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10002/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          templedf Daniel Templeton added a comment -

          Test failure is unrelated.

          Show
          templedf Daniel Templeton added a comment - Test failure is unrelated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Steve Loughran I checked filesystem.md and it says that FileSystem::append MAY throw UnsupportedOperationException. Assuming these are RFC2119 defs, should we update append, truncate, open and concat to SHOULD throw UnsupportedOperationException when the FileSystem does not provide a compliant implementation? Attaching proposed edits to Daniel's patch.

          TestGangliaMetrics passed on my machine, as did the TestRollingFileSystemSink* tests, so the failure does seem unrelated.

          Show
          chris.douglas Chris Douglas added a comment - Steve Loughran I checked filesystem.md and it says that FileSystem::append MAY throw UnsupportedOperationException . Assuming these are RFC2119 defs, should we update append , truncate , open and concat to SHOULD throw UnsupportedOperationException when the FileSystem does not provide a compliant implementation? Attaching proposed edits to Daniel's patch. TestGangliaMetrics passed on my machine, as did the TestRollingFileSystemSink* tests, so the failure does seem unrelated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 25s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 1m 50s Maven dependency ordering for branch
          +1 mvninstall 8m 53s trunk passed
          +1 compile 9m 15s trunk passed
          +1 checkstyle 1m 49s trunk passed
          +1 mvnsite 1m 36s trunk passed
          +1 mvneclipse 0m 31s trunk passed
          +1 findbugs 2m 20s trunk passed
          +1 javadoc 1m 20s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 5s the patch passed
          +1 compile 8m 12s the patch passed
          +1 javac 8m 12s the patch passed
          +1 checkstyle 1m 35s the patch passed
          +1 mvnsite 1m 29s the patch passed
          +1 mvneclipse 0m 38s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 24s the patch passed
          +1 javadoc 1m 10s the patch passed
          -1 unit 8m 10s hadoop-common in the patch failed.
          +1 unit 0m 23s hadoop-aws in the patch passed.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          76m 49s



          Reason Tests
          Failed junit tests hadoop.security.TestGroupsCaching
            hadoop.metrics2.impl.TestGangliaMetrics



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818073/HADOOP-12726.003.patch
          JIRA Issue HADOOP-12726
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e88010ef0090 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e549a9a
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10007/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10007/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10007/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 1m 50s Maven dependency ordering for branch +1 mvninstall 8m 53s trunk passed +1 compile 9m 15s trunk passed +1 checkstyle 1m 49s trunk passed +1 mvnsite 1m 36s trunk passed +1 mvneclipse 0m 31s trunk passed +1 findbugs 2m 20s trunk passed +1 javadoc 1m 20s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 8m 12s the patch passed +1 javac 8m 12s the patch passed +1 checkstyle 1m 35s the patch passed +1 mvnsite 1m 29s the patch passed +1 mvneclipse 0m 38s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 24s the patch passed +1 javadoc 1m 10s the patch passed -1 unit 8m 10s hadoop-common in the patch failed. +1 unit 0m 23s hadoop-aws in the patch passed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 76m 49s Reason Tests Failed junit tests hadoop.security.TestGroupsCaching   hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818073/HADOOP-12726.003.patch JIRA Issue HADOOP-12726 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e88010ef0090 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e549a9a Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10007/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10007/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10007/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Checked with Steve Loughran offline.

          I committed this. Thanks Daniel

          Show
          chris.douglas Chris Douglas added a comment - Checked with Steve Loughran offline. I committed this. Thanks Daniel
          Hide
          templedf Daniel Templeton added a comment -
          Show
          templedf Daniel Templeton added a comment - Thanks, Chris Douglas and Steve Loughran !
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10332 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10332/)
          HADOOP-12726. Unsupported FS operations should throw (cdouglas: rev c37346d0e3f9d39d0aec7a9c5bda3e9772aa969b)

          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/RollingFileSystemSink.java
          • (edit) hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FTPFileSystem.java
          • (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFs.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10332 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10332/ ) HADOOP-12726 . Unsupported FS operations should throw (cdouglas: rev c37346d0e3f9d39d0aec7a9c5bda3e9772aa969b) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/RollingFileSystemSink.java (edit) hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ftp/FTPFileSystem.java (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFs.java
          Hide
          kihwal Kihwal Lee added a comment -

          I think this broke MAPREDUCE-6767.

          Show
          kihwal Kihwal Lee added a comment - I think this broke MAPREDUCE-6767 .

            People

            • Assignee:
              templedf Daniel Templeton
              Reporter:
              templedf Daniel Templeton
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development