Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12344

LocatedFileStatus regression: no longer accepting null FSPermission

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-beta1
    • Fix Version/s: 3.0.0-beta1
    • Component/s: fs
    • Labels:
      None

      Description

      SPARK-21817 was opened to fix a NPE in Spark where it calls LocatedFileStatus with a null FSPermission. This breaks in current HEAD. However, the LocatedFileStatus is a stable/evolving API so this is actually a regression introduced in HDFS-6984.

      1. HDFS-12344.001.patch
        4 kB
        Ewan Higgs
      2. HDFS-12344.002.patch
        3 kB
        Ewan Higgs

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12237 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12237/)
          HDFS-12344. LocatedFileStatus regression: no longer accepting null (stevel: rev 8196a07c3211385ce85ae24f763b62696edc60b9)

          • (add) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocatedFileStatus.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocatedFileStatus.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12237 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12237/ ) HDFS-12344 . LocatedFileStatus regression: no longer accepting null (stevel: rev 8196a07c3211385ce85ae24f763b62696edc60b9) (add) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocatedFileStatus.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocatedFileStatus.java
          Hide
          stevel@apache.org Steve Loughran added a comment -

          LGTM

          +1

          committed to trunk, after fixing that whitespace complaint

          Show
          stevel@apache.org Steve Loughran added a comment - LGTM +1 committed to trunk, after fixing that whitespace complaint
          Hide
          ehiggs Ewan Higgs added a comment -

          javac issue is because i introduced a new call to a deprecated function. Of course, this is the purpose of the patch.

          Show
          ehiggs Ewan Higgs added a comment - javac issue is because i introduced a new call to a deprecated function. Of course, this is the purpose of the patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 15m 50s trunk passed
          +1 compile 16m 20s trunk passed
          +1 checkstyle 0m 36s trunk passed
          +1 mvnsite 1m 30s trunk passed
          +1 findbugs 1m 35s trunk passed
          +1 javadoc 0m 53s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 46s the patch passed
          +1 compile 12m 18s the patch passed
          -1 javac 12m 18s root generated 2 new + 1291 unchanged - 0 fixed = 1293 total (was 1291)
          -0 checkstyle 0m 38s hadoop-common-project/hadoop-common: The patch generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13)
          +1 mvnsite 1m 30s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 35s the patch passed
          +1 javadoc 0m 50s the patch passed
                Other Tests
          +1 unit 7m 33s hadoop-common in the patch passed.
          +1 asflicense 0m 29s The patch does not generate ASF License warnings.
          63m 46s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-12344
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883526/HDFS-12344.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 873abb559fba 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / de0cba7
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/20846/artifact/patchprocess/diff-compile-javac-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20846/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20846/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20846/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 15m 50s trunk passed +1 compile 16m 20s trunk passed +1 checkstyle 0m 36s trunk passed +1 mvnsite 1m 30s trunk passed +1 findbugs 1m 35s trunk passed +1 javadoc 0m 53s trunk passed       Patch Compile Tests +1 mvninstall 0m 46s the patch passed +1 compile 12m 18s the patch passed -1 javac 12m 18s root generated 2 new + 1291 unchanged - 0 fixed = 1293 total (was 1291) -0 checkstyle 0m 38s hadoop-common-project/hadoop-common: The patch generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13) +1 mvnsite 1m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 35s the patch passed +1 javadoc 0m 50s the patch passed       Other Tests +1 unit 7m 33s hadoop-common in the patch passed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 63m 46s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12344 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883526/HDFS-12344.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 873abb559fba 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / de0cba7 Default Java 1.8.0_144 findbugs v3.1.0-RC1 javac https://builds.apache.org/job/PreCommit-HDFS-Build/20846/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20846/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20846/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20846/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 15m 4s trunk passed
          +1 compile 15m 56s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 1m 31s trunk passed
          +1 findbugs 1m 34s trunk passed
          +1 javadoc 0m 51s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 41s the patch passed
          +1 compile 12m 7s the patch passed
          -1 javac 12m 7s root generated 2 new + 1291 unchanged - 0 fixed = 1293 total (was 1291)
          -0 checkstyle 0m 38s hadoop-common-project/hadoop-common: The patch generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13)
          +1 mvnsite 1m 37s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 49s the patch passed
          +1 javadoc 0m 50s the patch passed
                Other Tests
          -1 unit 8m 15s hadoop-common in the patch failed.
          +1 asflicense 0m 31s The patch does not generate ASF License warnings.
          63m 24s



          Reason Tests
          Failed junit tests hadoop.ha.TestZKFailoverController



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-12344
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883505/HDFS-12344.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 073aed1a4dff 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3123895
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/20845/artifact/patchprocess/diff-compile-javac-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20845/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/20845/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20845/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20845/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 15m 4s trunk passed +1 compile 15m 56s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 31s trunk passed +1 findbugs 1m 34s trunk passed +1 javadoc 0m 51s trunk passed       Patch Compile Tests +1 mvninstall 0m 41s the patch passed +1 compile 12m 7s the patch passed -1 javac 12m 7s root generated 2 new + 1291 unchanged - 0 fixed = 1293 total (was 1291) -0 checkstyle 0m 38s hadoop-common-project/hadoop-common: The patch generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13) +1 mvnsite 1m 37s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 49s the patch passed +1 javadoc 0m 50s the patch passed       Other Tests -1 unit 8m 15s hadoop-common in the patch failed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 63m 24s Reason Tests Failed junit tests hadoop.ha.TestZKFailoverController Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12344 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883505/HDFS-12344.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 073aed1a4dff 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3123895 Default Java 1.8.0_144 findbugs v3.1.0-RC1 javac https://builds.apache.org/job/PreCommit-HDFS-Build/20845/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20845/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20845/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20845/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20845/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ehiggs Ewan Higgs added a comment -

          Attaching updated patch based on Steve Loughran's comments.

          Show
          ehiggs Ewan Higgs added a comment - Attaching updated patch based on Steve Loughran 's comments.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          regarding the tests; get rid of the try/catch/fail blocks, that just loses a stack trace which will be needed to fix the bug. Just declare the test as throwing Exception and send it all the way up, uncaught

          Show
          stevel@apache.org Steve Loughran added a comment - regarding the tests; get rid of the try/catch/fail blocks, that just loses a stack trace which will be needed to fix the bug. Just declare the test as throwing Exception and send it all the way up, uncaught
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Ewan. hit the "submit patch" button...

          Show
          stevel@apache.org Steve Loughran added a comment - Ewan. hit the "submit patch" button...
          Hide
          ehiggs Ewan Higgs added a comment -

          Attaching a failing test and the fix. Steve Loughran and/or Chris Douglas can review.

          Show
          ehiggs Ewan Higgs added a comment - Attaching a failing test and the fix. Steve Loughran and/or Chris Douglas can review.

            People

            • Assignee:
              ehiggs Ewan Higgs
              Reporter:
              ehiggs Ewan Higgs
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development