Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8463

Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. HDFS-8463.001.patch
      2 kB
      Masatake Iwasaki
    2. HDFS-8463.002.patch
      3 kB
      Masatake Iwasaki

      Activity

      Hide
      iwasakims Masatake Iwasaki added a comment -

      I think there should be guard because seekToNewSource is exposed to public via FSDataInputStream.

      I attached 001. It keeps current behaviour and just throws IOException instead of NullPointerException when seekToNewSource is called without pre-condition satisfied.

      Show
      iwasakims Masatake Iwasaki added a comment - I think there should be guard because seekToNewSource is exposed to public via FSDataInputStream. I attached 001. It keeps current behaviour and just throws IOException instead of NullPointerException when seekToNewSource is called without pre-condition satisfied.
      Hide
      hadoopqa Hadoop QA added a comment -



      +1 overall



      Vote Subsystem Runtime Comment
      0 pre-patch 14m 37s Pre-patch trunk compilation is healthy.
      +1 @author 0m 0s The patch does not contain any @author tags.
      +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
      +1 javac 7m 28s There were no new javac warning messages.
      +1 javadoc 9m 38s There were no new javadoc warning messages.
      +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
      +1 checkstyle 2m 20s There were no new checkstyle issues.
      +1 whitespace 0m 1s The patch has no lines that end in whitespace.
      +1 install 1m 34s mvn install still works.
      +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
      +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
      +1 native 3m 15s Pre-build of native portion
      +1 hdfs tests 162m 5s Tests passed in hadoop-hdfs.
          204m 59s  



      Subsystem Report/Notes
      Patch URL http://issues.apache.org/jira/secure/attachment/12735128/HDFS-8463.001.patch
      Optional Tests javadoc javac unit findbugs checkstyle
      git revision trunk / ada233b
      hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11125/artifact/patchprocess/testrun_hadoop-hdfs.txt
      Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11125/testReport/
      Java 1.7.0_55
      uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
      Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11125/console

      This message was automatically generated.

      Show
      hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 37s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 28s There were no new javac warning messages. +1 javadoc 9m 38s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 20s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 15s Pre-build of native portion +1 hdfs tests 162m 5s Tests passed in hadoop-hdfs.     204m 59s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12735128/HDFS-8463.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / ada233b hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11125/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11125/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11125/console This message was automatically generated.
      Hide
      kihwal Kihwal Lee added a comment -

      It might be better to simply call blockSeekTo(targetPos) and return true, if currentNode is null.

      Show
      kihwal Kihwal Lee added a comment - It might be better to simply call blockSeekTo(targetPos) and return true, if currentNode is null.
      Hide
      iwasakims Masatake Iwasaki added a comment -

      It might be better to simply call blockSeekTo(targetPos) and return true, if currentNode is null.

      Thanks for the comment, Kihwal Lee. I agree and attached the 002 which fixes seekToNewSource to seek to the first node if currentNode is not set yet.

      Show
      iwasakims Masatake Iwasaki added a comment - It might be better to simply call blockSeekTo(targetPos) and return true, if currentNode is null. Thanks for the comment, Kihwal Lee . I agree and attached the 002 which fixes seekToNewSource to seek to the first node if currentNode is not set yet.
      Hide
      hadoopqa Hadoop QA added a comment -



      -1 overall



      Vote Subsystem Runtime Comment
      0 pre-patch 17m 43s Pre-patch trunk compilation is healthy.
      +1 @author 0m 0s The patch does not contain any @author tags.
      +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
      +1 javac 7m 31s There were no new javac warning messages.
      +1 javadoc 9m 40s There were no new javadoc warning messages.
      +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
      +1 checkstyle 2m 15s There were no new checkstyle issues.
      +1 whitespace 0m 0s The patch has no lines that end in whitespace.
      +1 install 1m 33s mvn install still works.
      +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
      +1 findbugs 3m 16s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
      +1 native 3m 13s Pre-build of native portion
      -1 hdfs tests 178m 29s Tests failed in hadoop-hdfs.
          224m 38s  



      Reason Tests
      Failed unit tests hadoop.hdfs.server.namenode.TestFsck
      Timed out tests org.apache.hadoop.hdfs.server.namenode.TestBackupNode



      Subsystem Report/Notes
      Patch URL http://issues.apache.org/jira/secure/attachment/12736616/HDFS-8463.002.patch
      Optional Tests javadoc javac unit findbugs checkstyle
      git revision trunk / 63e3fee
      hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11191/artifact/patchprocess/testrun_hadoop-hdfs.txt
      Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11191/testReport/
      Java 1.7.0_55
      uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
      Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11191/console

      This message was automatically generated.

      Show
      hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 43s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 31s There were no new javac warning messages. +1 javadoc 9m 40s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 15s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 3m 16s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 13s Pre-build of native portion -1 hdfs tests 178m 29s Tests failed in hadoop-hdfs.     224m 38s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestFsck Timed out tests org.apache.hadoop.hdfs.server.namenode.TestBackupNode Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12736616/HDFS-8463.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 63e3fee hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11191/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11191/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11191/console This message was automatically generated.
      Hide
      iwasakims Masatake Iwasaki added a comment -

      TestBackupNode and TestFsck succeeded on my env. The code path fixed by the patch is not related to them.

      Show
      iwasakims Masatake Iwasaki added a comment - TestBackupNode and TestFsck succeeded on my env. The code path fixed by the patch is not related to them.
      Hide
      kihwal Kihwal Lee added a comment -

      +1 the latest patch looks good.

      Show
      kihwal Kihwal Lee added a comment - +1 the latest patch looks good.
      Hide
      kihwal Kihwal Lee added a comment -

      I've committed this to trunk and branch-2. Thanks for reporting and fixing this issue, Masatake.

      Show
      kihwal Kihwal Lee added a comment - I've committed this to trunk and branch-2. Thanks for reporting and fixing this issue, Masatake.
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-trunk-Commit #7966 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7966/)
      HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)

      • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
      • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
      • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7966 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7966/ ) HDFS-8463 . Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #219 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/219/)
      HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)

      • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
      • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
      • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #219 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/219/ ) HDFS-8463 . Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      Hide
      hudson Hudson added a comment -

      SUCCESS: Integrated in Hadoop-Yarn-trunk #949 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/949/)
      HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)

      • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
      • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
      Show
      hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #949 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/949/ ) HDFS-8463 . Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
      Hide
      hudson Hudson added a comment -

      SUCCESS: Integrated in Hadoop-Hdfs-trunk #2147 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2147/)
      HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)

      • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
      • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
      • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      Show
      hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2147 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2147/ ) HDFS-8463 . Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/208/)
      HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)

      • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
      • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/208/ ) HDFS-8463 . Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-Mapreduce-trunk #2165 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2165/)
      HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)

      • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
      • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
      • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2165 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2165/ ) HDFS-8463 . Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #217 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/217/)
      HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)

      • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
      • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java
      • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #217 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/217/ ) HDFS-8463 . Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. (kihwal: rev ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

        People

        • Assignee:
          iwasakims Masatake Iwasaki
          Reporter:
          iwasakims Masatake Iwasaki
        • Votes:
          0 Vote for this issue
          Watchers:
          6 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development