Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2019

Fix all the places where Java method File.list is used with FileUtil.list API

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: datanode
    • Labels:
      None

      Description

      This new method FileUtil.list will throw an exception when disk is bad rather than returning null.

      1. HDFS-2019-1.patch
        3 kB
        Bharath Mundlapudi

        Issue Links

          Activity

          Bharath Mundlapudi created issue -
          Bharath Mundlapudi made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-7125 [ HADOOP-7125 ]
          Hide
          Bharath Mundlapudi added a comment -

          Attaching a patch.

          Show
          Bharath Mundlapudi added a comment - Attaching a patch.
          Bharath Mundlapudi made changes -
          Attachment HDFS-2019-1.patch [ 12480995 ]
          Matt Foley made changes -
          Link This issue is blocked by HADOOP-7342 [ HADOOP-7342 ]
          Hide
          Matt Foley added a comment -

          Since this issue can't pass patch-test until HADOOP-7342 is committed, I ran test-patch locally:

          [exec] -1 overall.
          [exec] +1 @author. The patch does not contain any @author tags.
          [exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
          [exec] Please justify why no new tests are needed for this patch.
          [exec] Also please list what manual steps were performed to verify this patch.
          [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
          [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
          [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
          [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
          [exec] +1 system test framework. The patch passed system test framework compile.

          These changes represent only a substitution from one API to an equivalent API with better error handling. No functionality is intended to change, so the existing suite of unit tests sufficiently tests the changes.

          +1. I will commit the patch after committing HADOOP-7342.

          Show
          Matt Foley added a comment - Since this issue can't pass patch-test until HADOOP-7342 is committed, I ran test-patch locally: [exec] -1 overall. [exec] +1 @author. The patch does not contain any @author tags. [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] +1 system test framework. The patch passed system test framework compile. These changes represent only a substitution from one API to an equivalent API with better error handling. No functionality is intended to change, so the existing suite of unit tests sufficiently tests the changes. +1. I will commit the patch after committing HADOOP-7342 .
          Hide
          Matt Foley added a comment -

          Since the above local test-patch run apparently didn't include core tests and contrib tests like the usual Hudson build, I ran them too:

          "ant -Dresolvers=internal test-contrib" built successfully.
          "ant -Dresolvers=internal test-core" had three unit test failures:

          • TestHDFSTrash timeout - passed when run with TestTrash patch (HADOOP-7326)
          • TestReplaceDatanodeOnFailure - passed when run by itself
          • TestLargeBlock timeout - also times out when run on pre-patch codebase, so not related to this patch

          Seems like a +1 here too.

          Show
          Matt Foley added a comment - Since the above local test-patch run apparently didn't include core tests and contrib tests like the usual Hudson build, I ran them too: "ant -Dresolvers=internal test-contrib" built successfully. "ant -Dresolvers=internal test-core" had three unit test failures: TestHDFSTrash timeout - passed when run with TestTrash patch ( HADOOP-7326 ) TestReplaceDatanodeOnFailure - passed when run by itself TestLargeBlock timeout - also times out when run on pre-patch codebase, so not related to this patch Seems like a +1 here too.
          Hide
          Matt Foley added a comment -

          Committed to trunk. Thanks Bharath!

          HDFS-2019 depends upon HADOOP-7342. It is possible that the HDFS trunk public build will fail until HADOOP-7342 passes through all stages of Hudson integration. This should only take a few hours. Thanks.

          Show
          Matt Foley added a comment - Committed to trunk. Thanks Bharath! HDFS-2019 depends upon HADOOP-7342 . It is possible that the HDFS trunk public build will fail until HADOOP-7342 passes through all stages of Hudson integration. This should only take a few hours. Thanks.
          Matt Foley made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #713 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/713/)
          HDFS-2019. Fix all the places where Java method File.list is used with FileUtil.list API. Contributed by Bharath Mundlapudi.

          mattf : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1131331
          Files :

          • /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java
          • /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java
          • /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
          • /hadoop/hdfs/trunk/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #713 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/713/ ) HDFS-2019 . Fix all the places where Java method File.list is used with FileUtil.list API. Contributed by Bharath Mundlapudi. mattf : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1131331 Files : /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java /hadoop/hdfs/trunk/CHANGES.txt
          Matt Foley made changes -
          Link This issue blocks HDFS-2023 [ HDFS-2023 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #688 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/688/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #688 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/688/ )
          Hide
          Eli Collins added a comment -

          Previously FSdataset#recoverDetachedBlocks just returns if the detach directory is empty, now it throws an IOException, why is that? Is it always the case that the detach directory be non-empty?

          Show
          Eli Collins added a comment - Previously FSdataset#recoverDetachedBlocks just returns if the detach directory is empty, now it throws an IOException, why is that? Is it always the case that the detach directory be non-empty?
          Hide
          Eli Collins added a comment -

          Never mind, list returns an empty list if the directory is empty. Sorry for the noise.

          Show
          Eli Collins added a comment - Never mind, list returns an empty list if the directory is empty. Sorry for the noise.
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue blocks HDFS-2023 [ HDFS-2023 ]
          Gavin made changes -
          Link This issue is depended upon by HDFS-2023 [ HDFS-2023 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          3d 11h 33m 1 Matt Foley 04/Jun/11 08:00
          Resolved Resolved Closed Closed
          163d 17h 52m 1 Arun C Murthy 15/Nov/11 00:53

            People

            • Assignee:
              Bharath Mundlapudi
              Reporter:
              Bharath Mundlapudi
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development