Hadoop Common
  1. Hadoop Common
  2. HADOOP-6890

Improve listFiles API introduced by HADOOP-6870

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This jira is mainly for addressing Suresh's review comments for HADOOP-6870:

      1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS.
      2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it?
      3. FileSystem.java

      • listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false.
      • listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown.
        4. TestListFiles.java
      • testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test.
      1. improveListFiles2.patch
        12 kB
        Hairong Kuang
      2. improveListFiles1.patch
        12 kB
        Hairong Kuang
      3. improveListFiles.patch
        11 kB
        Hairong Kuang
      4. listFilesInFS.patch
        8 kB
        Hairong Kuang

        Issue Links

          Activity

          Hairong Kuang created issue -
          Hairong Kuang made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-6870 [ HADOOP-6870 ]
          Hairong Kuang made changes -
          Attachment improveListFiles.patch [ 12450921 ]
          Hairong Kuang made changes -
          Attachment improveListFiles.patch [ 12450921 ]
          Hairong Kuang made changes -
          Attachment improveListFiles.patch [ 12450922 ]
          Hairong Kuang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hairong Kuang made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Hairong Kuang made changes -
          Attachment listFilesInFS.patch [ 12450938 ]
          Hairong Kuang made changes -
          Attachment improveListFiles.patch [ 12451054 ]
          Hairong Kuang made changes -
          Attachment improveListFiles.patch [ 12451054 ]
          Hairong Kuang made changes -
          Attachment improveListFiles.patch [ 12450922 ]
          Hairong Kuang made changes -
          Attachment improveListFiles.patch [ 12451055 ]
          Hairong Kuang made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hairong Kuang made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Hairong Kuang made changes -
          Attachment improveListFiles.patch [ 12451055 ]
          Hairong Kuang made changes -
          Attachment improveListFiles.patch [ 12451056 ]
          Hairong Kuang made changes -
          Attachment improveListFiles1.patch [ 12451058 ]
          Hairong Kuang made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hairong Kuang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hairong Kuang made changes -
          Attachment improveListFiles1.patch [ 12451060 ]
          Hairong Kuang made changes -
          Attachment improveListFiles1.patch [ 12451060 ]
          Hairong Kuang made changes -
          Attachment improveListFiles2.patch [ 12451066 ]
          Hairong Kuang made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hairong Kuang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hairong Kuang made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Hairong Kuang
              Reporter:
              Hairong Kuang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development