Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6890

Improve listFiles API introduced by HADOOP-6870

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.22.0
    • 0.22.0
    • fs
    • None
    • Reviewed

    Description

      This jira is mainly for addressing Suresh's review comments for HADOOP-6870:

      1. General comment: I have concerns about recursive listing. This could be abused by the applications, creating a lot of requests into HDFS.
      2. Any deletion of files/directories while reursing through directories results in RuntimeException and application has a partial result. Should we ignore if a directory was in stack and was not found later when iterating through it?
      3. FileSystem.java

      • listFile() - method javadoc could be better organized - first write about if path is directory and two cases recursive=true and false. Then if path is file and two cases recursive=true or false.
      • listFile() - document throwing RuntimeException, UnsupportedOperationException and the possible cause. IOException is no longer thrown.
        4. TestListFiles.java
      • testDirectory() - comments test empty directory and test directory with 1 file should be moved up to relevant sections of the test.

      Attachments

        1. improveListFiles.patch
          11 kB
          Hairong Kuang
        2. improveListFiles1.patch
          12 kB
          Hairong Kuang
        3. improveListFiles2.patch
          12 kB
          Hairong Kuang
        4. listFilesInFS.patch
          8 kB
          Hairong Kuang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            hairong Hairong Kuang
            hairong Hairong Kuang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment