Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3025

ChecksumFileSystem needs to support the new delete method

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.17.0
    • 0.17.0
    • fs
    • None

    Description

      The method FileSystem.delete(path) has been deprecated in favor of the new method delete(path, recursive). Temporary files gets created in the MapReduce framework and when the time for deletion comes, they are deleted via delete(path, recursive). This doesn't delete the associated checksum files. This has a big impact when the FileSystem is the InMemoryFileSystem, where space is at a premium and wasting space here might hurt the performance of MapReduce jobs overall. One solution to this problem is to implement the method delete(path, recursive) in the ChecksumFileSystem but is there is a reason why it was left out as part of HADOOP-771?

      Attachments

        1. HADOOP_3025_1.patch
          1 kB
          Mahadev Konar
        2. HADOOP_3025_2.patch
          1 kB
          Mahadev Konar
        3. HADOOP_3025_3.patch
          6 kB
          Mahadev Konar
        4. HADOOP_3025_4.patch
          7 kB
          Mahadev Konar

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mahadev Mahadev Konar
            ddas Devaraj Das
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment