Hadoop Common
  1. Hadoop Common
  2. HADOOP-3025

ChecksumFileSystem needs to support the new delete method

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.17.0
    • Fix Version/s: 0.17.0
    • Component/s: fs
    • Labels:
      None

      Description

      The method FileSystem.delete(path) has been deprecated in favor of the new method delete(path, recursive). Temporary files gets created in the MapReduce framework and when the time for deletion comes, they are deleted via delete(path, recursive). This doesn't delete the associated checksum files. This has a big impact when the FileSystem is the InMemoryFileSystem, where space is at a premium and wasting space here might hurt the performance of MapReduce jobs overall. One solution to this problem is to implement the method delete(path, recursive) in the ChecksumFileSystem but is there is a reason why it was left out as part of HADOOP-771?

      1. HADOOP_3025_1.patch
        1 kB
        Mahadev konar
      2. HADOOP_3025_2.patch
        1 kB
        Mahadev konar
      3. HADOOP_3025_3.patch
        6 kB
        Mahadev konar
      4. HADOOP_3025_4.patch
        7 kB
        Mahadev konar

        Issue Links

          Activity

          Nigel Daley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Mahadev konar made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mahadev konar made changes -
          Attachment HADOOP_3025_4.patch [ 12378157 ]
          Mahadev konar made changes -
          Attachment HADOOP_3025_3.patch [ 12378084 ]
          Mahadev konar made changes -
          Attachment HADOOP_3025_2.patch [ 12378064 ]
          Mahadev konar made changes -
          Attachment HADOOP_3025_1.patch [ 12378055 ]
          dhruba borthakur made changes -
          Link This issue is related to HADOOP-771 [ HADOOP-771 ]
          Mahadev konar made changes -
          Field Original Value New Value
          Assignee dhruba borthakur [ dhruba ] Mahadev konar [ mahadev ]
          Devaraj Das created issue -

            People

            • Assignee:
              Mahadev konar
              Reporter:
              Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development