Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1302

TrackerDistributedCacheManager can delete file asynchronously

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2, 0.21.0, 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      With the help of AsyncDiskService from MAPREDUCE-1213, we should be able to delete files from distributed cache asynchronously.

      That will help make task initialization faster, because task initialization calls the code that localizes files into the cache and may delete some other files.
      The deletion can slow down the task initialization speed.

      1. MAPREDUCE-1302.branch-0.20.on.top.of.MAPREDUCE-1213.3.patch
        42 kB
        Zheng Shao
      2. MAPREDUCE-1302.branch-0.20.on.top.of.MAPREDUCE-1213.2.patch
        42 kB
        Zheng Shao
      3. MAPREDUCE-1302.5.patch
        26 kB
        Zheng Shao
      4. MAPREDUCE-1302.4.patch
        26 kB
        Zheng Shao
      5. MAPREDUCE-1302.3.patch
        22 kB
        Zheng Shao
      6. MAPREDUCE-1302.2.patch
        21 kB
        Zheng Shao
      7. MAPREDUCE-1302.1.patch
        8 kB
        Zheng Shao
      8. MAPREDUCE-1302.0.patch
        19 kB
        Zheng Shao

        Issue Links

          Activity

          Zheng Shao created issue -
          Zheng Shao made changes -
          Field Original Value New Value
          Attachment MAPREDUCE-1302.0.patch [ 12428138 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue duplicates MAPREDUCE-1141 [ MAPREDUCE-1141 ]
          Vinod Kumar Vavilapalli made changes -
          Affects Version/s 0.20.2 [ 12314205 ]
          Affects Version/s 0.21.0 [ 12314045 ]
          Affects Version/s 0.22.0 [ 12314184 ]
          Component/s tasktracker [ 12312906 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue is blocked by MAPREDUCE-1213 [ MAPREDUCE-1213 ]
          Zheng Shao made changes -
          Attachment MAPREDUCE-1302.1.patch [ 12428349 ]
          Zheng Shao made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Zheng Shao made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Zheng Shao made changes -
          Attachment MAPREDUCE-1302.2.patch [ 12429084 ]
          Zheng Shao made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Zheng Shao made changes -
          Attachment MAPREDUCE-1302.3.patch [ 12429498 ]
          Zheng Shao made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Zheng Shao made changes -
          Attachment MAPREDUCE-1302.4.patch [ 12429575 ]
          Zheng Shao made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Zheng Shao made changes -
          Attachment MAPREDUCE-1302.5.patch [ 12429929 ]
          Zheng Shao made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          dhruba borthakur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.22.0 [ 12314184 ]
          Resolution Fixed [ 1 ]
          Zheng Shao made changes -
          Attachment MAPREDUCE-1302.branch-0.20.on.top.of.MAPREDUCE-1213.patch [ 12430070 ]
          Zheng Shao made changes -
          Zheng Shao made changes -
          Attachment MAPREDUCE-1302.branch-0.20.on.top.of.MAPREDUCE-1213.patch [ 12430070 ]
          Zheng Shao made changes -
          Tom White made changes -
          Fix Version/s 0.21.0 [ 12314045 ]
          Fix Version/s 0.22.0 [ 12314184 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Zheng Shao
              Reporter:
              Zheng Shao
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development