Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6857

Reduce number of exists() calls on the target object

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.0.0-alpha2
    • None
    • distcp
    • None

    Description

      CopyMapper.map(..) calls targetStatus = targetFS.getFileStatus(target).
      Few steps later RetriableFileCopyCommand.promoteTmpToTarget(..) will call again exists(target) and delete if present.
      The second exists() is useless, since if targetStatus==null it can be easily seen if overwrite mode is activated and so target object can be deleted.

      The propose of this patch is to delete target object by using targetStatus and thus avoid calling exists() method.

      Attachments

        1. HADOOP-6857-002.patch
          3 kB
          Gil Vernik

        Activity

          People

            Unassigned Unassigned
            gvernik Gil Vernik
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: