Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6857

Reduce number of exists() calls on the target object

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0-alpha2
    • Fix Version/s: None
    • Component/s: distcp
    • Labels:
      None
    • Target Version/s:

      Description

      CopyMapper.map(..) calls targetStatus = targetFS.getFileStatus(target).
      Few steps later RetriableFileCopyCommand.promoteTmpToTarget(..) will call again exists(target) and delete if present.
      The second exists() is useless, since if targetStatus==null it can be easily seen if overwrite mode is activated and so target object can be deleted.

      The propose of this patch is to delete target object by using targetStatus and thus avoid calling exists() method.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              gvernik Gil Vernik
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: