Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6490

Path.normalize should use StringUtils.replace in favor of String.replace

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.23.1
    • Component/s: fs
    • Labels:
    • Hadoop Flags:
      Reviewed
    • Tags:
      path

      Description

      in our environment, we are seeing that the JobClient is going out of memory because Path.normalizePath(String) is called several tens of thousands of times, and each time it calls "String.replace" twice.

      java.lang.String.replace compiles a regex to do the job which is very costly.
      We should use org.apache.commons.lang.StringUtils.replace which is much faster and consumes almost no extra memory.

        Attachments

        1. HADOOP-6490.patch
          1 kB
          Uma Maheswara Rao G

          Activity

            People

            • Assignee:
              umamaheswararao Uma Maheswara Rao G
              Reporter:
              zshao Zheng Shao
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: