Hadoop Common
  1. Hadoop Common
  2. HADOOP-6490

Path.normalize should use StringUtils.replace in favor of String.replace

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.23.1
    • Component/s: fs
    • Labels:
    • Hadoop Flags:
      Reviewed
    • Tags:
      path

      Description

      in our environment, we are seeing that the JobClient is going out of memory because Path.normalizePath(String) is called several tens of thousands of times, and each time it calls "String.replace" twice.

      java.lang.String.replace compiles a regex to do the job which is very costly.
      We should use org.apache.commons.lang.StringUtils.replace which is much faster and consumes almost no extra memory.

      1. HADOOP-6490.patch
        1 kB
        Uma Maheswara Rao G

        Activity

          People

          • Assignee:
            Uma Maheswara Rao G
            Reporter:
            Zheng Shao
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development