Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6490

Path.normalize should use StringUtils.replace in favor of String.replace

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.20.1
    • 0.23.1
    • fs
    • Reviewed
    • path

    Description

      in our environment, we are seeing that the JobClient is going out of memory because Path.normalizePath(String) is called several tens of thousands of times, and each time it calls "String.replace" twice.

      java.lang.String.replace compiles a regex to do the job which is very costly.
      We should use org.apache.commons.lang.StringUtils.replace which is much faster and consumes almost no extra memory.

      Attachments

        1. HADOOP-6490.patch
          1 kB
          Uma Maheswara Rao G

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            umamaheswararao Uma Maheswara Rao G
            zshao Zheng Shao
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment