Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4540

Yarn property "yarn.nodemanager.localizer.cache.cleanup.interval-ms" not working as expected

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.7.1
    • Fix Version/s: None
    • Component/s: nodemanager, yarn
    • Labels:
      None
    • Environment:

      Ambari – Version 2.1.2.1
      YARN - Version 2.7.1.2.3
      HDP - Version 2.3.2.0

      Description

      I manually add specific YARN configuration to reduce my YARN cache size, however it does not get updated for the interval specified by:

      yarn.nodemanager.localizer.cache.cleanup.interval-ms
      

      I did notice that eventually my server did reduce my YARN cache size over the weekend.

      Set Values (Should update size in 1 minute):
      yarn.nodemanager.localizer.cache.target-size-mb=4096
      yarn.nodemanager.localizer.cache.cleanup.interval-ms=60000
      
      Default values (Should update size in 10 minutes):
      yarn.nodemanager.localizer.cache.target-size-mb=10240
      yarn.nodemanager.localizer.cache.cleanup.interval-ms=600000
      
      Confirming YARN cache size
      date && du -m /hadoop/yarn | sort -nr | head -n 20
      #date && du -m . | sort -nr | head -n 20
      
      See also:

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              marks1900 Mark S
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified