Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-512

Log aggregation root directory check is more expensive than it needs to be

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta
    • Fix Version/s: 2.1.0-beta
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The log aggregation root directory check first does an exists call followed by a getFileStatus call. That effectively stats the file twice. It should just use getFileStatus and catch FileNotFoundException to handle the non-existent case.

      In addition we may consider caching the presence of the directory rather than checking it each time a node aggregates logs for an application.

        Attachments

        1. YARN-512.patch
          5 kB
          Maysam Yabandeh

          Activity

            People

            • Assignee:
              maysamyabandeh Maysam Yabandeh
              Reporter:
              jlowe Jason Darrell Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: