Hadoop YARN
  1. Hadoop YARN
  2. YARN-742

Log aggregation causes a lot of redundant setPermission calls

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.7, 2.0.4-alpha
    • Fix Version/s: 2.1.0-beta, 0.23.9
    • Component/s: nodemanager
    • Labels:
      None

      Description

      In one of our clusters, namenode RPC is spending 45% of its time on serving setPermission calls. Further investigation has revealed that most calls are redundantly made on /mapred/logs/<user>/logs. Also mkdirs calls are made before this.

      1. YARN-742-1.branch-0.23.patch
        15 kB
        Jason Lowe
      2. YARN-742-1.patch
        12 kB
        Jason Lowe
      3. YARN-742.patch
        12 kB
        Jason Lowe

        Activity

        Kihwal Lee created issue -
        Jason Lowe made changes -
        Field Original Value New Value
        Assignee Jason Lowe [ jlowe ]
        Jason Lowe made changes -
        Attachment YARN-742.patch [ 12585968 ]
        Jason Lowe made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Target Version/s 2.1.0-beta, 0.23.9 [ 12324029, 12324564 ]
        Jason Lowe made changes -
        Attachment YARN-742-1.patch [ 12586157 ]
        Jason Lowe made changes -
        Attachment YARN-742-1.branch-0.23.patch [ 12586204 ]
        Kihwal Lee made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 2.1.0-beta [ 12324029 ]
        Fix Version/s 0.23.9 [ 12324564 ]
        Fix Version/s 3.0.0 [ 12323268 ]
        Resolution Fixed [ 1 ]
        Kihwal Lee made changes -
        Hadoop Flags Reviewed [ 10343 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Allen Wittenauer made changes -
        Fix Version/s 3.0.0 [ 12323268 ]

          People

          • Assignee:
            Jason Lowe
            Reporter:
            Kihwal Lee
          • Votes:
            1 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development