Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9542

Fix LogsCLI guessAppOwner ignores custom file format suffix

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.2.0, 3.1.2
    • Fix Version/s: 3.2.1, 3.1.3
    • Component/s: log-aggregation
    • Labels:
      None

      Description

      LogsCLI guessAppOwner ignores custom file format suffix yarn.log-aggregation.%s.remote-app-log-dir-suffix / Default IndexedFileController Suffix (

      {yarn.nodemanager.remote-app-log-dir-suffix}

      -ifile or logs-ifile). It considers only yarn.nodemanager.remote-app-log-dir-suffix or default logs.

      Repro:

      
      yarn-site.xml
      
      yarn.log-aggregation.file-formats ifile
      yarn.log-aggregation.file-controller.ifile.class org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.LogAggregationIndexedFileController
      yarn.log-aggregation.ifile.remote-app-log-dir app-logs
      yarn.resourcemanager.connect.max-wait.ms 1000
      
      core-site.xml:
      
      ipc.client.connect.max.retries 3
      ipc.client.connect.retry.interval 10
      
      Run a Job with above configs and Stop the RM.
      
      [ambari-qa@yarn-ats-1 ~]$ yarn logs -applicationId application_1557482389195_0001
      2019-05-10 10:03:58,215 INFO client.RMProxy: Connecting to ResourceManager at yarn-ats-1/172.26.81.91:8050
      Unable to get ApplicationState. Attempting to fetch logs directly from the filesystem.
      Can not find the appOwner. Please specify the correct appOwner
      Could not locate application logs for application_1557482389195_0001
      
      
      [ambari-qa@yarn-ats-1 ~]$ hadoop fs -ls /app-logs/ambari-qa/logs-ifile/application_1557482389195_0001
      Found 1 items
      -rw-r-----   3 ambari-qa supergroup      18058 2019-05-10 10:01 /app-logs/ambari-qa/logs-ifile/application_1557482389195_0001/yarn-ats-1_45454
      
      

        Attachments

        1. YARN-9542-001.patch
          9 kB
          Prabhu Joseph
        2. YARN-9542-branch-3.2.001.patch
          28 kB
          Prabhu Joseph
        3. YARN-9542-branch-3.2.002.patch
          28 kB
          Prabhu Joseph

          Issue Links

            Activity

              People

              • Assignee:
                prabhujoseph Prabhu Joseph
                Reporter:
                prabhujoseph Prabhu Joseph
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: