Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9542

Fix LogsCLI guessAppOwner ignores custom file format suffix

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.2.0, 3.1.2
    • 3.2.1, 3.1.3
    • log-aggregation
    • None

    Description

      LogsCLI guessAppOwner ignores custom file format suffix yarn.log-aggregation.%s.remote-app-log-dir-suffix / Default IndexedFileController Suffix (

      {yarn.nodemanager.remote-app-log-dir-suffix}

      -ifile or logs-ifile). It considers only yarn.nodemanager.remote-app-log-dir-suffix or default logs.

      Repro:

      
      yarn-site.xml
      
      yarn.log-aggregation.file-formats ifile
      yarn.log-aggregation.file-controller.ifile.class org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.LogAggregationIndexedFileController
      yarn.log-aggregation.ifile.remote-app-log-dir app-logs
      yarn.resourcemanager.connect.max-wait.ms 1000
      
      core-site.xml:
      
      ipc.client.connect.max.retries 3
      ipc.client.connect.retry.interval 10
      
      Run a Job with above configs and Stop the RM.
      
      [ambari-qa@yarn-ats-1 ~]$ yarn logs -applicationId application_1557482389195_0001
      2019-05-10 10:03:58,215 INFO client.RMProxy: Connecting to ResourceManager at yarn-ats-1/172.26.81.91:8050
      Unable to get ApplicationState. Attempting to fetch logs directly from the filesystem.
      Can not find the appOwner. Please specify the correct appOwner
      Could not locate application logs for application_1557482389195_0001
      
      
      [ambari-qa@yarn-ats-1 ~]$ hadoop fs -ls /app-logs/ambari-qa/logs-ifile/application_1557482389195_0001
      Found 1 items
      -rw-r-----   3 ambari-qa supergroup      18058 2019-05-10 10:01 /app-logs/ambari-qa/logs-ifile/application_1557482389195_0001/yarn-ats-1_45454
      
      

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            prabhujoseph Prabhu Joseph Assign to me
            prabhujoseph Prabhu Joseph
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment