Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-575

[Pseudo-dist config] Should yarn.nodemanager.remote-app-log-dir really be set?

Agile BoardAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • backlog
    • None
    • None

    Description

      Currently we have in pseudo-dist config, the following set:

        <property>
          <description>Where to aggregate logs to.</description>
          <name>yarn.nodemanager.remote-app-log-dir</name>
          <value>/var/log/hadoop-yarn/apps</value>
        </property>
      

      This requires the admin user (hdfs) to create:

      hdfs dfs -mkdir /var/log/hadoop-yarn/apps, which is different from the local /var/log/hadoop-yarn/apps (is that even used? I see it being created locally).

      So the two questions:

      • I think we can leave this override as the default hdfs:///tmp/logs would be easier to start with?
      • Also, is the local /var/log/hadoop-yarn/apps really required?

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            qwertymaniac Harsh J

            Dates

              Created:
              Updated:

              Slack

                Issue deployment