Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9675

Expose log aggregation diagnostic messages through RM API

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.2.0
    • Fix Version/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      The ResourceManager collects the log aggregation status reports from the NodeManagers. Currently these reports are collected, but when app info API or similar high-level REST is called, only an overall status is displayed (RUNNING, RUNNING_WITH_FAILURES,FAILED etc.).

      The diagnostic messages are only available through the old RM web UI, so our internal tool currently crawls that page and extract the log aggregation diagnostic and error messages from the raw HTML. This is not a good practice, and more elegant API call may be preferable. It may be useful for others as well since log aggregation related failures are usually hard to debug since the lack of trace/debug messages.

        Attachments

          Activity

            People

            • Assignee:
              adam.antal Adam Antal
              Reporter:
              adam.antal Adam Antal
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: