Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9808

Zero length files in container log output haven't got a header

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.3.0
    • Component/s: log-aggregation, yarn
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Using the Yarn logs CLI for containers that have zero length files produces output similar to this:

      End of LogType:stderr
      ***********************************************************************
      
      
      End of LogType:prelaunch.err
      ******************************************************************************
      
      Container: container_e25_1567431105510_0001_01_000002 on host-1
      LogAggregationType: AGGREGATED
      =======================================================================================================
      LogType:container.log
      LogLastModifiedTime:Mon Sep 02 06:34:48 -0700 2019
      LogLength:5442
      LogContents:
      ...
      ...
      

      Note that stderr and prelaunch.err are both zero length files. Though the output is not misleading, the header is missing.

      I suggest to add the header for zero length files as well, primarily for the following reasons:

      • for applications having multiple files with the same name you may want to distinguish them by host - if many of those are of zero length, you can not extract this information from here. Note that this is a common case for stderr and prelaunch.err.
      • you may want to see the modification time (which corresponds to the creation time of the zero length file)
      • would explicitly display the "LogLength:0" line, which would avoid any confusion from end user side.

        Attachments

        1. YARN-9808.001.patch
          19 kB
          Adam Antal
        2. YARN-9808.002.patch
          46 kB
          Adam Antal
        3. YARN-9808.003.patch
          46 kB
          Adam Antal

          Issue Links

            Activity

              People

              • Assignee:
                adam.antal Adam Antal
                Reporter:
                adam.antal Adam Antal
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: