Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2443 Handling logs of long-running services on YARN
  3. YARN-3154

Should not upload partial logs for MR jobs or other "short-running' applications

          Details

          • Target Version/s:
          • Hadoop Flags:
            Incompatible change, Reviewed
          • Release Note:
            Applications which made use of the LogAggregationContext in their application will need to revisit this code in order to make sure that their logs continue to get rolled out.

            Description

            Currently, if we are running a MR job, and we do not set the log interval properly, we will have their partial logs uploaded and then removed from the local filesystem which is not right.

            We only upload the partial logs for LRS applications.

              Attachments

              1. YARN-3154.1.patch
                22 kB
                Xuan Gong
              2. YARN-3154.2.patch
                22 kB
                Xuan Gong
              3. YARN-3154.3.patch
                26 kB
                Xuan Gong
              4. YARN-3154.4.patch
                27 kB
                Xuan Gong

                Issue Links

                relates to

                Bug - A problem which impairs or prevents the functions of the product. YARN-3089 LinuxContainerExecutor does not handle file arguments to deleteAsUser

                • Blocker - Blocks development and/or testing work, production could not run
                • Closed

                  Activity

                    People

                    • Assignee:
                      xgong Xuan Gong
                      Reporter:
                      xgong Xuan Gong
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      11 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: