[YARN-1111] NM containerlogs servlet can't handle logs of more than a GB

• Type: Sub-task
• Status: Closed
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 2.1.0-beta
• Fix Version/s: 2.6.0
• Component/s: nodemanager

Environment

Long-lived service generating lots of log data from HBase running in debug level

• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 28/Aug/13 00:29
• Updated: 01/Dec/14 03:10
• Resolved: 12/Nov/14 13:59

Description

If a container is set up to log stdout to a file, the container log servlet will list the file

err.txt : Total file length is 551 bytes.
out.txt : Total file length is 1572099246 bytes.

If you actually click on out.txt then the tail logic takes a very long time to react. There is also the question of what will happen if the log fills up that volume

Issue Links

• is depended upon by: YARN-896 Roll up for long-lived services in YARN
• is related to: YARN-173 Page navigation support for container logs page and the logs web-service on NMs
• is related to: YARN-2082 Support for alternative log aggregation mechanism
• is related to: SLIDER-116 Log handling for long-lived applications
• relates to: YARN-1104 NMs to support rolling logs of stdout & stderr
• relates to: YARN-2468 Log handling for LRS