[YARN-2777] Mark the end of individual log in aggregated log

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.7.0
• Labels: log-aggregation
• Target Version/s: 2.7.0
• Assignee: Varun Saxena
• Reporter: Ted Yu
• Created: 30/Oct/14 17:01
• Updated: 10/Apr/15 20:10
• Resolved: 20/Mar/15 17:29

Description

Below is snippet of aggregated log showing hbase master log:

LogType: hbase-hbase-master-ip-172-31-34-167.log
LogUploadTime: 29-Oct-2014 22:31:55
LogLength: 24103045
Log Contents:
Wed Oct 29 15:43:57 UTC 2014 Starting master on ip-172-31-34-167
...
  at org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:124)
  at org.apache.hadoop.hbase.Chore.run(Chore.java:80)
  at java.lang.Thread.run(Thread.java:745)

LogType: hbase-hbase-master-ip-172-31-34-167.out

Since logs from various daemons are aggregated in one log file, it would be desirable to mark the end of one log before starting with the next.
e.g. with such a line:

End of LogType: hbase-hbase-master-ip-172-31-34-167.log

Issue Links

• breaks: YARN-3384 TestLogAggregationService.verifyContainerLogs fails after YARN-2777