[OOZIE-1737] Oozie log streaming is slow

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.1.0
• Assignee: Purshotam Shah
• Reporter: Purshotam Shah
• Created: 12/Mar/14 22:24
• Updated: 09/Dec/14 00:45
• Resolved: 16/May/14 16:08

Description

Most of cases, Oozie search huge set of log files and return lot of logs.
Most of our users has complained that log streaming is slow and they can't easily find relevant information.

To make log streaming fast and precise we should provide few options (which user can specify for log streaming) like
1. Start time : Start date (if number, then offset(in hour) from coord/wf/bundle start)
2. End time : End date (if number, then offset from coord/wf/bundle start)
3. Log level :
Setting log level to ERROR, will only give erred log and exception.
In most of cases user want to find out error message and exception.
4. Number of lines : limit number of lines of logs from start date.
5. Recent logs: In most of the case user is only interested in recent activity.
Similar to https://issues.apache.org/jira/browse/OOZIE-1716.
User should be able to specify recent logs in hours.
6. Grep for pattern.

+ in addition to this, we also need to clean up log statements( which may also include consolidating log statements).

Issue Links

• is duplicated by: OOZIE-1716 Add a way to stream the logs for only the last x hours