Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1737

Oozie log streaming is slow



    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.0
    • Component/s: None
    • Labels:


      Most of cases, Oozie search huge set of log files and return lot of logs.
      Most of our users has complained that log streaming is slow and they can't easily find relevant information.

      To make log streaming fast and precise we should provide few options (which user can specify for log streaming) like
      1. Start time : Start date (if number, then offset(in hour) from coord/wf/bundle start)
      2. End time : End date (if number, then offset from coord/wf/bundle start)
      3. Log level :
      Setting log level to ERROR, will only give erred log and exception.
      In most of cases user want to find out error message and exception.
      4. Number of lines : limit number of lines of logs from start date.
      5. Recent logs: In most of the case user is only interested in recent activity.
      Similar to https://issues.apache.org/jira/browse/OOZIE-1716.
      User should be able to specify recent logs in hours.
      6. Grep for pattern.

      + in addition to this, we also need to clean up log statements( which may also include consolidating log statements).


        1. OOZIE-1737-V8.patch
          6 kB
          Purshotam Shah
        2. OOZIE-1737-V7.patch
          136 kB
          Purshotam Shah
        3. OOZIE-1737-V6.patch
          146 kB
          Purshotam Shah
        4. OOZIE-1737-V5.patch
          146 kB
          Purshotam Shah

          Issue Links



              • Assignee:
                puru Purshotam Shah
                puru Purshotam Shah
              • Votes:
                0 Vote for this issue
                5 Start watching this issue


                • Created: