Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4433

Improve data loader for collecting metrics and log files from hadoop and system

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Red Hat Enterprise Linux 4.5, 5, Java 6

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      - Added startup and shutdown script
      - Added torque metrics data loader
      - Improve handling of Exec Plugin
      - Added Test cases for File Tailing Adaptors
      - Added Test cases for Start streaming at specific offset
      Show
      - Added startup and shutdown script - Added torque metrics data loader - Improve handling of Exec Plugin - Added Test cases for File Tailing Adaptors - Added Test cases for Start streaming at specific offset

      Description

      Enhancement should include:

      • Ability to stream /var/log/messages
      • Better handling of startup and shutdown chukwa processes
      • Ability to monitor torque/hod allocated clusters
      • Add serialization and deserialization of hadoop metrics to JSON log files.

        Attachments

        1. HADOOP-4433.patch
          131 kB
          Eric Yang
        2. HADOOP-4433-2.patch
          127 kB
          Eric Yang

          Issue Links

          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-4431 Add versionning/tags to Chukwa Chunk

          • Major - Major loss of function.
          • Closed
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-4459 - Improve Chukwa Collector

          • Major - Major loss of function.
          • Closed

            Activity

              People

              • Assignee:
                eyang Eric Yang
                Reporter:
                eyang Eric Yang
              • Votes:
                1 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 96h
                  96h
                  Remaining:
                  Remaining Estimate - 96h
                  96h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified