Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4709

Update chukwa source code and develop chukwa from public repository

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Redhat EL 4.5, Java 1.6, Hadoop trunk

    • Release Note:
      Hide
      What is new in HADOOP-4709:

        Dashboard for visual Hadoop metrics and system metrics.
        Database loader for loading chukwa records to jdbc connected database.
        FileAdaptor for streaming small file as one record for data processing.
        Scripts for capturing tcpdump output and stream to Chukwa for archive.

      Improvement

        New load blanace algorithm for ChukwaAgent to distribute data to Chukwa Collectors more evenly.
        Unit test and validation for agent, collector, and demux map reduce job.

      Contributed by Jerome Boulon, Andy Konwinski, Ariel Rabkin, Eric Yang
      Show
      What is new in HADOOP-4709 :   Dashboard for visual Hadoop metrics and system metrics.   Database loader for loading chukwa records to jdbc connected database.   FileAdaptor for streaming small file as one record for data processing.   Scripts for capturing tcpdump output and stream to Chukwa for archive. Improvement   New load blanace algorithm for ChukwaAgent to distribute data to Chukwa Collectors more evenly.   Unit test and validation for agent, collector, and demux map reduce job. Contributed by Jerome Boulon, Andy Konwinski, Ariel Rabkin, Eric Yang

      Description

      New features that were added to Chukwa source code since last time chukwa was committed to SVN repository:

      • Added Hadoop Infrastructure Care Center (UI for visualize data collected by Chukwa)
      • Added FileAdaptor for streaming small file in one chunk
      • Added compression to archive and demux output
      • Added unit tests and validation for agent, collector, and demux map reduce job
      • Added database loader for loading demux output (sequence file) to jdbc connected database
      • Added algorithm to distribute collector load more evenly

        Attachments

        1. HADOOP-4709-binary.tar.gz
          60 kB
          Eric Yang
        2. HADOOP-4709-3.patch
          1.85 MB
          Eric Yang

          Issue Links

            Activity

              People

              • Assignee:
                eyang Eric Yang
                Reporter:
                eyang Eric Yang
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: