Hadoop Common
  1. Hadoop Common
  2. HADOOP-4709

Update chukwa source code and develop chukwa from public repository

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Redhat EL 4.5, Java 1.6, Hadoop trunk

    • Release Note:
      Hide
      What is new in HADOOP-4709:

        Dashboard for visual Hadoop metrics and system metrics.
        Database loader for loading chukwa records to jdbc connected database.
        FileAdaptor for streaming small file as one record for data processing.
        Scripts for capturing tcpdump output and stream to Chukwa for archive.

      Improvement

        New load blanace algorithm for ChukwaAgent to distribute data to Chukwa Collectors more evenly.
        Unit test and validation for agent, collector, and demux map reduce job.

      Contributed by Jerome Boulon, Andy Konwinski, Ariel Rabkin, Eric Yang
      Show
      What is new in HADOOP-4709 :   Dashboard for visual Hadoop metrics and system metrics.   Database loader for loading chukwa records to jdbc connected database.   FileAdaptor for streaming small file as one record for data processing.   Scripts for capturing tcpdump output and stream to Chukwa for archive. Improvement   New load blanace algorithm for ChukwaAgent to distribute data to Chukwa Collectors more evenly.   Unit test and validation for agent, collector, and demux map reduce job. Contributed by Jerome Boulon, Andy Konwinski, Ariel Rabkin, Eric Yang

      Description

      New features that were added to Chukwa source code since last time chukwa was committed to SVN repository:

      • Added Hadoop Infrastructure Care Center (UI for visualize data collected by Chukwa)
      • Added FileAdaptor for streaming small file in one chunk
      • Added compression to archive and demux output
      • Added unit tests and validation for agent, collector, and demux map reduce job
      • Added database loader for loading demux output (sequence file) to jdbc connected database
      • Added algorithm to distribute collector load more evenly
      1. HADOOP-4709-3.patch
        1.85 MB
        Eric Yang
      2. HADOOP-4709-binary.tar.gz
        60 kB
        Eric Yang

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Eric Yang
              Reporter:
              Eric Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development