Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-759

Configuration for collect HBase metrics and log files

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.7.0
    • Component/s: Data Collection
    • Labels:
      None

      Description

      Chukwa can collect HBase metrics and logs with some preconfigured adaptors to stream HBase information.

      1. Provide example of log4j configuration for HBase to stream logs to Chukwa.
      2. Provide HBase metrics configuration to stream metrics to Chukwa.
      3. Provide the Chukwa adaptor list which maps the HBase log streams.
      4. Demux configuration to reuse Hadoop metrics parser to process HBase data.

      Port Description
      9104 HBase Metrics
      9105 HBase Security Audit Log
      9106 HBase Master Server Log
      9107 HBase Region Server Log
      9108 HBase Thrift Server Log
      9109 HBase ZooKeeper Log
      9110 HBase REST Log

        Activity

        Hide
        eyang Eric Yang added a comment -

        Configuration for monitoring HBase.
        Usage:

        Copy hbase-log4j.properties and hadoop-metrics2-hbase.properties to HBASE_CONF_DIR.

        In addition, hbase-env.sh defines the port numbers of each data source:

        export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -Dhbase.log.port=9106"
        export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Dhbase.log.port=9107"
        export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Dhbase.log.port=9108"
        export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Dhbase.log.port=9109"
        export HBASE_REST_OPTS="$HBASE_REST_OPTS -Dhbase.log.port=9110"
        
        Show
        eyang Eric Yang added a comment - Configuration for monitoring HBase. Usage: Copy hbase-log4j.properties and hadoop-metrics2-hbase.properties to HBASE_CONF_DIR. In addition, hbase-env.sh defines the port numbers of each data source: export HBASE_MASTER_OPTS= "$HBASE_MASTER_OPTS -Dhbase.log.port=9106" export HBASE_REGIONSERVER_OPTS= "$HBASE_REGIONSERVER_OPTS -Dhbase.log.port=9107" export HBASE_THRIFT_OPTS= "$HBASE_THRIFT_OPTS -Dhbase.log.port=9108" export HBASE_ZOOKEEPER_OPTS= "$HBASE_ZOOKEEPER_OPTS -Dhbase.log.port=9109" export HBASE_REST_OPTS= "$HBASE_REST_OPTS -Dhbase.log.port=9110"
        Hide
        eyang Eric Yang added a comment -

        I just committed this.

        Show
        eyang Eric Yang added a comment - I just committed this.

          People

          • Assignee:
            eyang Eric Yang
            Reporter:
            eyang Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development