Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-817

HBaseWriter leaks connections when parser error occurs

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: Data Collection
    • Labels:
      None

      Description

      If source data contains error, parser will fail and HBaseWriter would throw WriterException without closing existing connections. New connection will be initiated for new data. This leads to a slow silent leak of connections.

        Activity

        Hide
        eyang Eric Yang added a comment -

        Record the parse error entry to agent log, and continue to send data. Connection is only replaced when HBase error out.

        Show
        eyang Eric Yang added a comment - Record the parse error entry to agent log, and continue to send data. Connection is only replaced when HBase error out.
        Hide
        eyang Eric Yang added a comment -

        I just resolved this.

        Show
        eyang Eric Yang added a comment - I just resolved this.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Chukwa-master #580 (See https://builds.apache.org/job/Chukwa-master/580/)
        CHUKWA-817. Prevent connections leaks when HBaseWriter or parser (eyang: rev 971641927fd399143329bcc0480e0f1e6b032a74)

        • (edit) CHANGES.txt
        • (edit) src/main/java/org/apache/hadoop/chukwa/extraction/hbase/HadoopMetricsProcessor.java
        • (edit) src/main/java/org/apache/hadoop/chukwa/datacollection/writer/WriterException.java
        • (edit) src/main/java/org/apache/hadoop/chukwa/datacollection/writer/hbase/HBaseWriter.java
        • (edit) src/main/java/org/apache/hadoop/chukwa/datacollection/writer/PipelineStageWriter.java
        • (edit) src/main/java/org/apache/hadoop/chukwa/inputtools/log4j/Log4jMetricsSink.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Chukwa-master #580 (See https://builds.apache.org/job/Chukwa-master/580/ ) CHUKWA-817 . Prevent connections leaks when HBaseWriter or parser (eyang: rev 971641927fd399143329bcc0480e0f1e6b032a74) (edit) CHANGES.txt (edit) src/main/java/org/apache/hadoop/chukwa/extraction/hbase/HadoopMetricsProcessor.java (edit) src/main/java/org/apache/hadoop/chukwa/datacollection/writer/WriterException.java (edit) src/main/java/org/apache/hadoop/chukwa/datacollection/writer/hbase/HBaseWriter.java (edit) src/main/java/org/apache/hadoop/chukwa/datacollection/writer/PipelineStageWriter.java (edit) src/main/java/org/apache/hadoop/chukwa/inputtools/log4j/Log4jMetricsSink.java

          People

          • Assignee:
            eyang Eric Yang
            Reporter:
            eyang Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development