Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2594

Close Async HBase Client if there are large number of consecutive timeouts

        Details

        • Type: Bug
        • Status: Resolved
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: 1.6.0
        • Component/s: None
        • Labels:
          None

          Description

          A large number of timeouts can lead to several thousands of messages being buffered but never removed from the HBaseClient's buffers. This can cause a major heap spike. So we should close and dereference the HBaseClient if we hit many failures from the HBase side to clear up the buffers.

          1. Text File
            FLUME-2594.patch
            6 kB
            Hari Shreedharan

            Issue Links

            links to

            Web Link Review

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              jarcec Jarek Jarcec Cecho added a comment -

              +1

              Show
              jarcec Jarek Jarcec Cecho added a comment - +1
              Hide
              Permalink
              jira-bot ASF subversion and git services added a comment -

              Commit 82631f811a7c6a4d8e6ec886f80d1a21876947e5 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho
              [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=82631f8 ]

              FLUME-2594: Close Async HBase Client if there are large number of consecutive timeouts

              (Hari Shreedharan via Jarek Jarcec Cecho)

              Show
              jira-bot ASF subversion and git services added a comment - Commit 82631f811a7c6a4d8e6ec886f80d1a21876947e5 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=82631f8 ] FLUME-2594 : Close Async HBase Client if there are large number of consecutive timeouts (Hari Shreedharan via Jarek Jarcec Cecho)
              Hide
              Permalink
              jira-bot ASF subversion and git services added a comment -

              Commit da294a164c27d400dc0b7bc6754c883a4eb5ed24 in flume's branch refs/heads/flume-1.6 from Jarek Jarcec Cecho
              [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=da294a1 ]

              FLUME-2594: Close Async HBase Client if there are large number of consecutive timeouts

              (Hari Shreedharan via Jarek Jarcec Cecho)

              Show
              jira-bot ASF subversion and git services added a comment - Commit da294a164c27d400dc0b7bc6754c883a4eb5ed24 in flume's branch refs/heads/flume-1.6 from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=da294a1 ] FLUME-2594 : Close Async HBase Client if there are large number of consecutive timeouts (Hari Shreedharan via Jarek Jarcec Cecho)
              Hide
              Permalink
              jarcec Jarek Jarcec Cecho added a comment -

              Thank you for the contribution Hari Shreedharan!

              Show
              jarcec Jarek Jarcec Cecho added a comment - Thank you for the contribution Hari Shreedharan !
              Hide
              Permalink
              hudson Hudson added a comment -

              SUCCESS: Integrated in flume-trunk #708 (See https://builds.apache.org/job/flume-trunk/708/)
              FLUME-2594: Close Async HBase Client if there are large number of consecutive timeouts (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=82631f811a7c6a4d8e6ec886f80d1a21876947e5)

              • flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java
              Show
              hudson Hudson added a comment - SUCCESS: Integrated in flume-trunk #708 (See https://builds.apache.org/job/flume-trunk/708/ ) FLUME-2594 : Close Async HBase Client if there are large number of consecutive timeouts (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=82631f811a7c6a4d8e6ec886f80d1a21876947e5 ) flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java
              Hide
              Permalink
              hudson Hudson added a comment -

              UNSTABLE: Integrated in Flume-trunk-hbase-98 #65 (See https://builds.apache.org/job/Flume-trunk-hbase-98/65/)
              FLUME-2594: Close Async HBase Client if there are large number of consecutive timeouts (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=82631f811a7c6a4d8e6ec886f80d1a21876947e5)

              • flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java
              Show
              hudson Hudson added a comment - UNSTABLE: Integrated in Flume-trunk-hbase-98 #65 (See https://builds.apache.org/job/Flume-trunk-hbase-98/65/ ) FLUME-2594 : Close Async HBase Client if there are large number of consecutive timeouts (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=82631f811a7c6a4d8e6ec886f80d1a21876947e5 ) flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java

                People

                • Assignee:
                  hshreedharan Hari Shreedharan
                  Reporter:
                  hshreedharan Hari Shreedharan
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development