Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
      None

      Description

      If an optional channel is full, the queue backing the executor that is asynchronously submitting the events to the channel can grow indefinitely in size leading to a huge number of events on the heap and causing OOMs.

      1. FLUME-2886.patch
        4 kB
        Hari Shreedharan

        Activity

        Hide
        hudson Hudson added a comment -

        UNSTABLE: Integrated in Flume-trunk-hbase-1 #152 (See https://builds.apache.org/job/Flume-trunk-hbase-1/152/)
        FLUME-2891: Revert FLUME-2712 and FLUME-2886 (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=caa64a1a6d4bc97be5993cb468516e9ffe862794)

        • flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelProcessor.java
        • flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java
        Show
        hudson Hudson added a comment - UNSTABLE: Integrated in Flume-trunk-hbase-1 #152 (See https://builds.apache.org/job/Flume-trunk-hbase-1/152/ ) FLUME-2891 : Revert FLUME-2712 and FLUME-2886 (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=caa64a1a6d4bc97be5993cb468516e9ffe862794 ) flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelProcessor.java flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit c2562900f1ab9c21ae2d7c0fc26e270847e81b6d in flume's branch refs/heads/flume-1.7 from Jarek Jarcec Cecho
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=c256290 ]

        FLUME-2891: Revert FLUME-2712 and FLUME-2886

        (Hari Shreedharan via Jarek Jarcec Cecho)

        Show
        jira-bot ASF subversion and git services added a comment - Commit c2562900f1ab9c21ae2d7c0fc26e270847e81b6d in flume's branch refs/heads/flume-1.7 from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=c256290 ] FLUME-2891 : Revert FLUME-2712 and FLUME-2886 (Hari Shreedharan via Jarek Jarcec Cecho)
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit caa64a1a6d4bc97be5993cb468516e9ffe862794 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=caa64a1 ]

        FLUME-2891: Revert FLUME-2712 and FLUME-2886

        (Hari Shreedharan via Jarek Jarcec Cecho)

        Show
        jira-bot ASF subversion and git services added a comment - Commit caa64a1a6d4bc97be5993cb468516e9ffe862794 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=caa64a1 ] FLUME-2891 : Revert FLUME-2712 and FLUME-2886 (Hari Shreedharan via Jarek Jarcec Cecho)
        Hide
        hudson Hudson added a comment -

        UNSTABLE: Integrated in Flume-trunk-hbase-1 #150 (See https://builds.apache.org/job/Flume-trunk-hbase-1/150/)
        FLUME-2886: Optional Channels can cause OOMs (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=109ec30725a4c665a2ccf5f40af8a0e455cf4166)

        • flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelProcessor.java
        • flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java
        Show
        hudson Hudson added a comment - UNSTABLE: Integrated in Flume-trunk-hbase-1 #150 (See https://builds.apache.org/job/Flume-trunk-hbase-1/150/ ) FLUME-2886 : Optional Channels can cause OOMs (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=109ec30725a4c665a2ccf5f40af8a0e455cf4166 ) flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelProcessor.java flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        Thank you for your contribution Hari Shreedharan!

        Show
        jarcec Jarek Jarcec Cecho added a comment - Thank you for your contribution Hari Shreedharan !
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 44bbe97d9db8e82b309943f5e1f17cb7a4608aed in flume's branch refs/heads/flume-1.7 from Jarek Jarcec Cecho
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=44bbe97 ]

        FLUME-2886: Optional Channels can cause OOMs

        (Hari Shreedharan via Jarek Jarcec Cecho)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 44bbe97d9db8e82b309943f5e1f17cb7a4608aed in flume's branch refs/heads/flume-1.7 from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=44bbe97 ] FLUME-2886 : Optional Channels can cause OOMs (Hari Shreedharan via Jarek Jarcec Cecho)
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 109ec30725a4c665a2ccf5f40af8a0e455cf4166 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=109ec30 ]

        FLUME-2886: Optional Channels can cause OOMs

        (Hari Shreedharan via Jarek Jarcec Cecho)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 109ec30725a4c665a2ccf5f40af8a0e455cf4166 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=109ec30 ] FLUME-2886 : Optional Channels can cause OOMs (Hari Shreedharan via Jarek Jarcec Cecho)
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        +1

        Show
        jarcec Jarek Jarcec Cecho added a comment - +1
        Hide
        hshreedharan Hari Shreedharan added a comment -

        This fixes the issue and adds a unit test

        Show
        hshreedharan Hari Shreedharan added a comment - This fixes the issue and adds a unit test

          People

          • Assignee:
            hshreedharan Hari Shreedharan
            Reporter:
            hshreedharan Hari Shreedharan
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development