Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2886

Optional Channels can cause OOMs

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: 1.7.0
        • Component/s: None
        • Labels:
          None

          Description

          If an optional channel is full, the queue backing the executor that is asynchronously submitting the events to the channel can grow indefinitely in size leading to a huge number of events on the heap and causing OOMs.

          1. Text File
            FLUME-2886.patch
            4 kB
            Hari Shreedharan

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            hshreedharan Hari Shreedharan added a comment -

            This fixes the issue and adds a unit test

            Show
            hshreedharan Hari Shreedharan added a comment - This fixes the issue and adds a unit test
            Hide
            Permalink
            jarcec Jarek Jarcec Cecho added a comment -

            +1

            Show
            jarcec Jarek Jarcec Cecho added a comment - +1
            Hide
            Permalink
            jira-bot ASF subversion and git services added a comment -

            Commit 109ec30725a4c665a2ccf5f40af8a0e455cf4166 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho
            [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=109ec30 ]

            FLUME-2886: Optional Channels can cause OOMs

            (Hari Shreedharan via Jarek Jarcec Cecho)

            Show
            jira-bot ASF subversion and git services added a comment - Commit 109ec30725a4c665a2ccf5f40af8a0e455cf4166 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=109ec30 ] FLUME-2886 : Optional Channels can cause OOMs (Hari Shreedharan via Jarek Jarcec Cecho)
            Hide
            Permalink
            jira-bot ASF subversion and git services added a comment -

            Commit 44bbe97d9db8e82b309943f5e1f17cb7a4608aed in flume's branch refs/heads/flume-1.7 from Jarek Jarcec Cecho
            [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=44bbe97 ]

            FLUME-2886: Optional Channels can cause OOMs

            (Hari Shreedharan via Jarek Jarcec Cecho)

            Show
            jira-bot ASF subversion and git services added a comment - Commit 44bbe97d9db8e82b309943f5e1f17cb7a4608aed in flume's branch refs/heads/flume-1.7 from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=44bbe97 ] FLUME-2886 : Optional Channels can cause OOMs (Hari Shreedharan via Jarek Jarcec Cecho)
            Hide
            Permalink
            jarcec Jarek Jarcec Cecho added a comment -

            Thank you for your contribution Hari Shreedharan!

            Show
            jarcec Jarek Jarcec Cecho added a comment - Thank you for your contribution Hari Shreedharan !
            Hide
            Permalink
            hudson Hudson added a comment -

            UNSTABLE: Integrated in Flume-trunk-hbase-1 #150 (See https://builds.apache.org/job/Flume-trunk-hbase-1/150/)
            FLUME-2886: Optional Channels can cause OOMs (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=109ec30725a4c665a2ccf5f40af8a0e455cf4166)

            • flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelProcessor.java
            • flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java
            Show
            hudson Hudson added a comment - UNSTABLE: Integrated in Flume-trunk-hbase-1 #150 (See https://builds.apache.org/job/Flume-trunk-hbase-1/150/ ) FLUME-2886 : Optional Channels can cause OOMs (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=109ec30725a4c665a2ccf5f40af8a0e455cf4166 ) flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelProcessor.java flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java
            Hide
            Permalink
            jira-bot ASF subversion and git services added a comment -

            Commit caa64a1a6d4bc97be5993cb468516e9ffe862794 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho
            [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=caa64a1 ]

            FLUME-2891: Revert FLUME-2712 and FLUME-2886

            (Hari Shreedharan via Jarek Jarcec Cecho)

            Show
            jira-bot ASF subversion and git services added a comment - Commit caa64a1a6d4bc97be5993cb468516e9ffe862794 in flume's branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=caa64a1 ] FLUME-2891 : Revert FLUME-2712 and FLUME-2886 (Hari Shreedharan via Jarek Jarcec Cecho)
            Hide
            Permalink
            jira-bot ASF subversion and git services added a comment -

            Commit c2562900f1ab9c21ae2d7c0fc26e270847e81b6d in flume's branch refs/heads/flume-1.7 from Jarek Jarcec Cecho
            [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=c256290 ]

            FLUME-2891: Revert FLUME-2712 and FLUME-2886

            (Hari Shreedharan via Jarek Jarcec Cecho)

            Show
            jira-bot ASF subversion and git services added a comment - Commit c2562900f1ab9c21ae2d7c0fc26e270847e81b6d in flume's branch refs/heads/flume-1.7 from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=c256290 ] FLUME-2891 : Revert FLUME-2712 and FLUME-2886 (Hari Shreedharan via Jarek Jarcec Cecho)
            Hide
            Permalink
            hudson Hudson added a comment -

            UNSTABLE: Integrated in Flume-trunk-hbase-1 #152 (See https://builds.apache.org/job/Flume-trunk-hbase-1/152/)
            FLUME-2891: Revert FLUME-2712 and FLUME-2886 (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=caa64a1a6d4bc97be5993cb468516e9ffe862794)

            • flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelProcessor.java
            • flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java
            Show
            hudson Hudson added a comment - UNSTABLE: Integrated in Flume-trunk-hbase-1 #152 (See https://builds.apache.org/job/Flume-trunk-hbase-1/152/ ) FLUME-2891 : Revert FLUME-2712 and FLUME-2886 (jarcec: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=caa64a1a6d4bc97be5993cb468516e9ffe862794 ) flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelProcessor.java flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java

              People

              • Assignee:
                hshreedharan Hari Shreedharan
                Reporter:
                hshreedharan Hari Shreedharan
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development