Flume
  1. Flume
  2. FLUME-816

TestJdbcChannelProvider throws OOME based on RNG values

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: NG alpha 1
    • Fix Version/s: NG alpha 2
    • Component/s: Technical Debt, Test
    • Labels:
      None

      Description

      TestJdbcChannelProvider uses a random number to test header overflow. This causes the JVM running the tests to sometimes die from an OOME. We should cap the test so overflows the header table value field size but stays within a reasonable limit. I've been able to verify this by increasing the maven test JVM heap size to values > 128MB (or more).

      1. FLUME-816-1.patch
        0.8 kB
        Arvind Prabhakar

        Activity

        Hide
        Arvind Prabhakar added a comment -

        This change takes care of increasing the default heap size for JDBC channel tests.

        Show
        Arvind Prabhakar added a comment - This change takes care of increasing the default heap size for JDBC channel tests.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/2930/
        -----------------------------------------------------------

        Review request for Flume and Eric Sammer.

        Summary
        -------

        Fix for FLUME-816

        This addresses bug FLUME-816.
        https://issues.apache.org/jira/browse/FLUME-816

        Diffs


        .gitignore c319325
        flume-ng-channels/flume-jdbc-channel/pom.xml 8665eee

        Diff: https://reviews.apache.org/r/2930/diff

        Testing
        -------

        Ran all JDBC channel tests on a different system where I was able to see the OOM errors. With this change, the tests pass.

        Thanks,

        Arvind

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2930/ ----------------------------------------------------------- Review request for Flume and Eric Sammer. Summary ------- Fix for FLUME-816 This addresses bug FLUME-816 . https://issues.apache.org/jira/browse/FLUME-816 Diffs .gitignore c319325 flume-ng-channels/flume-jdbc-channel/pom.xml 8665eee Diff: https://reviews.apache.org/r/2930/diff Testing ------- Ran all JDBC channel tests on a different system where I was able to see the OOM errors. With this change, the tests pass. Thanks, Arvind
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/2930/#review3582
        -----------------------------------------------------------

        Ship it!

        lgtm!

        • Eric

        On 2011-11-24 20:22:06, Arvind Prabhakar wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/2930/

        -----------------------------------------------------------

        (Updated 2011-11-24 20:22:06)

        Review request for Flume and Eric Sammer.

        Summary

        -------

        Fix for FLUME-816

        This addresses bug FLUME-816.

        https://issues.apache.org/jira/browse/FLUME-816

        Diffs

        -----

        .gitignore c319325

        flume-ng-channels/flume-jdbc-channel/pom.xml 8665eee

        Diff: https://reviews.apache.org/r/2930/diff

        Testing

        -------

        Ran all JDBC channel tests on a different system where I was able to see the OOM errors. With this change, the tests pass.

        Thanks,

        Arvind

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2930/#review3582 ----------------------------------------------------------- Ship it! lgtm! Eric On 2011-11-24 20:22:06, Arvind Prabhakar wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2930/ ----------------------------------------------------------- (Updated 2011-11-24 20:22:06) Review request for Flume and Eric Sammer. Summary ------- Fix for FLUME-816 This addresses bug FLUME-816 . https://issues.apache.org/jira/browse/FLUME-816 Diffs ----- .gitignore c319325 flume-ng-channels/flume-jdbc-channel/pom.xml 8665eee Diff: https://reviews.apache.org/r/2930/diff Testing ------- Ran all JDBC channel tests on a different system where I was able to see the OOM errors. With this change, the tests pass. Thanks, Arvind
        Hide
        Arvind Prabhakar added a comment -

        Patch committed.

        Show
        Arvind Prabhakar added a comment - Patch committed.
        Hide
        Hudson added a comment -

        Integrated in flume-728 #63 (See https://builds.apache.org/job/flume-728/63/)
        FLUME-816. Increasing the heap size for running JDBC channel tests.

        arvind : http://svn.apache.org/viewvc/?view=rev&rev=1208655
        Files :

        • /incubator/flume/branches/flume-728/.gitignore
        • /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/pom.xml
        Show
        Hudson added a comment - Integrated in flume-728 #63 (See https://builds.apache.org/job/flume-728/63/ ) FLUME-816 . Increasing the heap size for running JDBC channel tests. arvind : http://svn.apache.org/viewvc/?view=rev&rev=1208655 Files : /incubator/flume/branches/flume-728/.gitignore /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/pom.xml

          People

          • Assignee:
            Arvind Prabhakar
            Reporter:
            E. Sammer
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development