Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2154

Reducing duplicate events caused by reset-connection-interval

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.5.0
    • Component/s: None
    • Labels:
      None

      Description

      In avro-sink/source it would appear that every time the connection is reset some duplicate logs can be generated. I haven't been able to figure out the cause of this(they should have been rolled back when the connection gets closed by the timer)

      I've put together a patch that serializes the reset. This gets rid of the duplicates and shouldn't result in a performance penalty as events sent during the reset end up getting reset anyway.

      1. FLUME-2154-2.patch
        4 kB
        Juhani Connolly

        Issue Links

          Activity

          Hide
          hshreedharan Hari Shreedharan added a comment -

          Juhani Connolly - Could you please upload the latest patch from rb?

          Show
          hshreedharan Hari Shreedharan added a comment - Juhani Connolly - Could you please upload the latest patch from rb?
          Hide
          juhanic Juhani Connolly added a comment -

          Latest patch from RB

          Show
          juhanic Juhani Connolly added a comment - Latest patch from RB
          Hide
          hshreedharan Hari Shreedharan added a comment -

          Committed, rev: 99db32ccd163daf9d7685f0e8485941701e1133d. Thanks Juhani!

          Show
          hshreedharan Hari Shreedharan added a comment - Committed, rev: 99db32ccd163daf9d7685f0e8485941701e1133d. Thanks Juhani!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in flume-trunk #488 (See https://builds.apache.org/job/flume-trunk/488/)
          FLUME-2154. Reducing duplicate events caused by reset-connection-interval (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=99db32ccd163daf9d7685f0e8485941701e1133d)

          • flume-ng-core/src/main/java/org/apache/flume/sink/AbstractRpcSink.java
          • flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in flume-trunk #488 (See https://builds.apache.org/job/flume-trunk/488/ ) FLUME-2154 . Reducing duplicate events caused by reset-connection-interval (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=99db32ccd163daf9d7685f0e8485941701e1133d ) flume-ng-core/src/main/java/org/apache/flume/sink/AbstractRpcSink.java flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java

            People

            • Assignee:
              juhanic Juhani Connolly
              Reporter:
              juhanic Juhani Connolly
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development