Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2154

Reducing duplicate events caused by reset-connection-interval

        Details

        • Type: Improvement
        • Status: Resolved
        • Priority: Minor
        • Resolution: Fixed
        • Affects Version/s: 1.4.0
        • Fix Version/s: 1.5.0
        • Component/s: None
        • Labels:
          None

          Description

          In avro-sink/source it would appear that every time the connection is reset some duplicate logs can be generated. I haven't been able to figure out the cause of this(they should have been rolled back when the connection gets closed by the timer)

          I've put together a patch that serializes the reset. This gets rid of the duplicates and shouldn't result in a performance penalty as events sent during the reset end up getting reset anyway.

          1. Text File
            FLUME-2154-2.patch
            4 kB
            Juhani Connolly

            Issue Links

            links to

            Web Link Review request

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              hshreedharan Hari Shreedharan added a comment -

              Juhani Connolly - Could you please upload the latest patch from rb?

              Show
              hshreedharan Hari Shreedharan added a comment - Juhani Connolly - Could you please upload the latest patch from rb?
              Hide
              Permalink
              juhanic Juhani Connolly added a comment -

              Latest patch from RB

              Show
              juhanic Juhani Connolly added a comment - Latest patch from RB
              Hide
              Permalink
              hshreedharan Hari Shreedharan added a comment -

              Committed, rev: 99db32ccd163daf9d7685f0e8485941701e1133d. Thanks Juhani!

              Show
              hshreedharan Hari Shreedharan added a comment - Committed, rev: 99db32ccd163daf9d7685f0e8485941701e1133d. Thanks Juhani!
              Hide
              Permalink
              hudson Hudson added a comment -

              SUCCESS: Integrated in flume-trunk #488 (See https://builds.apache.org/job/flume-trunk/488/)
              FLUME-2154. Reducing duplicate events caused by reset-connection-interval (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=99db32ccd163daf9d7685f0e8485941701e1133d)

              • flume-ng-core/src/main/java/org/apache/flume/sink/AbstractRpcSink.java
              • flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java
              Show
              hudson Hudson added a comment - SUCCESS: Integrated in flume-trunk #488 (See https://builds.apache.org/job/flume-trunk/488/ ) FLUME-2154 . Reducing duplicate events caused by reset-connection-interval (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=99db32ccd163daf9d7685f0e8485941701e1133d ) flume-ng-core/src/main/java/org/apache/flume/sink/AbstractRpcSink.java flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java

                People

                • Assignee:
                  juhanic Juhani Connolly
                  Reporter:
                  juhanic Juhani Connolly
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development