Flume
  1. Flume
  2. FLUME-2910

AsyncHBaseSink - Failure callbacks should log the exception that caused them

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.6.0
    • Fix Version/s: v1.7.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      Failure callbacks in the AsyncHBaseSink currently do not log the exception that causes them to be called, this should be fixed

        Activity

        Hide
        Hari Shreedharan added a comment -

        Does the Asynchbase API actually send the exception back as the argument of the call method? Do you have the javadoc that specifies this available?

        Show
        Hari Shreedharan added a comment - Does the Asynchbase API actually send the exception back as the argument of the call method? Do you have the javadoc that specifies this available?
        Hide
        Abraham Fine added a comment -

        Looking at the javadocs for stumbleupon's async (which asynchbase uses) here: http://www.stumbleupon.com/su/19ESrH/tsunanet.net/~tsuna/async/api/

        When the current result of a Deferred is an instance of Exception, the next errback is invoked. As for normal callbacks, whatever the errback returns becomes the current result. If the current result is still an instance of Exception, the next errback is invoked. If the current result is no longer an Exception, the next callback is invoked.

        The code that calls the callback is a little clearer (https://github.com/OpenTSDB/async/blob/v1.4.0/src/Deferred.java#L688, https://github.com/OpenTSDB/async/blob/v1.4.0/src/Deferred.java#L1257):

        doCall(result instanceof Exception ? eb : cb)
        

        where eb is the "errback" and cb is the normal callback.

        Show
        Abraham Fine added a comment - Looking at the javadocs for stumbleupon's async (which asynchbase uses) here: http://www.stumbleupon.com/su/19ESrH/tsunanet.net/~tsuna/async/api/ When the current result of a Deferred is an instance of Exception, the next errback is invoked. As for normal callbacks, whatever the errback returns becomes the current result. If the current result is still an instance of Exception, the next errback is invoked. If the current result is no longer an Exception, the next callback is invoked. The code that calls the callback is a little clearer ( https://github.com/OpenTSDB/async/blob/v1.4.0/src/Deferred.java#L688 , https://github.com/OpenTSDB/async/blob/v1.4.0/src/Deferred.java#L1257): doCall(result instanceof Exception ? eb : cb) where eb is the "errback" and cb is the normal callback.
        Hide
        Mike Percy added a comment -

        Abraham Fine: Has this patch been tested?

        Show
        Mike Percy added a comment - Abraham Fine : Has this patch been tested?
        Hide
        Mike Percy added a comment -

        +1. Abe told me that he tested this offline.

        Show
        Mike Percy added a comment - +1. Abe told me that he tested this offline.
        Hide
        ASF subversion and git services added a comment -

        Commit ee4999bc23f42bc300ed87b0d46fd96418d6a185 in flume's branch refs/heads/trunk from Mike Percy
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=ee4999b ]

        FLUME-2910. AsyncHBaseSink: Failure callbacks should log the exception that caused them

        (Abraham Fine via Mike Percy)

        Show
        ASF subversion and git services added a comment - Commit ee4999bc23f42bc300ed87b0d46fd96418d6a185 in flume's branch refs/heads/trunk from Mike Percy [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=ee4999b ] FLUME-2910 . AsyncHBaseSink: Failure callbacks should log the exception that caused them (Abraham Fine via Mike Percy)
        Hide
        Mike Percy added a comment -

        Pushed to trunk. Thanks for the patch Abe!

        Show
        Mike Percy added a comment - Pushed to trunk. Thanks for the patch Abe!
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Flume-trunk-hbase-1 #161 (See https://builds.apache.org/job/Flume-trunk-hbase-1/161/)
        FLUME-2910. AsyncHBaseSink: Failure callbacks should log the exception (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=ee4999bc23f42bc300ed87b0d46fd96418d6a185)

        • flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java
        Show
        Hudson added a comment - FAILURE: Integrated in Flume-trunk-hbase-1 #161 (See https://builds.apache.org/job/Flume-trunk-hbase-1/161/ ) FLUME-2910 . AsyncHBaseSink: Failure callbacks should log the exception (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=ee4999bc23f42bc300ed87b0d46fd96418d6a185 ) flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java

          People

          • Assignee:
            Abraham Fine
            Reporter:
            Abraham Fine
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development