Flume
  1. Flume
  2. FLUME-1316

AvroSink should be configurable for connect-timeout and request-timeout

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.1.0
    • Fix Version/s: v1.2.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      The Flume SDK provides connect-timeout and request-timeout props, but similar props are not currently exposed in AvroSink.

        Activity

        Hide
        Mubarak Seyed added a comment -

        @Will

        NettyAvroRpcClient uses two constants

        private final static long DEFAULT_CONNECT_TIMEOUT_MILLIS =
              TimeUnit.MILLISECONDS.convert(60, TimeUnit.SECONDS);
        
        private final static long DEFAULT_REQUEST_TIMEOUT_MILLIS =
              TimeUnit.MILLISECONDS.convert(60, TimeUnit.SECONDS);
        

        connect() uses DEFAULT_CONNECT_TIMEOUT_MILLIS and append() uses DEFAULT_REQUEST_TIMEOUT_MILLIS, looks like both are constants.

        Both RpcClient.getDefaultInstance() and AvroSink are making use of NettyAvroRpcClient. Which SDK are you referring to? Are you saying that these two values are default and config can override? for example

        agent.sinks.avro-sink1.connect-timeout=60
        agent.sinks.avro-sink1.request-timeout=60
        
        Show
        Mubarak Seyed added a comment - @Will NettyAvroRpcClient uses two constants private final static long DEFAULT_CONNECT_TIMEOUT_MILLIS = TimeUnit.MILLISECONDS.convert(60, TimeUnit.SECONDS); private final static long DEFAULT_REQUEST_TIMEOUT_MILLIS = TimeUnit.MILLISECONDS.convert(60, TimeUnit.SECONDS); connect() uses DEFAULT_CONNECT_TIMEOUT_MILLIS and append() uses DEFAULT_REQUEST_TIMEOUT_MILLIS, looks like both are constants. Both RpcClient.getDefaultInstance() and AvroSink are making use of NettyAvroRpcClient. Which SDK are you referring to? Are you saying that these two values are default and config can override? for example agent.sinks.avro-sink1.connect-timeout=60 agent.sinks.avro-sink1.request-timeout=60
        Hide
        Hari Shreedharan added a comment -

        Patch committed. Thanks Mike!

        Show
        Hari Shreedharan added a comment - Patch committed. Thanks Mike!
        Hide
        Hudson added a comment -

        Integrated in flume-trunk #272 (See https://builds.apache.org/job/flume-trunk/272/)
        FLUME-1316. AvroSink should be configurable for connect-timeout and request-timeout.

        (Mike Percy via Hari Shreedharan) (Revision 1357932)

        Result = FAILURE
        hshreedharan : http://svn.apache.org/viewvc/?view=rev&rev=1357932
        Files :

        • /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java
        • /incubator/flume/trunk/flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java
        • /incubator/flume/trunk/flume-ng-doc/sphinx/FlumeUserGuide.rst
        • /incubator/flume/trunk/flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java
        • /incubator/flume/trunk/flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java
        • /incubator/flume/trunk/flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientFactory.java
        • /incubator/flume/trunk/flume-ng-sdk/src/test/java/org/apache/flume/api/RpcTestUtils.java
        • /incubator/flume/trunk/flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java
        Show
        Hudson added a comment - Integrated in flume-trunk #272 (See https://builds.apache.org/job/flume-trunk/272/ ) FLUME-1316 . AvroSink should be configurable for connect-timeout and request-timeout. (Mike Percy via Hari Shreedharan) (Revision 1357932) Result = FAILURE hshreedharan : http://svn.apache.org/viewvc/?view=rev&rev=1357932 Files : /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java /incubator/flume/trunk/flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java /incubator/flume/trunk/flume-ng-doc/sphinx/FlumeUserGuide.rst /incubator/flume/trunk/flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java /incubator/flume/trunk/flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java /incubator/flume/trunk/flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientFactory.java /incubator/flume/trunk/flume-ng-sdk/src/test/java/org/apache/flume/api/RpcTestUtils.java /incubator/flume/trunk/flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java

          People

          • Assignee:
            Mike Percy
            Reporter:
            Will McQueen
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development