Flume
  1. Flume
  2. FLUME-2210

UnresolvedAddressException when using multiple hostNames in Elasticsearch sink configuration

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.4.0
    • Fix Version/s: v1.5.0
    • Component/s: Sinks+Sources
    • Environment:

      Centos 6
      Elasticsearch 0.90.5
      Lucene 4.4.0
      Java 1.7

      Description

      Hello,

      I receive the following error when I define multiple hostnames in elasticsearch sink configuration.

      These work

      a1.sinks.k2.hostNames = 10.0.110.50:9300
      
      a1.sinks.k2.hostNames = 10.0.110.51:9300
      

      This doesn't

      a1.sinks.k2.hostNames = 10.0.110.50:9300, 10.0.110.51:9300
      

      Could be a bug? According to flume documentation the hostNames property is a "Comma separated list of hostname:port" .

      10 Oct 2013 16:09:14,077 WARN  [elasticsearch[Big Man][generic][T#1]] (org.elasticsearch.common.logging.log4j.Log4jESLogger.internalWarn:129)  - [Big Man] exception caught on transport layer [[id: 0xac7f61fa]], closing connection
      java.nio.channels.UnresolvedAddressException
      	at sun.nio.ch.Net.checkAddress(Net.java:127)
      	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:613)
      	at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:108)
      	at org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:70)
      	at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:574)
      	at org.elasticsearch.common.netty.channel.Channels.connect(Channels.java:634)
      	at org.elasticsearch.common.netty.channel.AbstractChannel.connect(AbstractChannel.java:207)
      	at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:229)
      	at org.elasticsearch.common.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:182)
      	at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:662)
      	at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:608)
      	at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:576)
      	at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:129)
      	at org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler.sample(TransportClientNodesService.java:305)
      	at org.elasticsearch.client.transport.TransportClientNodesService$ScheduledNodeSampler.run(TransportClientNodesService.java:284)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:724)
      

        Activity

        Hide
        Hudson added a comment -

        FAILURE: Integrated in flume-trunk #516 (See https://builds.apache.org/job/flume-trunk/516/)
        FLUME-2210. UnresolvedAddressException when using multiple hostNames in Elasticsearch sink configuration (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=603bcf2d0ef0d68357d0d40e34484fbdb96aa3f9)

        • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchSink.java
        • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSink.java
        Show
        Hudson added a comment - FAILURE: Integrated in flume-trunk #516 (See https://builds.apache.org/job/flume-trunk/516/ ) FLUME-2210 . UnresolvedAddressException when using multiple hostNames in Elasticsearch sink configuration (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=603bcf2d0ef0d68357d0d40e34484fbdb96aa3f9 ) flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchSink.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSink.java
        Hide
        Dib Ghosh added a comment -

        Thanks Hari, Roshan, Jeff and everyone else on flume community!!

        Show
        Dib Ghosh added a comment - Thanks Hari, Roshan, Jeff and everyone else on flume community!!
        Hide
        Hari Shreedharan added a comment -

        Committed,rev: 603bcf2. Thanks Dib!

        Show
        Hari Shreedharan added a comment - Committed,rev: 603bcf2. Thanks Dib!
        Hide
        Hari Shreedharan added a comment -

        +1. Running tests and committing.

        Show
        Hari Shreedharan added a comment - +1. Running tests and committing.
        Hide
        Dib Ghosh added a comment -
        Show
        Dib Ghosh added a comment - Opened reviewboard: https://reviews.apache.org/r/14639/
        Hide
        Dib Ghosh added a comment -

        Thank you Hari for pointing it out. I was not aware that I can assign tickets to myself.

        • dib
        Show
        Dib Ghosh added a comment - Thank you Hari for pointing it out. I was not aware that I can assign tickets to myself. dib
        Hide
        Hari Shreedharan added a comment -

        Dib Ghosh - You should be able to assign tickets to yourself, since I added you as a contributor a few days ago. Let me know if you are unable to.

        Show
        Hari Shreedharan added a comment - Dib Ghosh - You should be able to assign tickets to yourself, since I added you as a contributor a few days ago. Let me know if you are unable to.
        Hide
        Dib Ghosh added a comment -

        Thank you Roshan. Working on it now.

        Show
        Dib Ghosh added a comment - Thank you Roshan. Working on it now.
        Hide
        Roshan Naik added a comment -

        its assigned to you now Dib Ghosh.

        Show
        Roshan Naik added a comment - its assigned to you now Dib Ghosh .
        Hide
        Dib Ghosh added a comment -

        Glad that it worked Nikolaos. I have a fix. Can someone please assign this ticket to me.

        Show
        Dib Ghosh added a comment - Glad that it worked Nikolaos. I have a fix. Can someone please assign this ticket to me.
        Hide
        Nikolaos Tsipas added a comment -

        Thank you Dib, what you suggested worked fine.

        Regards,
        Nikolaos

        Show
        Nikolaos Tsipas added a comment - Thank you Dib, what you suggested worked fine. Regards, Nikolaos
        Hide
        Dib Ghosh added a comment -

        Hi Nikolaos,

        I am also encountering the bug. The temporary fix is to remove the extra whitespace between hostnames. Example,

        instead of a1.sinks.k2.hostNames = 10.0.110.50:9300, 10.0.110.51:9300

        use,

        a1.sinks.k2.hostNames = 10.0.110.50:9300,10.0.110.51:9300.

        The host:port combination should be stripped in the ElasticSearchSink to make it work.

        I am using Flume 1.4.0-cdh4.4.0
        Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
        Revision: 154d35659212f07edc896b414a43996fb8121773
        Compiled by jenkins on Tue Sep 3 20:53:28 PDT 2013
        From source with checksum f95b4a7f48080f876d6482bb88bcc342.

        java version "1.6.0_27"
        Java(TM) SE Runtime Environment (build 1.6.0_27-b07)
        Java HotSpot(TM) Server VM (build 20.2-b06, mixed mode)

        Thanks,

        • Dib
        Show
        Dib Ghosh added a comment - Hi Nikolaos, I am also encountering the bug. The temporary fix is to remove the extra whitespace between hostnames. Example, instead of a1.sinks.k2.hostNames = 10.0.110.50:9300, 10.0.110.51:9300 use, a1.sinks.k2.hostNames = 10.0.110.50:9300,10.0.110.51:9300. The host:port combination should be stripped in the ElasticSearchSink to make it work. I am using Flume 1.4.0-cdh4.4.0 Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git Revision: 154d35659212f07edc896b414a43996fb8121773 Compiled by jenkins on Tue Sep 3 20:53:28 PDT 2013 From source with checksum f95b4a7f48080f876d6482bb88bcc342. java version "1.6.0_27" Java(TM) SE Runtime Environment (build 1.6.0_27-b07) Java HotSpot(TM) Server VM (build 20.2-b06, mixed mode) Thanks, Dib

          People

          • Assignee:
            Dib Ghosh
            Reporter:
            Nikolaos Tsipas
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development