Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.4.1, 1.5.0
    • Component/s: None
    • Labels:
      None

      Description

      Existing ElasticSearchSink uses ES TransportClient. As such, one cannot use the ES HTTP API, which is sometimes easier, and doesn't have issues around client and server/cluster components using incompatible versions - currently, both client and server/cluster need to be on the same version.

      See
      http://search-hadoop.com/m/k76HH9Te68/otis&subj=Elasticsearch+sink+that+uses+HTTP+API

      1. FLUME-2225-0.patch
        106 kB
        Pawel Rog
      2. FLUME-2225-1.patch
        111 kB
        Pawel Rog
      3. FLUME-2225-5.patch
        97 kB
        Pawel Rog
      4. FLUME-2225-6.patch
        97 kB
        Pawel Rog

        Issue Links

          Activity

          Hide
          prog_ Pawel Rog added a comment -

          I attach patch for this. There is no new sink created. Existing one was modified to be able to send documents to elasticsearch by two different clients. One of clients is Transport client and second one i REST client (using Jest library).

          There is also possibility to change it a little as Chiwan Park suggested: http://mail-archives.apache.org/mod_mbox/flume-dev/201401.mbox/%3CCC8881BF-E635-430D-8D2A-F466C6C2EAAE%40icloud.com%3E

          Please let me know if I should write some more detailed information about the change.

          Show
          prog_ Pawel Rog added a comment - I attach patch for this. There is no new sink created. Existing one was modified to be able to send documents to elasticsearch by two different clients. One of clients is Transport client and second one i REST client (using Jest library). There is also possibility to change it a little as Chiwan Park suggested: http://mail-archives.apache.org/mod_mbox/flume-dev/201401.mbox/%3CCC8881BF-E635-430D-8D2A-F466C6C2EAAE%40icloud.com%3E Please let me know if I should write some more detailed information about the change.
          Hide
          prog_ Pawel Rog added a comment -

          I think that discovery in http client should be configurable, so I'll prepare a patch later

          Show
          prog_ Pawel Rog added a comment - I think that discovery in http client should be configurable, so I'll prepare a patch later
          Hide
          prog_ Pawel Rog added a comment -

          I attach patch-1 which is patch-0 extended by feature to configure rest client.

          In configuration of sink there is possibility to configure discoveryFrequency

          a1.sinks.k1.type = elasticsearch
          a1.sinks.k1.hostNames = 127.0.0.2:9200
          a1.sinks.k1.indexName = foo_index
          a1.sinks.k1.indexType = bar_type
          a1.sinks.k1.client = rest
          a1.sinks.k1.client.discoveryFrequency = 10000
          a1.sinks.k1.batchSize = 2
          

          If discovery frequency (expressed in milliseconds) is grater than 0 rest client is configured to enable discovery and sets discovery frequency to specified period of time.

          Show
          prog_ Pawel Rog added a comment - I attach patch-1 which is patch-0 extended by feature to configure rest client. In configuration of sink there is possibility to configure discoveryFrequency a1.sinks.k1.type = elasticsearch a1.sinks.k1.hostNames = 127.0.0.2:9200 a1.sinks.k1.indexName = foo_index a1.sinks.k1.indexType = bar_type a1.sinks.k1.client = rest a1.sinks.k1.client.discoveryFrequency = 10000 a1.sinks.k1.batchSize = 2 If discovery frequency (expressed in milliseconds) is grater than 0 rest client is configured to enable discovery and sets discovery frequency to specified period of time.
          Hide
          otis Otis Gospodnetic added a comment -

          Any chance this could be reviewed and committed before the next release?

          Show
          otis Otis Gospodnetic added a comment - Any chance this could be reviewed and committed before the next release?
          Hide
          ejsarge Edward Sargisson added a comment -

          I'm still working through reviewing this. However, could you please put the -1 patch into a review board please. Preferably as a diff from the -0 patch but cumulative would be okay too.

          Show
          ejsarge Edward Sargisson added a comment - I'm still working through reviewing this. However, could you please put the -1 patch into a review board please. Preferably as a diff from the -0 patch but cumulative would be okay too.
          Hide
          prog_ Pawel Rog added a comment -

          Accidentally I pushed patch 1 to review instead of patch 0, but I cannot change a name of existing link.

          Should I send only patch 0 to review or just difference between patch 0 and patch 1?

          Show
          prog_ Pawel Rog added a comment - Accidentally I pushed patch 1 to review instead of patch 0, but I cannot change a name of existing link. Should I send only patch 0 to review or just difference between patch 0 and patch 1?
          Hide
          paliwalashish Ashish Paliwal added a comment -

          I can spend some time to review this. Given my ES experience is beginner, not sure how much I can help.

          Doc changes are missing, so may be worth adding in next patch.

          Show
          paliwalashish Ashish Paliwal added a comment - I can spend some time to review this. Given my ES experience is beginner, not sure how much I can help. Doc changes are missing, so may be worth adding in next patch.
          Hide
          ejsarge Edward Sargisson added a comment -

          Pawel - Having the complete patch 1 in the review as you have it is fine.
          Ashish - sure! great!

          There's a lot to go through and I have a few minor concerns so hopefully I'll find some time today.

          Show
          ejsarge Edward Sargisson added a comment - Pawel - Having the complete patch 1 in the review as you have it is fine. Ashish - sure! great! There's a lot to go through and I have a few minor concerns so hopefully I'll find some time today.
          Hide
          paliwalashish Ashish Paliwal added a comment -

          Pawel Rog I am reviewing it in chunks (short on time), overall I like the way the transports have been forked. Shall try to push more comments today.

          Show
          paliwalashish Ashish Paliwal added a comment - Pawel Rog I am reviewing it in chunks (short on time), overall I like the way the transports have been forked. Shall try to push more comments today.
          Hide
          prog_ Pawel Rog added a comment -

          Thanks for that review. Now I have a lot of other work, so that's why I didn't send any patches for comments you've posted. When you make review of next chunk please write comments.

          Show
          prog_ Pawel Rog added a comment - Thanks for that review. Now I have a lot of other work, so that's why I didn't send any patches for comments you've posted. When you make review of next chunk please write comments.
          Hide
          paliwalashish Ashish Paliwal added a comment -

          Thanks! IMHO, better to wait for review comments from Edward before updating the patch.

          Show
          paliwalashish Ashish Paliwal added a comment - Thanks! IMHO, better to wait for review comments from Edward before updating the patch.
          Hide
          ejsarge Edward Sargisson added a comment -

          Hi Pawel,
          Thank you for submitting this patch; it clearly took a lot of work and should be useful to many other people.

          I've finished my review and added lots of comments. My biggest concern is that backwards compatibility with custom derived classes has been broken. We did extensive work on keeping backwards compatibility in previous versions; we should continue to keep this so that we do not needlessly break code.

          I look forward to seeing the results of your rework.

          Cheers,
          Edward

          Show
          ejsarge Edward Sargisson added a comment - Hi Pawel, Thank you for submitting this patch; it clearly took a lot of work and should be useful to many other people. I've finished my review and added lots of comments. My biggest concern is that backwards compatibility with custom derived classes has been broken. We did extensive work on keeping backwards compatibility in previous versions; we should continue to keep this so that we do not needlessly break code. I look forward to seeing the results of your rework. Cheers, Edward
          Hide
          prog_ Pawel Rog added a comment -

          Hi,
          Thank you for that review. I have a lot of work to do there to fix all the things you wrote.
          Please clarify here if you mean that I just should repair the backwards compatibility in all places. Do I understand it correctly?

          Show
          prog_ Pawel Rog added a comment - Hi, Thank you for that review. I have a lot of work to do there to fix all the things you wrote. Please clarify here if you mean that I just should repair the backwards compatibility in all places. Do I understand it correctly?
          Hide
          ejsarge Edward Sargisson added a comment -

          > repair the backwards compatibility in all places
          Where our docs say that a custom class can be specified, then we should support that custom class in a backwards compatible fashion.

          Show
          ejsarge Edward Sargisson added a comment - > repair the backwards compatibility in all places Where our docs say that a custom class can be specified, then we should support that custom class in a backwards compatible fashion.
          Hide
          prog_ Pawel Rog added a comment -

          Ok, so you only mena external/custom classes? Right?
          For a while I tought you suggest to make more changes.

          Show
          prog_ Pawel Rog added a comment - Ok, so you only mena external/custom classes? Right? For a while I tought you suggest to make more changes.
          Hide
          prog Pawel Rog added a comment -

          Hi,
          I attached a new patchset. I'll analyze it and try to find something to change, but I'll be also glad if you can watch that patchset.

          Show
          prog Pawel Rog added a comment - Hi, I attached a new patchset. I'll analyze it and try to find something to change, but I'll be also glad if you can watch that patchset.
          Hide
          prog Pawel Rog added a comment -

          New patchset has been added.

          Show
          prog Pawel Rog added a comment - New patchset has been added.
          Hide
          otis Otis Gospodnetic added a comment -

          Looks like Edward Sargisson is happy, judging from comments in the review board. How do we go from patch + review to commit?

          Show
          otis Otis Gospodnetic added a comment - Looks like Edward Sargisson is happy, judging from comments in the review board. How do we go from patch + review to commit?
          Hide
          hshreedharan Hari Shreedharan added a comment -

          Pawel Rog - I am wary of bumping Guava from 11 to 14 since Guava breaks compatibility between releases. Hadoop currently depends on Guava 11.x and we should stay on that version to make sure we don't end up breaking some of the current components

          Show
          hshreedharan Hari Shreedharan added a comment - Pawel Rog - I am wary of bumping Guava from 11 to 14 since Guava breaks compatibility between releases. Hadoop currently depends on Guava 11.x and we should stay on that version to make sure we don't end up breaking some of the current components
          Hide
          prog Pawel Rog added a comment -

          I see. So do you have any suggestion what to do in that case (Jest library uses guava 14) ?

          Show
          prog Pawel Rog added a comment - I see. So do you have any suggestion what to do in that case (Jest library uses guava 14) ?
          Hide
          ejsarge Edward Sargisson added a comment -

          Hi Pawel Rog,
          Let me acknowledge that it's super-frustrating to get code through the Flume review process. Thank you for your contributin.

          To replace Jest, can I suggest that there is Netty and Jackson JSON already in the dependencies. They are older versions but trustworthy.

          Show
          ejsarge Edward Sargisson added a comment - Hi Pawel Rog , Let me acknowledge that it's super-frustrating to get code through the Flume review process. Thank you for your contributin. To replace Jest, can I suggest that there is Netty and Jackson JSON already in the dependencies. They are older versions but trustworthy.
          Hide
          hshreedharan Hari Shreedharan added a comment -

          Sorry this is taking so long to be committed, but if what Edward suggested above is possible, then could we do that?

          Show
          hshreedharan Hari Shreedharan added a comment - Sorry this is taking so long to be committed, but if what Edward suggested above is possible, then could we do that?
          Hide
          prog Pawel Rog added a comment -

          Hi,
          OK I'll try to do it using httpclient or netty.

          Show
          prog Pawel Rog added a comment - Hi, OK I'll try to do it using httpclient or netty.
          Hide
          otis Otis Gospodnetic added a comment -

          Hari Shreedharan - I see another Guava upgrade to 14.x in FLUME-2286.

          Show
          otis Otis Gospodnetic added a comment - Hari Shreedharan - I see another Guava upgrade to 14.x in FLUME-2286 .
          Hide
          hshreedharan Hari Shreedharan added a comment -

          The guava upgrade in this patch seems like a problem. Maybe shading guava would help? Otherwise we'd have compatibility issues between Hadoop and Guava.

          Show
          hshreedharan Hari Shreedharan added a comment - The guava upgrade in this patch seems like a problem. Maybe shading guava would help? Otherwise we'd have compatibility issues between Hadoop and Guava.
          Hide
          otis Otis Gospodnetic added a comment -

          Hari Shreedharan if you think shading is the way to go, I think we can do it. You're OK with that approach?

          Show
          otis Otis Gospodnetic added a comment - Hari Shreedharan if you think shading is the way to go, I think we can do it. You're OK with that approach?
          Hide
          hshreedharan Hari Shreedharan added a comment -

          At this point, if we really need this patch in, but not affect HDFS, I don't see another option - do you? We will need to shade the guava deps for this patch so elastic search sink uses guava 14.x while HDFS sink uses Guava 11.x.

          It might be a bit of a pain though, as it looks like it is a dependency introduced by this patch that requires the guava packages - so we'd need to shade guava in the dependency too. As I hear it, it is possible to do this in the binary, without modifying the dependencies of the source code, but just painful to do.

          So yeah, I am ok with it, as long as we can do it.

          Show
          hshreedharan Hari Shreedharan added a comment - At this point, if we really need this patch in, but not affect HDFS, I don't see another option - do you? We will need to shade the guava deps for this patch so elastic search sink uses guava 14.x while HDFS sink uses Guava 11.x. It might be a bit of a pain though, as it looks like it is a dependency introduced by this patch that requires the guava packages - so we'd need to shade guava in the dependency too. As I hear it, it is possible to do this in the binary, without modifying the dependencies of the source code, but just painful to do. So yeah, I am ok with it, as long as we can do it.
          Hide
          prog Pawel Rog added a comment -

          Hi,
          I tried to use shading here. But I'm not sure if it is possible to shade dependency of dependency (Jest uses guava 14.0 ... ElasticsearchSink doesn't use it directly). I didn't find any examples how to shade dependencies of dependency. Do you have any idea?

          Show
          prog Pawel Rog added a comment - Hi, I tried to use shading here. But I'm not sure if it is possible to shade dependency of dependency (Jest uses guava 14.0 ... ElasticsearchSink doesn't use it directly). I didn't find any examples how to shade dependencies of dependency. Do you have any idea?
          Hide
          otis Otis Gospodnetic added a comment -

          2 thoughts:

          1. does Jest by some chance also work withe older Guava?
          2. Flume does just one type of call to ES - to index docs, right? Maybe it would be much easier to then just do that "manually" with HttpClient and ditch Jest.
          Show
          otis Otis Gospodnetic added a comment - 2 thoughts: does Jest by some chance also work withe older Guava? Flume does just one type of call to ES - to index docs, right? Maybe it would be much easier to then just do that "manually" with HttpClient and ditch Jest.
          Hide
          prog Pawel Rog added a comment - - edited

          Hi, I attach patch without Jest library. Only direct http library calls.

          That patches was also sent to reviews.apache.org. Now there are no external dependencies.

          Show
          prog Pawel Rog added a comment - - edited Hi, I attach patch without Jest library. Only direct http library calls. That patches was also sent to reviews.apache.org. Now there are no external dependencies.
          Hide
          hshreedharan Hari Shreedharan added a comment -

          If someone familiar with elastic search could review this patch, I will commit it.

          Show
          hshreedharan Hari Shreedharan added a comment - If someone familiar with elastic search could review this patch, I will commit it.
          Hide
          ejsarge Edward Sargisson added a comment -

          Jeremy Karlson and I will take a look.

          Thanks for all your efforts Pawel!

          Show
          ejsarge Edward Sargisson added a comment - Jeremy Karlson and I will take a look. Thanks for all your efforts Pawel!
          Hide
          otis Otis Gospodnetic added a comment -

          Btw. FLUME-2222 mentions Flume + Elasticsearch and duplication. I see this mentioned on the Review board.

          Show
          otis Otis Gospodnetic added a comment - Btw. FLUME-2222 mentions Flume + Elasticsearch and duplication. I see this mentioned on the Review board.
          Hide
          prog Pawel Rog added a comment -

          Hi, Any news about the change?
          Mostly question to Edward about solution for handling error during bulk on ES

          Show
          prog Pawel Rog added a comment - Hi, Any news about the change? Mostly question to Edward about solution for handling error during bulk on ES
          Hide
          ejsarge Edward Sargisson added a comment -

          Oh, I thought I'd replied 2 days ago but PEBKAC with using Review Board.

          I'm happy with the change - I think Hari, etc. should commit it.

          I could argue to adding something to the docs about strategies for de-duplication with elasticsearch - however, that's outside the scope of this work item.

          Show
          ejsarge Edward Sargisson added a comment - Oh, I thought I'd replied 2 days ago but PEBKAC with using Review Board. I'm happy with the change - I think Hari, etc. should commit it. I could argue to adding something to the docs about strategies for de-duplication with elasticsearch - however, that's outside the scope of this work item.
          Hide
          hshreedharan Hari Shreedharan added a comment -

          Ok, great. +1 then. I am going to run tests and commit.

          Show
          hshreedharan Hari Shreedharan added a comment - Ok, great. +1 then. I am going to run tests and commit.
          Hide
          prog Pawel Rog added a comment -

          But I hope you take the patch from review or I should upload that patch also here?

          Show
          prog Pawel Rog added a comment - But I hope you take the patch from review or I should upload that patch also here?
          Hide
          hshreedharan Hari Shreedharan added a comment -

          Is the version the jira outdated? Please upload it here to grant the copyright etc to ASF.

          Show
          hshreedharan Hari Shreedharan added a comment - Is the version the jira outdated? Please upload it here to grant the copyright etc to ASF.
          Hide
          prog Pawel Rog added a comment -

          I attach current patch which was reviewed here: https://reviews.apache.org/r/16650/diff/

          Show
          prog Pawel Rog added a comment - I attach current patch which was reviewed here: https://reviews.apache.org/r/16650/diff/
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit e12f0a7a46752eff2ff3aaddfa0b93fb606074d7 in flume's branch refs/heads/trunk from Hari Shreedharan
          [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=e12f0a7 ]

          FLUME-2225. Elasticsearch Sink for ES HTTP API

          (Pawel Rog via Edward Sargisson, Hari Shreedharan)

          Show
          jira-bot ASF subversion and git services added a comment - Commit e12f0a7a46752eff2ff3aaddfa0b93fb606074d7 in flume's branch refs/heads/trunk from Hari Shreedharan [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=e12f0a7 ] FLUME-2225 . Elasticsearch Sink for ES HTTP API (Pawel Rog via Edward Sargisson, Hari Shreedharan)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 042f53a5f45391a064aa96cba16d9f7f0866db1e in flume's branch refs/heads/flume-1.5 from Hari Shreedharan
          [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=042f53a ]

          FLUME-2225. Elasticsearch Sink for ES HTTP API

          (Pawel Rog via Edward Sargisson, Hari Shreedharan)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 042f53a5f45391a064aa96cba16d9f7f0866db1e in flume's branch refs/heads/flume-1.5 from Hari Shreedharan [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=042f53a ] FLUME-2225 . Elasticsearch Sink for ES HTTP API (Pawel Rog via Edward Sargisson, Hari Shreedharan)
          Hide
          hshreedharan Hari Shreedharan added a comment -

          I committed this. Thanks Pawel for spending so much time and effort working on this. Thanks a lot Edward for your help reviewing it (gave you credit in the commit message too!).

          Show
          hshreedharan Hari Shreedharan added a comment - I committed this. Thanks Pawel for spending so much time and effort working on this. Thanks a lot Edward for your help reviewing it (gave you credit in the commit message too!).
          Hide
          prog Pawel Rog added a comment -

          Nice, Thanks for commiting this

          Show
          prog Pawel Rog added a comment - Nice, Thanks for commiting this
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in flume-trunk #628 (See https://builds.apache.org/job/flume-trunk/628/)
          FLUME-2225. Elasticsearch Sink for ES HTTP API (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo/?p=flume.git&a=commit&h=e12f0a7a46752eff2ff3aaddfa0b93fb606074d7)

          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/RoundRobinList.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/ElasticSearchRestClient.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/TimestampedEvent.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/ElasticSearchClient.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/RoundRobinListTest.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/TimeBasedIndexNameBuilder.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchSink.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchDynamicSerializer.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/NoSuchClientTypeException.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TimestampedEventTest.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/IndexNameBuilder.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSink.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TimeBasedIndexNameBuilderTest.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/ElasticSearchClientFactory.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/ElasticSearchTransportClient.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/pom.xml
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchLogStashEventSerializer.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchIndexRequestBuilderFactory.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/AbstractElasticSearchIndexRequestBuilderFactory.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/TestElasticSearchRestClient.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSinkConstants.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/TestElasticSearchClientFactory.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/TestElasticSearchTransportClient.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/SimpleIndexNameBuilder.java
          • flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchIndexRequestBuilderFactory.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in flume-trunk #628 (See https://builds.apache.org/job/flume-trunk/628/ ) FLUME-2225 . Elasticsearch Sink for ES HTTP API (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo/?p=flume.git&a=commit&h=e12f0a7a46752eff2ff3aaddfa0b93fb606074d7 ) flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/RoundRobinList.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/ElasticSearchRestClient.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/TimestampedEvent.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/ElasticSearchClient.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/RoundRobinListTest.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/TimeBasedIndexNameBuilder.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchSink.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchDynamicSerializer.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/NoSuchClientTypeException.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TimestampedEventTest.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/IndexNameBuilder.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSink.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TimeBasedIndexNameBuilderTest.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/ElasticSearchClientFactory.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/client/ElasticSearchTransportClient.java flume-ng-sinks/flume-ng-elasticsearch-sink/pom.xml flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchLogStashEventSerializer.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchIndexRequestBuilderFactory.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/AbstractElasticSearchIndexRequestBuilderFactory.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/TestElasticSearchRestClient.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSinkConstants.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/TestElasticSearchClientFactory.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/client/TestElasticSearchTransportClient.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/SimpleIndexNameBuilder.java flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchIndexRequestBuilderFactory.java
          Hide
          roshan_naik Roshan Naik added a comment -

          Looks like this feature was not documented in the User guide.

          Show
          roshan_naik Roshan Naik added a comment - Looks like this feature was not documented in the User guide.
          Hide
          prog Pawel Rog added a comment -

          Right. Should I document it somehow?

          Show
          prog Pawel Rog added a comment - Right. Should I document it somehow?
          Hide
          mbonica Martin Bonica added a comment -

          Thank you for all your work implementing this. It looks like documentation for this feature didn't make it into the user guide for 1.5.0 or 1.6.0. I'd like to use this feature; could I ask for some documentation on how to use this? I apologize if this isn't the right place; I wasn't sure where else to ask.

          Show
          mbonica Martin Bonica added a comment - Thank you for all your work implementing this. It looks like documentation for this feature didn't make it into the user guide for 1.5.0 or 1.6.0. I'd like to use this feature; could I ask for some documentation on how to use this? I apologize if this isn't the right place; I wasn't sure where else to ask.
          Hide
          roshan_naik Roshan Naik added a comment -

          Martin Bonica canu open a new jira for this ?

          Show
          roshan_naik Roshan Naik added a comment - Martin Bonica canu open a new jira for this ?
          Hide
          mbonica Martin Bonica added a comment -

          Roshan Naik I have created issue FLUME-2791 for this.

          Show
          mbonica Martin Bonica added a comment - Roshan Naik I have created issue FLUME-2791 for this.

            People

            • Assignee:
              prog Pawel Rog
              Reporter:
              otis Otis Gospodnetic
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development