[FLINK-10269] Elasticsearch 6 UpdateRequest fail because of binary incompatibility

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 1.6.1, 1.7.0
• Component/s: Connectors / ElasticSearch
• Labels: pull-request-available
• Assignee: Timo Walther
• Reporter: Timo Walther
• Created: 31/Aug/18 06:20
• Updated: 13/Sep/18 11:28
• Resolved: 13/Sep/18 11:28

Description

When trying to send UpdateRequest(s) to ElasticSearch6, and one gets the following
error:

Caused by: java.lang.NoSuchMethodError:
org.elasticsearch.action.bulk.BulkProcessor.add(Lorg/elasticsearch/action/ActionRequest;)Lorg/elasticsearch/action/bulk/BulkProcessor;
	at
org.apache.flink.streaming.connectors.elasticsearch.BulkProcessorIndexer.add(BulkProcessorIndexer.java:76)

ElasticsearchSinkFunction:

	import org.elasticsearch.action.update.UpdateRequest
	def upsertRequest(element: T): UpdateRequest = {
		new UpdateRequest(
			"myIndex",
			"record",
			s"${element.id}")
	        	.doc(element.toMap())
	}
	override def process(element: T, runtimeContext: RuntimeContext,
requestIndexer: RequestIndexer): Unit = {
		requestIndexer.add(upsertRequest(element))
	}

This is due to a binary compatibility issue between the base module (which is compiled against a very old ES version and the current Elasticsearch version).

As a work around you can simply copy org.apache.flink.streaming.connectors.elasticsearch.BulkProcessorIndexer to your project. This should ensure that the class is compiled correctly.

Issue Links

• is duplicated by: FLINK-10271 flink-connector-elasticsearch6_2.11 have exception
• links to: GitHub Pull Request #6682