[BEAM-1640] data file missing when submit a job on Flink

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Not A Problem
• Affects Version/s: 0.6.0
• Fix Version/s: Not applicable
• Component/s: runner-flink
• Assignee: Aljoscha Krettek
• Reporter: Mingmin Xu
• Created: 07/Mar/17 19:26
• Updated: 16/May/20 13:46
• Resolved: 31/Mar/17 00:37

Description

I've one file with path 'META-INF/jaas/kafka_jaas.conf' in my jar package. it works with Beam 0.5.0, when I re-package it with 0.6.0-SNAPSHOT, it fails to submit with bin/flink command. --Both run on YARN.

The error is show as below, I guess this file maybe lost in Flink-Runner.

Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702)
	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:557)
	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:540)
	at org.apache.beam.sdk.io.kafka.KafkaIO$Read$2.apply(KafkaIO.java:503)
	at org.apache.beam.sdk.io.kafka.KafkaIO$Read$2.apply(KafkaIO.java:501)
	at org.apache.beam.sdk.io.kafka.KafkaIO$UnboundedKafkaSource.generateInitialSplits(KafkaIO.java:620)
	at org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper.<init>(UnboundedSourceWrapper.java:159)
	at org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$UnboundedReadSourceTranslator.translateNode(FlinkStreamingTransformTranslators.java:267)
	... 33 more
Caused by: org.apache.kafka.common.KafkaException: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in jaas config.
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86)
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:70)
	at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83)
	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:623)
	... 40 more
Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in jaas config.
	at io.ebay.rheos.kafka.security.iaf.IAFLogin.login(IAFLogin.java:54)
	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:53)
	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:75)
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78)
	... 43 more