Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Cannot Reproduce
-
None
-
None
-
None
-
None
Description
I am trying to read kafka messages using spart api in batch mode.To achieve
this , I need start and last offset of the mentioned topic. To get start and last offset I am creating simple consumer with below code
-----------------------------------------------------------------------------------------
var consumer = new SimpleConsumer(seedBroker, seedBrokerPort, 100000, 64 * 1024, clientName);
var topicAndPartition = new TopicAndPartition(topic, partition.toInt)
var requestInfo = new HashMap[TopicAndPartition,PartitionOffsetRequestInfo];
requestInfo.put(topicAndPartition, new PartitionOffsetRequestInfo(whichTime, 1))
logger.info("requestInfo - " + requestInfo);
var request = new kafka.javaapi.OffsetRequest(requestInfo, OffsetRequest.CurrentVersion, clientName);
var response =consumer.getOffsetsBefore(request);
-------------------------------------------------------------------
I am using BIG IP addresses for seed broker which actually has four brokers . Now if I do spark submit then some times it get exception with below response
response - OffsetResponse(0,Map([ecp.demo.patents.control,0] -> error: kafka.common.UnknownTopicOrPartitionException offsets: ))
but some time i get proper response..... could anybody explain why it is like that?