[BAHIR-203] Pubsub manual acknowledgement

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: Spark-2.4.0
• Component/s: Spark Streaming Connectors
• Assignee: Lukasz Antoniak
• Reporter: Danny Tachev
• Created: 09/Apr/19 11:33
• Updated: 17/May/19 22:49
• Resolved: 17/May/19 22:49

Description

Hi, 

We have a use case where acknowledgement has to be sent at a later stage when streaming data from google pubsub. Any chance for the acknowledgement in PubsubReceiver to be made optional and ackId to be included in the SparkPubsubMessage model?

Example:

store(receivedMessages
    .map(x => {
      val sm = new SparkPubsubMessage
      sm.message = x.getMessage
      sm.ackId = x.getAckId
      sm
    })
    .iterator)

if ( ... ) {
  val ackRequest = new AcknowledgeRequest()
  ackRequest.setAckIds(receivedMessages.map(x => x.getAckId).asJava)
  client.projects().subscriptions().acknowledge(subscriptionFullName, ackRequest).execute()
}

 

Issue Links

• links to: GitHub Pull Request #85