[BAHIR-51] Add additional MQTT options/parameters to MQTTInputDStream and MqttStreamSource

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Spark-2.0.1, Spark-2.1.0
• Component/s: Spark Streaming Connectors, Spark Structured Streaming Connectors
• Assignee: Prashant Sharma
• Reporter: Sebastian Woehrl
• Created: 10/Aug/16 12:39
• Updated: 07/Feb/20 17:23
• Resolved: 05/Oct/16 06:35

Description

We are using Spark Streaming in the automotive IOT environment with MQTT as the data source.
For security reasons our MQTT broker is protected by username and password (as is default for these kind of environments). At the moment it is not possible to set username/password when creating an MQTT Receiver (MQTTInputDStream or MqttStreamSource).

I propose that the MQTTInputDStream and MqttStreamSource be extended to optionally allow setting the following mqtt options / parameters:

• username
• password
• clientId
• cleanSession
• QoS
• connectionTimeout
• keepAliveInterval
• mqttVersion

If this proposal meets your approval I am willing to create a pull request with these changes implemented.

Note: The part for MqttInputDStream has been split off into BAHIR-53.

Issue Links

• links to: GitHub Pull Request #22
• links to: GitHub Pull Request #23