Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9618

NullPointerException in FlinkKinesisProducer when aws.region is not set and aws.endpoint is set

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Component/s: Connectors / Kinesis
    • Labels:
      None
    • Environment:

      N/A

      Description

      This problem arose while trying to write to a local kinesalite instance. Specifying the aws.region and the aws.endpoint is not allowed. However when the aws.region is not present, a NullPointer exception is thrown.

      Here is some example Scala code:

      /**
        *
        * @param region the AWS region the stream lives in
        * @param streamName the stream to write records to
        * @param endpoint if in local dev, this points to a kinesalite instance
        * @return
        */
      def getSink(region: String,
                  streamName: String,
                  endpoint: Option[String]): FlinkKinesisProducer[ProcessedMobilePageView] = {
        val props = new Properties()
        props.put(AWSConfigConstants.AWS_CREDENTIALS_PROVIDER, "AUTO")
      
        endpoint match {
          case Some(uri) => props.put(AWSConfigConstants.AWS_ENDPOINT, uri)
          case None => props.put(AWSConfigConstants.AWS_REGION, region)
        }
      
        val producer = new FlinkKinesisProducer[ProcessedMobilePageView](
          new JsonSerializer[ProcessedMobilePageView](DefaultSerializationBuilder),
          props
        )
        producer.setDefaultStream(streamName)
      
        producer
      }
      

      To produce the NullPointerException, pass in `Some("localhost:4567")` for endpoint.

      The source of the error is found at org.apache.flink.streaming.connectors.kinesis.util.KinesisConfigUtil.java, on line 194. This line should perform some kind of check if aws.endpoint is present before grabbing it from the Properties object.

       

        Attachments

          Activity

            People

            • Assignee:
              aaron.langford31 Aaron Langford
              Reporter:
              aaron.langford31 Aaron Langford
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 3h
                3h
                Remaining:
                Remaining Estimate - 3h
                3h
                Logged:
                Time Spent - Not Specified
                Not Specified