Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-8249

Kinesis Producer didnt configure region

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.5.0, 1.4.1
    • Component/s: Kinesis Connector
    • Labels:
      None

      Description

      Hi,

      setting this configurations to FlinkKinesisProducer:

      properties.put(AWSConfigConstants.AWS_REGION, "eu-west-1");
      properties.put(AWSConfigConstants.AWS_ACCESS_KEY_ID, "accessKey");
      properties.put(AWSConfigConstants.AWS_SECRET_ACCESS_KEY, "secretKey");
      

      is throwing this error:

      17/12/13 10:50:11 ERROR LogInputStreamReader: [2017-12-13 10:50:11.290786] [0x000057ba][0x00007f31cbce5780] [error] [main.cc:266] Could not configure the region. It was not given in the config and we were unable to retrieve it from EC2 metadata.
      17/12/13 10:50:12 ERROR KinesisProducer: Error in child process
      org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.IrrecoverableError: Child process exited with code 1
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:525)
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:497)
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:475)
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63)
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700441] [0x000057c4][0x00007ffb152b5780] [error] [AWS Log: ERROR](CurlHttpClient)Curl returned error code 28
      17/12/13 10:50:15 ERROR LogInputStreamReader: [2017-12-13 10:50:15.700521] [0x000057c4][0x00007ffb152b5780] [error] [AWS Log: ERROR](EC2MetadataClient)Http request to Ec2MetadataService failed.
      

      making some investigations the region is never setted and i think this is the reason:
      in this commit: https://github.com/apache/flink/commit/9ed5d9a180dcd871e33bf8982434e3afd90ed295#diff-f3c6c35f3b045df8408b310f8f8a6bc7

      -		KinesisProducerConfiguration producerConfig = new KinesisProducerConfiguration();
      -		producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
      +		// check and pass the configuration properties
      +		KinesisProducerConfiguration producerConfig = KinesisConfigUtil.validateProducerConfiguration(configProps);
       		producerConfig.setCredentialsProvider(AWSUtil.getCredentialsProvider(configProps));
      

      this line was removed

      producerConfig.setRegion(configProps.getProperty(ProducerConfigConstants.AWS_REGION));
      

      cc Tzu-Li (Gordon) Tai, Bowen Li

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              eskabetxe Joao Boto
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: