Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-7600

shorten delay of KinesisProducerConfiguration.setCredentialsRefreshDelay() to avoid updateCredentials Exception

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.3.2
    • Fix Version/s: 1.4.0, 1.3.3
    • Component/s: Kinesis Connector
    • Labels:
      None

      Description

      we saw the following warning in Flink log:

      2017-08-11 02:33:24,473 WARN  org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon  - Exception during updateCredentials
      java.lang.InterruptedException: sleep interrupted
      	at java.lang.Thread.sleep(Native Method)
      	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon$5.run(Daemon.java:316)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

      According to discussion in https://github.com/awslabs/amazon-kinesis-producer/issues/10, setting the delay to 100 will fix this issue

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user bowenli86 opened a pull request:

          https://github.com/apache/flink/pull/4657

          FLINK-7600[kinesis] shorten delay of KinesisProducerConfiguration.setCredentialsRefreshDelay() to avoid updateCredentials Exception

            1. What is the purpose of the change

          we saw the following warning in Flink log:

          ```
          2017-08-11 02:33:24,473 WARN org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon - Exception during updateCredentials
          java.lang.InterruptedException: sleep interrupted
          at java.lang.Thread.sleep(Native Method)
          at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon$5.run(Daemon.java:316)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
          ```

          According to discussion in https://github.com/awslabs/amazon-kinesis-producer/issues/10, setting the delay to 100 will fix this issue

            1. Brief change log
          • shorten aws credentials refresh delay to 100 millisec
            1. Verifying this change

          This change is a trivial rework / code cleanup without any test coverage.

          I've been running the fixed flink-connector-kinesis for a few days on AWS, and that warning log never happens again

            1. Does this pull request potentially affect one of the following parts:
            1. Documentation
          • Does this pull request introduce a new feature? (no)

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/bowenli86/flink FLINK-7600

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/4657.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #4657


          commit 6386983239bd3024b395c865ec4fd33e232ca5a3
          Author: Bowen Li <bowenli86@gmail.com>
          Date: 2017-08-30T16:35:03Z

          FLINK-7422 Upgrade Kinesis Client Library (KCL) and AWS SDK in flink-connector-kinesis

          commit 381cd4156b84673a1d32d2db3f7b2d748d90d980
          Author: Bowen Li <bowenli86@gmail.com>
          Date: 2017-09-07T06:33:37Z

          Merge remote-tracking branch 'upstream/master'

          commit 38e8654142939061aa9595d67311619c9eb759ec
          Author: Bowen Li <bowenli86@gmail.com>
          Date: 2017-09-07T21:36:28Z

          FLINK-7600 shorten delay of KinesisProducerConfiguration.setCredentialsRefreshDelay() to avoid updateCredentials Exception


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user bowenli86 opened a pull request: https://github.com/apache/flink/pull/4657 FLINK-7600 [kinesis] shorten delay of KinesisProducerConfiguration.setCredentialsRefreshDelay() to avoid updateCredentials Exception What is the purpose of the change we saw the following warning in Flink log: ``` 2017-08-11 02:33:24,473 WARN org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon - Exception during updateCredentials java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon$5.run(Daemon.java:316) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ``` According to discussion in https://github.com/awslabs/amazon-kinesis-producer/issues/10 , setting the delay to 100 will fix this issue Brief change log shorten aws credentials refresh delay to 100 millisec Verifying this change This change is a trivial rework / code cleanup without any test coverage. I've been running the fixed flink-connector-kinesis for a few days on AWS, and that warning log never happens again Does this pull request potentially affect one of the following parts: Documentation Does this pull request introduce a new feature? (no) You can merge this pull request into a Git repository by running: $ git pull https://github.com/bowenli86/flink FLINK-7600 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4657.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4657 commit 6386983239bd3024b395c865ec4fd33e232ca5a3 Author: Bowen Li <bowenli86@gmail.com> Date: 2017-08-30T16:35:03Z FLINK-7422 Upgrade Kinesis Client Library (KCL) and AWS SDK in flink-connector-kinesis commit 381cd4156b84673a1d32d2db3f7b2d748d90d980 Author: Bowen Li <bowenli86@gmail.com> Date: 2017-09-07T06:33:37Z Merge remote-tracking branch 'upstream/master' commit 38e8654142939061aa9595d67311619c9eb759ec Author: Bowen Li <bowenli86@gmail.com> Date: 2017-09-07T21:36:28Z FLINK-7600 shorten delay of KinesisProducerConfiguration.setCredentialsRefreshDelay() to avoid updateCredentials Exception
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bowenli86 commented on the issue:

          https://github.com/apache/flink/pull/4657

          @tzulitai Hi Gordon, can you please take a look at this PR?

          Show
          githubbot ASF GitHub Bot added a comment - Github user bowenli86 commented on the issue: https://github.com/apache/flink/pull/4657 @tzulitai Hi Gordon, can you please take a look at this PR?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user tzulitai commented on the issue:

          https://github.com/apache/flink/pull/4657

          LGTM, thanks for the investigation.

          I would add a comment about the specific change though, otherwise it would be meaningless for other readers. Will add that when merging.

          Show
          githubbot ASF GitHub Bot added a comment - Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/4657 LGTM, thanks for the investigation. I would add a comment about the specific change though, otherwise it would be meaningless for other readers. Will add that when merging.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bowenli86 commented on the issue:

          https://github.com/apache/flink/pull/4657

          @tzulitai Thank you, Gordon!

          Show
          githubbot ASF GitHub Bot added a comment - Github user bowenli86 commented on the issue: https://github.com/apache/flink/pull/4657 @tzulitai Thank you, Gordon!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/4657

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4657
          Hide
          tzulitai Tzu-Li (Gordon) Tai added a comment -

          Thanks for the contribution!

          Fixed for master via 6c1a946562ec8ac4825b871aefdec040cc02aaf2.
          Fixed for 1.3.3 via cc0de8486b2c83bf03da0f8115df4b5ff72f6ed6.

          Show
          tzulitai Tzu-Li (Gordon) Tai added a comment - Thanks for the contribution! Fixed for master via 6c1a946562ec8ac4825b871aefdec040cc02aaf2. Fixed for 1.3.3 via cc0de8486b2c83bf03da0f8115df4b5ff72f6ed6.

            People

            • Assignee:
              phoenixjiangnan Bowen Li
              Reporter:
              phoenixjiangnan Bowen Li
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development