Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5528

tests: reduce the retry delay in QueryableStateITCase

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.0, 1.3.0
    • Component/s: Queryable State
    • Labels:
      None

      Description

      The QueryableStateITCase uses a retry of 1 second, e.g. if a queried key does not exist yet. This seems a bit too conservative as the job may not take that long to deploy and especially since getKvStateWithRetries() recovers from failures by retrying.

        Issue Links

          Activity

          Hide
          NicoK Nico Kruber added a comment -

          Using 100ms instead does not impose too much additional query load and reduces the test suite's duration from 16-20s to 13-15s on my machine with the current set of unit tests. Further reductions in the retry delay do not yield more improvements so far.

          Show
          NicoK Nico Kruber added a comment - Using 100ms instead does not impose too much additional query load and reduces the test suite's duration from 16-20s to 13-15s on my machine with the current set of unit tests. Further reductions in the retry delay do not yield more improvements so far.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user NicoK opened a pull request:

          https://github.com/apache/flink/pull/3139

          FLINK-5528[query][tests] reduce the retry delay in QueryableStateITCase

          Using 100ms instead of the 1s previously used does not impose too much
          additional query load and reduces the test suite's duration from 16-20s to
          13-15s on my machine with the current set of unit tests. Further reductions
          in the retry delay do not yield more improvements so far.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/NicoK/flink flink-5528

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3139.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3139


          commit d9070946d6f56e2b48d69b61608b10b063b456b4
          Author: Nico Kruber <nico@data-artisans.com>
          Date: 2017-01-17T14:01:32Z

          FLINK-5528[query][tests] reduce the retry delay in QueryableStateITCase

          Using 100ms instead of the 1s previously used does not impose too much
          additional query load and reduces the test suite's duration from 16-20s to
          13-15s on my machine with the current set of unit tests. Further reductions
          in the retry delay do not yield more improvements so far.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3139 FLINK-5528 [query] [tests] reduce the retry delay in QueryableStateITCase Using 100ms instead of the 1s previously used does not impose too much additional query load and reduces the test suite's duration from 16-20s to 13-15s on my machine with the current set of unit tests. Further reductions in the retry delay do not yield more improvements so far. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NicoK/flink flink-5528 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3139.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3139 commit d9070946d6f56e2b48d69b61608b10b063b456b4 Author: Nico Kruber <nico@data-artisans.com> Date: 2017-01-17T14:01:32Z FLINK-5528 [query] [tests] reduce the retry delay in QueryableStateITCase Using 100ms instead of the 1s previously used does not impose too much additional query load and reduces the test suite's duration from 16-20s to 13-15s on my machine with the current set of unit tests. Further reductions in the retry delay do not yield more improvements so far.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3139#discussion_r97091734

          — Diff: flink-tests/src/test/java/org/apache/flink/test/query/QueryableStateITCase.java —
          @@ -1073,6 +1056,20 @@ public Integer getKey(Tuple2<Integer, Long> value) throws Exception {
          }
          }

          + /**
          + * Runs

          {@link #getKvStateWithRetries(QueryableStateClient, JobID, String, + * int, byte[], FiniteDuration)}

          with a fixed retry delay of 100ms.
          + */
          + private static Future<byte[]> getKvStateWithRetries(
          + final QueryableStateClient client,
          + final JobID jobId,
          + final String queryName,
          + final int key,
          + final byte[] serializedKey) {
          + return getKvStateWithRetries(client, jobId, queryName, key, serializedKey,
          + new FiniteDuration(100, TimeUnit.MILLISECONDS));
          — End diff –

          I will make this a static final field of the class

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on a diff in the pull request: https://github.com/apache/flink/pull/3139#discussion_r97091734 — Diff: flink-tests/src/test/java/org/apache/flink/test/query/QueryableStateITCase.java — @@ -1073,6 +1056,20 @@ public Integer getKey(Tuple2<Integer, Long> value) throws Exception { } } + /** + * Runs {@link #getKvStateWithRetries(QueryableStateClient, JobID, String, + * int, byte[], FiniteDuration)} with a fixed retry delay of 100ms. + */ + private static Future<byte[]> getKvStateWithRetries( + final QueryableStateClient client, + final JobID jobId, + final String queryName, + final int key, + final byte[] serializedKey) { + return getKvStateWithRetries(client, jobId, queryName, key, serializedKey, + new FiniteDuration(100, TimeUnit.MILLISECONDS)); — End diff – I will make this a static final field of the class
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on the issue:

          https://github.com/apache/flink/pull/3139

          Good idea! Will merge this with one minor change (see inline comment).

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on the issue: https://github.com/apache/flink/pull/3139 Good idea! Will merge this with one minor change (see inline comment).
          Hide
          uce Ufuk Celebi added a comment -

          Fixed in 03a1f25 (release-1.2), 8d64263 (master).

          Show
          uce Ufuk Celebi added a comment - Fixed in 03a1f25 (release-1.2), 8d64263 (master).
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3139

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3139

            People

            • Assignee:
              NicoK Nico Kruber
              Reporter:
              NicoK Nico Kruber
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development