XMLWordPrintableJSON

    Details

    • Release Note:
      Hide
      In case when partitions are not (yet) found, because we are requesting partitions sooner (now we are requesting partitions before initialise operator states), partitions request re-triggering will also happen sooner, so the incremental back of counter will start counting also sooner.

      It is possible that some Jobs (with long deployment times and large state) might start failing more frequently with {{PartitionNotFound}} exceptions compared to Flink 1.8. If that's the case users should increase {{taskmanager.network.request-backoff.max}} value in order to have the same effective request partitions time out as it was in Flink 1.8.
      Show
      In case when partitions are not (yet) found, because we are requesting partitions sooner (now we are requesting partitions before initialise operator states), partitions request re-triggering will also happen sooner, so the incremental back of counter will start counting also sooner. It is possible that some Jobs (with long deployment times and large state) might start failing more frequently with {{PartitionNotFound}} exceptions compared to Flink 1.8. If that's the case users should increase {{taskmanager.network.request-backoff.max}} value in order to have the same effective request partitions time out as it was in Flink 1.8.

      Description

      Currently SingleInputGate requests partitions only on the first attempt to fetch the data. Before requesting partitions, no data availability notifications can get through. This doesn't work well with a new non blocking InputGate interface, since on newly created SingleInputGates, InputGate#isAvailable() might return not available, and it will be only able to switch to available, after first call to SingleInputGate#pollNext(). However this might never happen, since caller could be waiting indefinitely on SingleInputGate#isAvailable().

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                pnowojski Piotr Nowojski
                Reporter:
                pnowojski Piotr Nowojski
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m