[FLINK-34384] Release Testing: Verify FLINK-33735 Improve the exponential-delay restart-strategy

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.19.0
• Fix Version/s: 1.19.0
• Component/s: Runtime / Coordination
• Labels: release-testing
• Assignee: Caican Cai
• Reporter: lincoln lee
• Created: 06/Feb/24 06:56
• Updated: 07/Feb/24 06:03
• Resolved: 07/Feb/24 06:03

Description

Test suggestion:

1. Prepare a datastream job that all tasks throw exception directly.
   1. Set the parallelism to 5 or above
2. Prepare some configuration options:
   • restart-strategy.type : exponential-delay
   • restart-strategy.exponential-delay.attempts-before-reset-backoff : 7
3. Start the cluster: ./bin/start-cluster.sh
4. Run the job: ./bin/flink run -c className jarName
5. Check the result
   • Check whether job will be retried 7 times
   • Check the exception history, the list has 7 exceptions
   • Each retries except the last one can see the 5 subtasks(They are concurrent exceptions).

 
 

Note: Set these options mentioned at step2 at 2 level separately

• Cluster level: set them in the config.yaml
• Job level: Set them in the code

 

Job level demo:

public static void main(String[] args) throws Exception {
    Configuration conf = new Configuration();

    conf.setString("restart-strategy", "exponential-delay");
    conf.setString("restart-strategy.exponential-delay.attempts-before-reset-backoff", "6");
    StreamExecutionEnvironment env =  StreamExecutionEnvironment.getExecutionEnvironment(conf);
    env.setParallelism(5);

    DataGeneratorSource<Long> generatorSource =
            new DataGeneratorSource<>(
                    value -> value,
                    300,
                    RateLimiterStrategy.perSecond(10),
                    Types.LONG);

    env.fromSource(generatorSource, WatermarkStrategy.noWatermarks(), "Data Generator")
            .map(new RichMapFunction<Long, Long>() {
                @Override
                public Long map(Long value) {
                    throw new RuntimeException(
                            "Excepted testing exception, subtaskIndex: " + getRuntimeContext().getIndexOfThisSubtask());
                }
            })
            .print();

    env.execute();
} 

Issue Links

• is a clone of: FLINK-34288 Release Testing Instructions: Verify FLINK-33735 Improve the exponential-delay restart-strategy