[FLINK-14043] SavepointMigrationTestBase is super slow

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.1, 1.9.0, 1.10.0
• Fix Version/s: 1.8.3, 1.9.1, 1.10.0
• Component/s: Runtime / State Backends, Tests
• Labels: pull-request-available
• Assignee: Till Rohrmann
• Reporter: Till Rohrmann
• Created: 10/Sep/19 13:22
• Updated: 11/Sep/19 12:27
• Resolved: 11/Sep/19 12:27

Description

The subclasses of SavepointMigrationTestBase take super long to execute. On my local machine

• TypeSerializerSnapshotMigrationITCase takes 2min 30s
• StatefulJobWBroadcastStateMigrationITCase takes 1min 45s
• StatefulJobSavepointMigrationITCase takes 2min 5s

to execute. The reasons for the long runtimes seem to be that we are using the AccumulatorCountingSink which uses the accumulators to signal when a job is done. Since the accumulators are being sent with the TM heartbeats, the heartbeat interval how fast the client realizes that the job can be shut down. The default heartbeat interval is 10 s and hence it takes always at least 10 seconds until the client stops the job.

I suggest to decrease the heartbeat interval in the SavepointMigrationTestBase to 300ms in order to speed up the tests. On my machine the test runtimes with this settings are:

• TypeSerializerSnapshotMigrationITCase takes 13s
• StatefulJobWBroadcastStateMigrationITCase takes 10s
• StatefulJobSavepointMigrationITCase takes 11s

Issue Links

• links to: GitHub Pull Request #9664