[IOTDB-3702] [ wal recovery ] wal recovery by thread pool to improve performance

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Component/s: Core/WAL, mpp-cluster
• Labels: pull-request-available
• Sprint: 2022-7-Cluster
• Assignee: Haiming Zhu
• Reporter: 刘珍
• Created: 30/Jun/22 03:35
• Updated: 04/Jul/22 13:39
• Resolved: 04/Jul/22 13:39

Description

通过main恢复慢：

从日志看出，耗时约1小时50分钟：
2022-06-30 10:05:30,751 [pool-16-IoTDB-WAL-Recover-1] INFO o.a.i.d.w.r.WALNodeRecoverTask:94 - Successfully recover WAL node in the directory ./sbin/../data/wal/0, so delete these wal files.
2022-06-30 10:05:30,752 [pool-16-IoTDB-WAL-Recover-2] INFO o.a.i.d.w.r.WALNodeRecoverTask:94 - Successfully recover WAL node in the directory ./sbin/../data/wal/1, so delete these wal files.
*2022-06-30 10:05:56,291 *[main] WARN o.a.i.t.r.TsFileSequenceReader:1527 - TsFile /data/liuzhen_test/schema_file_0629/v1/datanode/./sbin/../data/data/sequence/root.test.g_15/0/0/1656512061163-1-0-0.tsfile self-check cannot proceed at position 48644141 recovered, because : reach the end of the file.
2022-06-30 11:11:03,284 [Service Thread] INFO o.a.i.m.m.MicrometerMetricManager:200 - create getOrCreateTimer jvm.gc.pause
2022-06-30 11:11:03,285 [Service Thread] INFO o.a.i.m.m.MicrometerMetricManager:200 - create getOrCreateTimer jvm.gc.pause
2022-06-30 11:53:51,717 [pool-14-IoTDB-Data-Region-Recover-7] INFO o.a.i.d.e.s.DataRegion$DataRegionRecoveryContext:380 - The data region root.test.g_36[0] has recovered 1.0%, please wait a moment.

（数据备份：192.168.10.64 /data/liuzhen_test/schema_file_0629/v1/datanode/bk_2022-06-30_02_56_11_50client_normal
）

Issue Links

• links to: GitHub Pull Request #6533