Details
-
New Feature
-
Status: In Progress
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
- Desciprtion
When IoTDB is executing compaction and merge, sometimes we want to monitor the performance of it. In the past, we could only judge whether the compaction and merge was in progress through the logs printed before and after the compaction and merge, and judge the frequency of the compaction and merge by analyzing the frequency of compaction and merge log printing in all logs. But this is very troublesome, and the analysis is not accurate. So we hope to have a tool that can accurately monitor the performance of the compaction and merge.
- Features
To achieve our goals, we use a `CompactionMonitor` service to monitor compaction and merge performance. In more detail, we concern about the following index:
- The cpu consumption percentage of compaction rhreads and merge threads.
- The number of files participating in compaction.
- The number of files participating in merge.
- The number of compaction of each sg.
- The number of merge of each sg.
`CompactionMonitor` collects these data periodically, and save the data to the IoTDB, using a storage group named "root.compaction_monitor". Moreover, We wrote a python script to visualize these data. The repository of visualization script is https://github.com/THUMarkLau/iotdb-compaction-monitor-visualize .
