[IOTDB-6036] [delete timeseries] the mods file is too large, causing Query very slow even OOM problem

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Fix Version/s: master branch
• Component/s: Core/Server
• Assignee: Zhijia Cao
• Reporter: Zhijia Cao
• Created: 29/Jun/23 09:36
• Updated: 29/Jun/23 10:04

Description

In some scenarios, users perform ttl operations by deleting the time series.

There is a user every 5 seconds to delete 14 days old data, pseudo-SQL is as follows:

delete timeseres root.cisdi.im.steel.sentry.** where time < systemTime - 14d 

Each execution of this SQL will append a line of records to the mods file, and after a while, the file will become very large, as shown in the attachment, the mods file has reached 200 of megabytes, and the deleted records in the file have been about 3 million lines.

 

When you query the contents related to this sequence, the query engine filters out the invalid contents in the mods file. Therefore, all the contents of the mods file are loaded into the memory. This operation may cause that the query result cannot be loaded for a long time or even the OOM result is displayed.