Uploaded image for project: 'Apache IoTDB'
  1. Apache IoTDB
  2. IOTDB-6036

[delete timeseries] the mods file is too large, causing Query very slow even OOM problem

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Blocker
    • Resolution: Unresolved
    • None
    • master branch
    • Core/Server
    • None

    Description

      In some scenarios, users perform ttl operations by deleting the time series.

      There is a user every 5 seconds to delete 14 days old data, pseudo-SQL is as follows:

      delete timeseres root.cisdi.im.steel.sentry.** where time < systemTime - 14d 

      Each execution of this SQL will append a line of records to the mods file, and after a while, the file will become very large, as shown in the attachment, the mods file has reached 200 of megabytes, and the deleted records in the file have been about 3 million lines.

       

      When you query the contents related to this sequence, the query engine filters out the invalid contents in the mods file. Therefore, all the contents of the mods file are loaded into the memory. This operation may cause that the query result cannot be loaded for a long time or even the OOM result is displayed.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            caozhijia1 Zhijia Cao
            caozhijia1 Zhijia Cao

            Dates

              Created:
              Updated:

              Slack

                Issue deployment