Uploaded image for project: 'Apache IoTDB'
  1. Apache IoTDB
  2. IOTDB-1442

[Distributed] Time filter & TTL is not used during the construction of ClusterTimeGenerator

    XMLWordPrintableJSON

Details

    Description

      When executing a read/write workload over an IoTDB cluster where queries involving value filters are included, the overall throughput decreases over time and the disk usage keeps increasing and is dominated by disk reads.

      We found out that the number of chunk reads was significantly high compared with the number of queries through profiling. Considering that the query time range was fixed, this phenomenon was abnormal.

      By intuition, we suspected that the time filter did not work expectedly and it turned a range query into a full query. So we checked the code where the readers are generated in the cluster module and found the following mistake that time filter is not provided during the reader construction. It seems that the time filter is included in the whole filter so it did not matter whether to pass it separately or not previously, but recent modifications required to extract it and pass it explicitly.

      The solution is rather simple, we can extract the time filter and pass it to the constructor so the time filter will not be missing.

      We also found out that TTL is not introduced during the reader construction, but it can be added readily.

      Attachments

        1. image-2021-06-17-10-03-18-771.png
          35 kB
          Tian Jiang
        2. image-2021-06-17-09-50-28-254.png
          8 kB
          Tian Jiang

        Activity

          People

            jt2594838 Tian Jiang
            jt2594838 Tian Jiang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: