Uploaded image for project: 'Community Development'
  1. Community Development
  2. COMDEV-411

Apache IoTDB: use netty as the memory buffer pool to reduce GC problem and take full use of memory

    XMLWordPrintableJSON

Details

    Description

      Apache IoTDB [1] is an Open Source IoT database designed to meet the rigorous data, storage, and analytics requirements of large-scale Internet of Things (IoT) and Industrial Internet of Things (IIoT) applications.

       

      Memory control is very very important for DBMS.

       

      Currently, we are using a customized memory buffer pool, which contains a pool of int[], a pool of long[], a pool of boolean[] and as well as float[] and double[].

       

      However, there are two things left:

      1. it is complex to implement a buffer pool for String[] or byte[][], as the size of String and byte[] are variable.
      2. We are using HeapByteBuffer, while in many cases, directByteBuffer is more efficient.

      As Netty has provided a high efficient buffer pool, we'd like to try to migrate the current buffer pool to the Netty implementation.

       

      Things you should know:

      1. Know Java well
      2. Know Netty well
      3. read codes of IoTDB (mainly in StorageEngine, StorageGroupProcessor and related classes)

       

       

       

       

       

       

       

       

       

       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            hxd Xiangdong Huang
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: