Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-7289 Memory allocation of RocksDB can be problematic in container environments
  3. FLINK-15512

Refactor the mechanism of how to constructure the cache and write buffer manager shared across RocksDB instances

    XMLWordPrintableJSON

Details

    Description

      FLINK-14484 introduce a LRUCache to share among RocksDB instances, so that the memory usage by RocksDB could be controlled well. However, due to the implementation and some bugs in RocksDB (issue-6247), we cannot limit the memory strictly.

      The way to walk around this issue is to consider the buffer which memtable would overuse (1/2 write buffer manager size). By introducing this, the actual cache size for user to share is not the same as the managed off-heap memory or user configured memory.

      Attachments

        Issue Links

          Activity

            People

              yunta Yun Tang
              yunta Yun Tang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m