In here , Anoop Sam John has a comment:
There is a concern here. Even if the block is on an exclusive heap memory area, we will keep ref to that in this list. In a Phoenix Aggregation kind of use case where many blocks might get fetched and not immediately shipped, we are keeping the ref unwantedly here for longer time. This makes the GC not able to reclaim the heap memory area for the blocks. This might be a hidden bomb IMO. Its not good to remove the MemType. Lets create the block with memory type as EXCLUSIVE when the block data is on heap. The block might be coming from LRU cache or by fetching the block data from HDFS into heap memory area. When the block comes from off heap BC or if it is backed by a BB from the pool (While reading from HDFS, read into pooled BB) lets create the block with mem type as SHARED. Every block can have the retain and release method but let the EXCLUSIVE types do a noop here.
We've a discussion about this, say need to address two thing in this jira:
1. separate the HFileBlock into shared or non-shared;
2. Make the retain/release of non-shared block as a noop, say don't do reference count change for heap block.