[TAJO-1148] Support for offheap-based scanner

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Storage
• Assignee: Jinho Kim
• Reporter: Jinho Kim
• Created: 31/Oct/14 11:59
• Updated: 20/Nov/14 05:55

Description

Currently, scanners implemented the heap-based codes. so that we met often GC puase.
We should refactor the codes for memory efficient

• change to ByteBuffer-based read scanner
• change to Offheap-based buffer

https://issues.apache.org/jira/browse/HDFS-2834
http://netty.io/wiki/using-as-a-generic-library.html

Issue Links

• relates to: TAJO-631 Zero copy physical layer processing
• relates to: TAJO-1041 Tuple Block Iteration Model

Sub-Tasks

• Implement direct read of DelimitedTextFile: Resolved
• Implement the ByteBuffer-based De/Serializer: Resolved
• RawFile ByteBuffer should be reuse: Resolved
• Implement direct read of RCFile: Resolved
• Implement direct read of SequenceFile: Resolved
• Implement direct read of Parquet: Open