Description
Currently HCatRecordReader copies the underlying LazyHCatTuple into a DefaultHCatTuple. In cases where the underlying record is from an RCFile this is very inefficient, as it forces a read of all the columns. We need a way in this case to copy the underlying Writable returned by RCFileInputFormat and then just annotate it with necessary partition columns. In these cases the early projection provided by the caller should be ignored, though some remapping of the schema may be necessary to make sure that column positions appear correct to the caller.
Attachments
Issue Links
- relates to
-
HCATALOG-258 LazyHCatTuple introduction to prevent paying full cost of deserialization of LazyHCatRecord
-
- Open
-