Description
In Hive to make vectorization work, the reader needs to be also vectorized, which means that the reader can read a chunk of rows (or a list of column chunks) instead of one row at a time. However, we use Spark RDD for reading, which again utilized the underlying inputformat to read. Subsequent processing also needs to hapen in batches. We need to make sure that vectorizatoin is working as expected.
Attachments
Issue Links
- relates to
-
HIVE-7333 Create RDD translator, translating Hive Tables into Spark RDDs [Spark Branch]
-
- Resolved
-