Details
-
Bug
-
Status: Reopened
-
Major
-
Resolution: Unresolved
-
0.8.0
-
None
-
None
Description
Currently, if you create a column partitioned table
create table tbl (col1 int4, col2 int4, null_col int4) partition by column(key float8)
and later insert into it
insert overwrite into tbl(col1, col2, key) select l_orderkey, l_partkey, l_quantity from lineitem
From the code of ColumnPartitionedTableStoreExec.java, it seems that we didn't distinguish between the real result columns (col1 and col2 here) and the column partition keys. Just store the result including the partition keys. Hive doesn't save them like this way,cuz the value of partition columns in the same partition are the same.
public Tuple next() throws IOException { StringBuilder sb = new StringBuilder(); while((tuple = child.next()) != null) { // set subpartition directory name sb.delete(0, sb.length()); if (partitionColumnIndices != null) { for(int i = 0; i < partitionColumnIndices.length; i++) { Datum datum = tuple.get(partitionColumnIndices[i]); if(i > 0) sb.append("/"); sb.append(partitionColumnNames[i]).append("="); sb.append(datum.asChars()); } } // add tuple Appender appender = getAppender(sb.toString()); appender.addTuple(tuple); // this line add the whole tuple into result record include the partition keys } ... }
Attachments
Issue Links
- depends upon
-
TAJO-461 Implementation of hash partition
- Open