Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12827

Vectorization: VectorCopyRow/VectorAssignRow/VectorDeserializeRow assign needs explicit isNull[offset] modification

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      commit 9cab4414caf1bba2eb1852536a9d3676ba7eab21
      Author: Gopal V <gopalv@apache.org>
      Date: Mon Jan 18 16:03:40 2016 -0800

          Vectorization: VectorCopyRow/VectorAssignRow/VectorDeserializeRow assign needs explicit isNull[offset] modification (Gopal V, reviewed by Sergey Shelukhin)
          
          Signed-off-by: Gopal V <gopalv@apache.org>
      Show
      commit 9cab4414caf1bba2eb1852536a9d3676ba7eab21 Author: Gopal V < gopalv@apache.org > Date: Mon Jan 18 16:03:40 2016 -0800     Vectorization: VectorCopyRow/VectorAssignRow/VectorDeserializeRow assign needs explicit isNull[offset] modification (Gopal V, reviewed by Sergey Shelukhin)          Signed-off-by: Gopal V < gopalv@apache.org >

      Description

      Some scenarios do set Double.NaN instead of isNull=true, but all types aren't consistent.

      Examples of un-set isNull for the valid values are

        private class FloatReader extends AbstractDoubleReader {
      
          FloatReader(int columnIndex) {
            super(columnIndex);
          }
      
          @Override
          void apply(VectorizedRowBatch batch, int batchIndex) throws IOException {
            DoubleColumnVector colVector = (DoubleColumnVector) batch.cols[columnIndex];
      
            if (deserializeRead.readCheckNull()) {
              VectorizedBatchUtil.setNullColIsNullValue(colVector, batchIndex);
            } else {
              float value = deserializeRead.readFloat();
              colVector.vector[batchIndex] = (double) value;
            }
          }
        }
      
        private class DoubleCopyRow extends CopyRow {
      
          DoubleCopyRow(int inColumnIndex, int outColumnIndex) {
            super(inColumnIndex, outColumnIndex);
          }
      
          @Override
          void copy(VectorizedRowBatch inBatch, int inBatchIndex, VectorizedRowBatch outBatch, int outBatchIndex) {
            DoubleColumnVector inColVector = (DoubleColumnVector) inBatch.cols[inColumnIndex];
            DoubleColumnVector outColVector = (DoubleColumnVector) outBatch.cols[outColumnIndex];
      
            if (inColVector.isRepeating) {
              if (inColVector.noNulls || !inColVector.isNull[0]) {
                outColVector.vector[outBatchIndex] = inColVector.vector[0];
              } else {
                VectorizedBatchUtil.setNullColIsNullValue(outColVector, outBatchIndex);
              }
            } else {
              if (inColVector.noNulls || !inColVector.isNull[inBatchIndex]) {
                outColVector.vector[outBatchIndex] = inColVector.vector[inBatchIndex];
              } else {
                VectorizedBatchUtil.setNullColIsNullValue(outColVector, outBatchIndex);
              }
            }
          }
        }
      
       private static abstract class VectorDoubleColumnAssign
          extends VectorColumnAssignVectorBase<DoubleColumnVector> {
      
          protected void assignDouble(double value, int destIndex) {
            outCol.vector[destIndex] = value;
          }
        }
      

      The pattern to imitate would be the earlier code from VectorBatchUtil

          case DOUBLE: {
            DoubleColumnVector dcv = (DoubleColumnVector) batch.cols[offset + colIndex];
            if (writableCol != null) {
              dcv.vector[rowIndex] = ((DoubleWritable) writableCol).get();
              dcv.isNull[rowIndex] = false;
            } else {
              dcv.vector[rowIndex] = Double.NaN;
              setNullColIsNullValue(dcv, rowIndex);
            }
          }
            break;
      

        Attachments

        1. HIVE-12827.2.patch
          19 kB
          Gopal Vijayaraghavan

          Issue Links

            Activity

              People

              • Assignee:
                gopalv Gopal Vijayaraghavan
                Reporter:
                gopalv Gopal Vijayaraghavan
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: