Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18622

Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 3.0.0
    • Hive
    • None

    Description

       
      Many vector expression classes are setting noNulls to true which does not work if the VRB is a scratch column being reused. The previous use may have set noNulls to false and the isNull array will have some rows marked as NULL. The result is wrong query results and sometimes NPEs (for BytesColumnVector).

      So, many vector expressions need this:

            // Carefully handle NULLs...
      
            /*
             * For better performance on LONG/DOUBLE we don't want the conditional
             * statements inside the for loop.
             */
            outputColVector.noNulls = false;
       

      And, vector expressions need to make sure the isNull array entry is set when outputColVector.noNulls is false.

      And, all place that assign column value need to set noNulls to false when the value is NULL.

      Almost all cases where noNulls is set to true are incorrect.

      Attachments

        1. HIVE-18622.03.patch
          590 kB
          Matt McCline
        2. HIVE-18622.04.patch
          736 kB
          Matt McCline
        3. HIVE-18622.05.patch
          702 kB
          Matt McCline
        4. HIVE-18622.06.patch
          786 kB
          Matt McCline
        5. HIVE-18622.07.patch
          907 kB
          Matt McCline
        6. HIVE-18622.08.patch
          1.65 MB
          Matt McCline
        7. HIVE-18622.09.patch
          2.03 MB
          Matt McCline
        8. HIVE-18622.091.patch
          2.26 MB
          Matt McCline
        9. HIVE-18622.092.patch
          2.28 MB
          Matt McCline
        10. HIVE-18622.093.patch
          2.29 MB
          Matt McCline
        11. HIVE-18622.094.patch
          2.33 MB
          Matt McCline
        12. HIVE-18622.095.patch
          2.36 MB
          Matt McCline
        13. HIVE-18622.096.patch
          2.36 MB
          Matt McCline
        14. HIVE-18622.097.patch
          2.44 MB
          Matt McCline
        15. HIVE-18622.098.patch
          2.45 MB
          Matt McCline
        16. HIVE-18622.099.patch
          2.43 MB
          Matt McCline
        17. HIVE-18622.0992.patch
          2.43 MB
          Matt McCline
        18. HIVE-18622.0993.patch
          2.43 MB
          Matt McCline
        19. HIVE-18622.0994.patch
          2.45 MB
          Matt McCline

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mmccline Matt McCline Assign to me
            mmccline Matt McCline
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment