Hive
  1. Hive
  2. HIVE-4160 Vectorized Query Execution in Hive
  3. HIVE-4606

Implement vector group by hash spill

        Details

          Description

          the row mode GBy operator is capable of spilling the hash map (by emitting it into the M/R and deleting the keys emitted) to conform to emmory constraints. The VGBy should be able to do the same.

          1. Text File
            HIVE-4606.1.patch.txt
            94 kB
            Remus Rusanu
          2. Text File
            HIVE-4606.2.patch.txt
            94 kB
            Remus Rusanu
          3. Text File
            HIVE-4606.3.patch.txt
            93 kB
            Remus Rusanu

            Activity

            Ascending order - Click to sort in descending order
            Remus Rusanu created issue -
            Remus Rusanu made changes -
            Field Original Value New Value
            Status Open [ 1 ] Patch Available [ 10002 ]
            Fix Version/s vectorization-branch [ 12324334 ]
            Remus Rusanu made changes -
            Attachment HIVE-4606.1.patch.txt [ 12586868 ]
            Hide
            Permalink
            Remus Rusanu added a comment -

            https://reviews.apache.org/r/11747/

            Show
            Remus Rusanu added a comment - https://reviews.apache.org/r/11747/
            Remus Rusanu made changes -
            Attachment HIVE-4606.2.patch.txt [ 12586882 ]
            Hide
            Permalink
            Ashutosh Chauhan added a comment -

            It probably is a good idea to use(or enhance) ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java for estimating memory sizes of object than adding a new one.

            Show
            Ashutosh Chauhan added a comment - It probably is a good idea to use(or enhance) ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java for estimating memory sizes of object than adding a new one.
            Hide
            Permalink
            Remus Rusanu added a comment -

            Removed JavaObjectSizeUtil, replaced with JavaDataModel

            Show
            Remus Rusanu added a comment - Removed JavaObjectSizeUtil, replaced with JavaDataModel
            Remus Rusanu made changes -
            Attachment HIVE-4606.3.patch.txt [ 12587226 ]
            Hide
            Permalink
            Ashutosh Chauhan added a comment -

            Committed to branch. Thanks, Remus!

            Show
            Ashutosh Chauhan added a comment - Committed to branch. Thanks, Remus!
            Ashutosh Chauhan made changes -
            Status Patch Available [ 10002 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Thejas M Nair made changes -
            Fix Version/s 0.13.0 [ 12324986 ]

              People

              • Assignee:
                Remus Rusanu
                Reporter:
                Remus Rusanu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development