Details

      Description

      the row mode GBy operator is capable of spilling the hash map (by emitting it into the M/R and deleting the keys emitted) to conform to emmory constraints. The VGBy should be able to do the same.

      1. HIVE-4606.1.patch.txt
        94 kB
        Remus Rusanu
      2. HIVE-4606.2.patch.txt
        94 kB
        Remus Rusanu
      3. HIVE-4606.3.patch.txt
        93 kB
        Remus Rusanu

        Activity

        Hide
        Ashutosh Chauhan added a comment -

        Committed to branch. Thanks, Remus!

        Show
        Ashutosh Chauhan added a comment - Committed to branch. Thanks, Remus!
        Hide
        Remus Rusanu added a comment -

        Removed JavaObjectSizeUtil, replaced with JavaDataModel

        Show
        Remus Rusanu added a comment - Removed JavaObjectSizeUtil, replaced with JavaDataModel
        Hide
        Ashutosh Chauhan added a comment -

        It probably is a good idea to use(or enhance) ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java for estimating memory sizes of object than adding a new one.

        Show
        Ashutosh Chauhan added a comment - It probably is a good idea to use(or enhance) ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java for estimating memory sizes of object than adding a new one.
        Show
        Remus Rusanu added a comment - https://reviews.apache.org/r/11747/

          People

          • Assignee:
            Remus Rusanu
            Reporter:
            Remus Rusanu
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development