Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-8017

Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark Branch]

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.1.0
    • Spark

    Description

      HiveKey should be used as the key type because it holds the hash code for partitioning. While BytesWritable serves partitioning well for simple cases, we have to use HiveKey.hashCode for more complicated ones, e.g. join, bucketed table, etc.

      Attachments

        1. HIVE-8017.2-spark.patch
          76 kB
          Rui Li
        2. HIVE-8017.3-spark.patch
          97 kB
          Rui Li
        3. HIVE-8017.4-spark.patch
          75 kB
          Rui Li
        4. HIVE-8017.5-spark.patch
          76 kB
          Rui Li
        5. HIVE-8017-spark.patch
          31 kB
          Rui Li

        Issue Links

          is depended upon by

          Sub-task - The sub-task of the issue HIVE-7856 Enable parallelism in Reduce Side Join [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-7956 When inserting into a bucketed table, all data goes to a single bucket [Spark Branch]

          • Major - Major loss of function.
          • Resolved
          is related to

          Task - A task that needs to be done. HIVE-8098 The spark golden file for union_remove_25 is different from MR version [Spark Branch]

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          Test - A new unit, integration or system test. HIVE-8035 Add SORT_QUERY_RESULTS for test that doesn't guarantee order

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          links to

          Web Link RB request

          Activity

            People

              lirui Rui Li
              lirui Rui Li
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: