Hive
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-7328

Refactoring: make Hive reduce side data processing reusable [Spark Branch]

    Details

    • Type: Sub-task Sub-task
    • Status: Reopened
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.13.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      ExecReducer is Hive's reducer implementation for MapReduce. Table rows are shuffled by MR framework to ExecReducer and further processed by ExecReducer.reduce() method, which invokes Hive's reduce-side operator tree starting. This task is to extract the reduce-side data processing offered by the operator tree so that it can be reused by other execution engine such as Spark. This is purely refactoring the existing code.

        Activity

        Hide
        Xuefu Zhang added a comment -

        It seems it's easier to use ExecReducer directly than any refactoring. Postpone this item for now for later consideration.

        Show
        Xuefu Zhang added a comment - It seems it's easier to use ExecReducer directly than any refactoring. Postpone this item for now for later consideration.
        Hide
        Xuefu Zhang added a comment -

        Closed as "will not fix". Will reopen if need comes back.

        Show
        Xuefu Zhang added a comment - Closed as "will not fix". Will reopen if need comes back.
        Hide
        Chengxiang Li added a comment -

        We have implement native HiveReduceFunction instead of using ExecRedecer directly in HIVE-7677, reopen this JIRA to track further refactoring.

        Show
        Chengxiang Li added a comment - We have implement native HiveReduceFunction instead of using ExecRedecer directly in HIVE-7677 , reopen this JIRA to track further refactoring.

          People

          • Assignee:
            Xuefu Zhang
            Reporter:
            Xuefu Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development