Hive
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-7327

Refactoring: make Hive map side data processing reusable [Spark Branch]

    Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not A Problem
    • Affects Version/s: 0.13.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      ExecMapper is Hive's mapper implementation for MapReduce. Table rows are read by MR framework and processed by ExecMapper.map() method, which invokes Hive's map-side operator tree starting from MapOperator. This task is to extract the map-side data processing offered by the operator tree so that it can be used by other execution engine such as Spark. This is purely refactoring the existing code.

        Activity

        Xuefu Zhang created issue -
        Xuefu Zhang made changes -
        Field Original Value New Value
        Summary Refactoring: Make Hive map side data processing reusable Refactoring: make Hive map side data processing reusable
        Hide
        Xuefu Zhang added a comment -

        It seems it's easier to use ExecMapper directly than any refactoring. Postpone this item for now for later consideration.

        Show
        Xuefu Zhang added a comment - It seems it's easier to use ExecMapper directly than any refactoring. Postpone this item for now for later consideration.
        Hide
        Xuefu Zhang added a comment -

        Closed as not fix. Will reopen if need comes back.

        Show
        Xuefu Zhang added a comment - Closed as not fix. Will reopen if need comes back.
        Xuefu Zhang made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Won't Fix [ 2 ]
        Hide
        Chengxiang Li added a comment -

        We have implement native HiveMapFunction instead of using ExecMapper directly in HIVE-7675, reopen this JIRA to track further refactoring.

        Show
        Chengxiang Li added a comment - We have implement native HiveMapFunction instead of using ExecMapper directly in HIVE-7675 , reopen this JIRA to track further refactoring.
        Chengxiang Li made changes -
        Resolution Won't Fix [ 2 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Brock Noland made changes -
        Summary Refactoring: make Hive map side data processing reusable Refactoring: make Hive map side data processing reusable [Spark Branch]
        Xuefu Zhang made changes -
        Labels Spark-M1
        Hide
        Xuefu Zhang added a comment -

        Since Hive on Spark doesn't use ExecMapper directly now, this is no longer an issue. Closed this for now.

        Show
        Xuefu Zhang added a comment - Since Hive on Spark doesn't use ExecMapper directly now, this is no longer an issue. Closed this for now.
        Xuefu Zhang made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Resolution Not a Problem [ 8 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        27d 15h 11m 1 Xuefu Zhang 30/Jul/14 10:49
        Resolved Resolved Reopened Reopened
        15d 16h 4m 1 Chengxiang Li 15/Aug/14 02:53
        Reopened Reopened Resolved Resolved
        21d 21h 48m 1 Xuefu Zhang 06/Sep/14 00:42

          People

          • Assignee:
            Xuefu Zhang
            Reporter:
            Xuefu Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development