Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-7675

Implement native HiveMapFunction [Spark Branch]

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Spark
    • Labels:
      None

      Description

      Currently, Hive on Spark depend on ExecMapper to execute operator logic, full stack is like: Spark FrameWork=>HiveMapFunction=>ExecMapper=>Hive operators. HiveMapFunction is just a thin wrapper of ExecMapper, this introduce several problems as following:

      1. ExecMapper is designed for MR single process task mode, it does not work well under Spark multi-thread task node.
      2. ExecMapper introduce extra API level restriction and process logic.

      We need implement native HiveMapFunction, as the bridge between Spark framework and Hive operators.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                chengxiang li Chengxiang Li
                Reporter:
                chengxiang li Chengxiang Li
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: