Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-1408

add option to let hive automatically run in local mode based on tunable heuristics



    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: Query Processor
    • Labels:


      as a followup to HIVE-543 - we should have a simple option (enabled by default) to let hive run in local mode if possible.

      two levels of options are desirable:
      1. hive.exec.mode.local.auto=true/false // control whether local mode is automatically chosen
      2. Options to control different heuristics, some naiive examples:
      hive.exec.mode.local.auto.input.size.max=1G // don't choose local mode if data > 1G
      hive.exec.mode.local.auto.script.enable=true/false // choose if local mode is enabled for queries with user scripts

      this can be implemented as a pre/post execution hook. It makes sense to provide this as a standard hook in the hive codebase since it's likely to improve response time for many users (especially for test queries).

      the initial proposal is to choose this at a query level and not at per hive-task (ie. hadoop job) level. per job-level requires more changes to compilation (to not pre-commit to hdfs or local scratch directories at compile time).


        1. 1408.7.patch
          1.09 MB
          Joydeep Sen Sarma
        2. hive-1408.6.patch
          1.09 MB
          Joydeep Sen Sarma
        3. 1408.2.q.out.patch
          113 kB
          Joydeep Sen Sarma
        4. 1408.2.patch
          1.47 MB
          Joydeep Sen Sarma
        5. 1408.1.patch
          114 kB
          Joydeep Sen Sarma

          Issue Links



              • Assignee:
                jsensarma Joydeep Sen Sarma
                jsensarma Joydeep Sen Sarma
              • Votes:
                0 Vote for this issue
                4 Start watching this issue


                • Created: