Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-3816

Typo in the MapRedTask? Which may affect the debugging purpose with IDE.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Trivial
    • Resolution: Unresolved
    • Affects Version/s: 0.9.1
    • Fix Version/s: 0.9.1
    • Component/s: CLI
    • Labels:
      None

      Description

      There may be a typo in the java file

      org.apache.hadoop.hive.ql.exec.MapRedTask
            // Origin: here may have a typo?
         // runningViaChild = ShimLoader.getHadoopShims().isLocalMode(conf) ||
         //   conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD);
            // New: should be the "!"
            runningViaChild = !ShimLoader.getHadoopShims().isLocalMode(conf) ||
              conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD);
      
            if(!runningViaChild) {
              // we are not running this mapred task via child jvm
              // so directly invoke ExecDriver
              return super.execute(driverContext);
            }
      

      Per my understanding, there would be 3 cases:
      1) Run the job in a NEW Child process(maybe pseudo hadoop mode) in the same node
      mapred.job.tracker=local
      hive.exec.submitviachild=true

      2) Run the job in the same process in the same node (very helpful in debugging)
      mapred.job.tracker=local
      hive.exec.submitviachild=false

      3) Run the job in a real Hadoop Cluster specified
      mapred.job.tracker!=local
      hive.exec.submitviachild=true/false

      The typo may cause the second case NEVER happens, which we need in developing (debugging);

      Sorry, not sure if I understand that correctly.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              chenghao Cheng Hao
            • Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: