Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-3816

Typo in the MapRedTask? Which may affect the debugging purpose with IDE.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Trivial
    • Resolution: Unresolved
    • 0.9.1
    • 0.9.1
    • CLI
    • None

    Description

      There may be a typo in the java file

      org.apache.hadoop.hive.ql.exec.MapRedTask
            // Origin: here may have a typo?
         // runningViaChild = ShimLoader.getHadoopShims().isLocalMode(conf) ||
         //   conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD);
            // New: should be the "!"
            runningViaChild = !ShimLoader.getHadoopShims().isLocalMode(conf) ||
              conf.getBoolVar(HiveConf.ConfVars.SUBMITVIACHILD);
      
            if(!runningViaChild) {
              // we are not running this mapred task via child jvm
              // so directly invoke ExecDriver
              return super.execute(driverContext);
            }
      

      Per my understanding, there would be 3 cases:
      1) Run the job in a NEW Child process(maybe pseudo hadoop mode) in the same node
      mapred.job.tracker=local
      hive.exec.submitviachild=true

      2) Run the job in the same process in the same node (very helpful in debugging)
      mapred.job.tracker=local
      hive.exec.submitviachild=false

      3) Run the job in a real Hadoop Cluster specified
      mapred.job.tracker!=local
      hive.exec.submitviachild=true/false

      The typo may cause the second case NEVER happens, which we need in developing (debugging);

      Sorry, not sure if I understand that correctly.

      Attachments

        Activity

          People

            Unassigned Unassigned
            chenghao Cheng Hao
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated: