[SPARK-33358] Spark SQL CLI command processing loop can't exit while one comand fail

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.0.0, 3.0.1, 3.1.0
• Fix Version/s: 3.0.2, 3.1.0
• Component/s: SQL
• Labels: correctness
• Assignee: Lichuanliang
• Reporter: Lichuanliang
• Created: 05/Nov/20 09:57
• Updated: 16/Nov/20 05:50
• Resolved: 16/Nov/20 00:59

Description

 

When submit a multiple statements sql script through bin/spark-sql, if one of the prior command fail, the processing loop will not exit and continuing executing the following statements, and finally makes the whole program success. 

 

for (oneCmd <- commands) {
  if (StringUtils.endsWith(oneCmd, "\\")) {
    command += StringUtils.chop(oneCmd) + ";"
  } else {
    command += oneCmd
    if (!StringUtils.isBlank(command)) {
      val ret = processCmd(command)
      command = ""
      lastRet = ret
      val ignoreErrors = HiveConf.getBoolVar(conf, HiveConf.ConfVars.CLIIGNOREERRORS)
      if (ret != 0 && !ignoreErrors) {
        CommandProcessorFactory.clean(conf.asInstanceOf[HiveConf])
        ret // for loop will not return if one of the commands fail
      }
    }
  }

 

Issue Links

• links to: [Github] Pull Request #30263 (artiship)