Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33358

Spark SQL CLI command processing loop can't exit while one comand fail

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 3.0.0, 3.0.1, 3.1.0
    • 3.0.2, 3.1.0
    • SQL

    Description

       

      When submit a multiple statements sql script through bin/spark-sql, if one of the prior command fail, the processing loop will not exit and continuing executing the following statements, and finally makes the whole program success. 

       

      for (oneCmd <- commands) {
        if (StringUtils.endsWith(oneCmd, "\\")) {
          command += StringUtils.chop(oneCmd) + ";"
        } else {
          command += oneCmd
          if (!StringUtils.isBlank(command)) {
            val ret = processCmd(command)
            command = ""
            lastRet = ret
            val ignoreErrors = HiveConf.getBoolVar(conf, HiveConf.ConfVars.CLIIGNOREERRORS)
            if (ret != 0 && !ignoreErrors) {
              CommandProcessorFactory.clean(conf.asInstanceOf[HiveConf])
              ret // for loop will not return if one of the commands fail
            }
          }
        }
      
      

       

      Attachments

        Activity

          People

            artiship Lichuanliang
            artiship Lichuanliang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: