Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16451

Race condition between HiveStatement.getQueryLog and HiveStatement.runAsyncOnServer

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: Beeline, JDBC
    • Labels:
      None

      Description

      During the BeeLineDriver testing I have met the following race condition:

      • Run the query asynchronously through BeeLine
      • Querying the logs in the BeeLine

      In the following code:

      HiveStatement.runAsyncOnServer
        private void runAsyncOnServer(String sql) throws SQLException {
          checkConnection("execute");
      
          closeClientOperation();
          initFlags();
      [..]
        }
      
      HiveStatement.getQueryLog
        public List<String> getQueryLog(boolean incremental, int fetchSize)
            throws SQLException, ClosedOrCancelledStatementException {
      [..]
          try {
            if (stmtHandle != null) {
      [..]
            } else {
              if (isQueryClosed) {
                throw new ClosedOrCancelledStatementException("Method getQueryLog() failed. The " +
                    "statement has been closed or cancelled.");
              } else {
                return logs;
              }
            }
          } catch (SQLException e) {
      [..]
          }
      [..]
        }
      

      The runAsyncOnServer closeClientOperation sets isQueryClosed flag to true:

      HiveStatement.closeClientOperation
        void closeClientOperation() throws SQLException {
      [..]
          isQueryClosed = true;
          isExecuteStatementFailed = false;
          stmtHandle = null;
        }
      

      The initFlags sets it to false:

        private void initFlags() {
          isCancelled = false;
          isQueryClosed = false;
          isLogBeingGenerated = true;
          isExecuteStatementFailed = false;
          isOperationComplete = false;
        }
      

      If the getQueryLog is called after the closeClientOperation, but before the initFlags, then we will have a following warning if verbose mode is set to true in BeeLine:

      Warning: org.apache.hive.jdbc.ClosedOrCancelledStatementException: Method getQueryLog() failed. The statement has been closed or cancelled. (state=,code=0)
      

      This caused this fail:
      https://builds.apache.org/job/PreCommit-HIVE-Build/4691/testReport/org.apache.hadoop.hive.cli/TestBeeLineDriver/testCliDriver_smb_mapjoin_11_/

      Error Message
      
      Client result comparison failed with error code = 1 while executing fname=smb_mapjoin_11
      16a17
      > Warning: org.apache.hive.jdbc.ClosedOrCancelledStatementException: Method getQueryLog() failed. The statement has been closed or cancelled. (state=,code=0)
      

        Attachments

        1. HIVE-16451.02.patch
          2 kB
          Peter Vary
        2. HIVE-16451.03.patch
          2 kB
          Peter Vary
        3. HIVE-16451.patch
          2 kB
          Peter Vary

          Activity

            People

            • Assignee:
              pvary Peter Vary
              Reporter:
              pvary Peter Vary
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: