[HIVE-16451] Race condition between HiveStatement.getQueryLog and HiveStatement.runAsyncOnServer

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Beeline, JDBC
• Assignee: Peter Vary
• Reporter: Peter Vary
• Created: 14/Apr/17 14:12
• Updated: 17/Dec/18 07:32
• Resolved: 09/May/17 14:27

Description

During the BeeLineDriver testing I have met the following race condition:

• Run the query asynchronously through BeeLine
• Querying the logs in the BeeLine

In the following code:

HiveStatement.runAsyncOnServer

  private void runAsyncOnServer(String sql) throws SQLException {
    checkConnection("execute");

    closeClientOperation();
    initFlags();
[..]
  }

HiveStatement.getQueryLog

  public List<String> getQueryLog(boolean incremental, int fetchSize)
      throws SQLException, ClosedOrCancelledStatementException {
[..]
    try {
      if (stmtHandle != null) {
[..]
      } else {
        if (isQueryClosed) {
          throw new ClosedOrCancelledStatementException("Method getQueryLog() failed. The " +
              "statement has been closed or cancelled.");
        } else {
          return logs;
        }
      }
    } catch (SQLException e) {
[..]
    }
[..]
  }

The runAsyncOnServer closeClientOperation sets isQueryClosed flag to true:

HiveStatement.closeClientOperation

  void closeClientOperation() throws SQLException {
[..]
    isQueryClosed = true;
    isExecuteStatementFailed = false;
    stmtHandle = null;
  }

The initFlags sets it to false:

  private void initFlags() {
    isCancelled = false;
    isQueryClosed = false;
    isLogBeingGenerated = true;
    isExecuteStatementFailed = false;
    isOperationComplete = false;
  }

If the getQueryLog is called after the closeClientOperation, but before the initFlags, then we will have a following warning if verbose mode is set to true in BeeLine:

Warning: org.apache.hive.jdbc.ClosedOrCancelledStatementException: Method getQueryLog() failed. The statement has been closed or cancelled. (state=,code=0)

This caused this fail:
https://builds.apache.org/job/PreCommit-HIVE-Build/4691/testReport/org.apache.hadoop.hive.cli/TestBeeLineDriver/testCliDriver_smb_mapjoin_11_/

Error Message

Client result comparison failed with error code = 1 while executing fname=smb_mapjoin_11
16a17
> Warning: org.apache.hive.jdbc.ClosedOrCancelledStatementException: Method getQueryLog() failed. The statement has been closed or cancelled. (state=,code=0)