[HBASE-16132] Scan does not return all the result when regionserver is busy

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0, 1.1.6, 1.2.3, 2.0.0
• Component/s: Client
• Hadoop Flags: Reviewed
• Assignee: Lijin Bin
• Reporter: Lijin Bin
• Created: 28/Jun/16 00:54
• Updated: 12/Jul/16 19:21
• Resolved: 04/Jul/16 13:50

Description

We have find some corner case, when regionserver is busy and last a long time. Some scanner may return null even if they do not scan all data.
We find in ScannerCallableWithReplicas there is a case do not handler correct, when cs.poll timeout and do not return any result , it is will return a null result, so scan get null result, and end the scan.

    try {
      Future<Pair<Result[], ScannerCallable>> f = cs.poll(timeout, TimeUnit.MILLISECONDS);
      if (f != null) {
        Pair<Result[], ScannerCallable> r = f.get(timeout, TimeUnit.MILLISECONDS);
        if (r != null && r.getSecond() != null) {
          updateCurrentlyServingReplica(r.getSecond(), r.getFirst(), done, pool);
        }
        return r == null ? null : r.getFirst(); // great we got an answer
      }
    } catch (ExecutionException e) {
      RpcRetryingCallerWithReadReplicas.throwEnrichedException(e, retries);
    } catch (CancellationException e) {
      throw new InterruptedIOException(e.getMessage());
    } catch (InterruptedException e) {
      throw new InterruptedIOException(e.getMessage());
    } catch (TimeoutException e) {
      throw new InterruptedIOException(e.getMessage());
    } finally {
      // We get there because we were interrupted or because one or more of the
      // calls succeeded or failed. In all case, we stop all our tasks.
      cs.cancelAll();
    }
    return null; // unreachable
 

Issue Links

• relates to: HBASE-16172 Unify the retry logic in ScannerCallableWithReplicas and RpcRetryingCallerWithReadReplicas