[HBASE-13662] RSRpcService.scan() throws an OutOfOrderScannerNext if the scan has a retriable failure

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.1, 1.1.0, 0.98.10.1, 2.0.0
• Fix Version/s: 0.98.13, 1.0.2, 1.2.0, 1.1.1, 2.0.0
• Component/s: IPC/RPC, regionserver
• Assignee: Matteo Bertozzi
• Reporter: Matteo Bertozzi
• Created: 11/May/15 18:00
• Updated: 10/Sep/16 00:31
• Resolved: 12/May/15 16:48

Description

while fixing HBASE-13651 I noticed that if we have a failure inside the RSRpcService.scan(), when the request has a hasNextCallSeq() the nextCallSeq is incremented and not rolledback, which means that the client retry will send a request with a nextCallSeq not up to date, which result in an OutOfOrderScannerNextException.

if (rows > 0) {
  if (request.hasNextCallSeq()) {
    if (request.getNextCallSeq() != rsh.nextCallSeq) {
      throw new OutOfOrderScannerNextException(...)
    }
    // Increment the nextCallSeq value which is the next expected from client.
    rsh.nextCallSeq++;
  }
}
try {
  ...scan code...
}

after the scanner heartbeat patches HBASE-13090, we seems to be able to recover from that OutOfOrder exception, but the error show up anyway.

After a discussion with [e-mail] we ended up saying that decrementing the callSeq on exception seems to be fine. but we had the open question about having that nextCallSeq to be atomic, if that was supposed to prevent concurrent requests with the same id. any thoughts?

Issue Links

• relates to: HBASE-16604 Scanner retries on IOException can cause the scans to miss data