[KUDU-1888] Java: if batch times out before being sent, wrong deferred is returned

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.0
• Component/s: java
• Target Version/s: 1.3.0
• Code Review: https://gerrit.cloudera.org/#/c/6051/
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 17/Feb/17 05:57
• Updated: 19/Feb/17 03:20
• Resolved: 19/Feb/17 03:20

Description

I spent some time looking into a test failure of TestAsyncKuduSession.testInsertIntoUnavailableTablet(). The test fails if the machine is under load because of what looks something likke the following:

• the DeadlineTracker for a Batch is created with timeout 1
• the process loses the CPU for a few milliseconds
  We get to the following:

          addBatchCallbacks(batch);
          batchResponses.add(client.sendRpcToTablet(batch));
  

Typically sendRpcToTablet() returns request.getDeferred(). But in the case that it has already timed out, it returns an error deferred immediately, without firing the request's errback. This means that the deferred in batchResponses contains an error, because the batch callbacks added by `addBatchCallbacks` don't actually get triggered.