Affects Version/s: None
Fix Version/s: 0.90.0
The flow during a HRegionServer.put() seems to be the following. [For now, let's just consider single row Put containing edits to multiple column families/columns.
HRegionServer.put() does a:
syncWal() (the HDFS sync call). /* this is assuming we have
HRegion.put() does a:
for each column family
HLog.append() does a :
foreach edit in a single column family
doWrite() does a:
There seems to be two related issues here that could result in inconsistencies.
Issue #1: A put() does a bunch of HLog.append() calls. These in turn do a bunch of "write" calls on the underlying DFS stream. If we crash after having written out some append's to DFS, recovery will run and apply a partial transaction to memstore.
Issue #2: The updates to memstore should happen after the sync rather than before. Otherwise, there is the danger that the write to DFS (sync) fails for some reason & we return an error to the client, but we have already taken edits to the memstore. So subsequent reads will serve uncommitted data.