Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
0.6.4
-
None
Description
While we were testing Tephra with CDH 5.5.1, we noticed that cells deleted transactionally were becoming visible to clients after a few hours. We traced the cause for this issue to be the StoreScanner.optimize() method added in HBASE-13109 to change SEEKs to SKIPs in certain cases.
Tehpra maintains multiple versions of a cell to account for in-progress transactions. Transaction co-processors filter out in-progress and deleted cells during scans and other read operations. To do this the co-processor sets max versions to INT_MAX for read operations. While going through all versions of a cell, when the co-processor sees that a particular version satisfies transaction visibility, it returns NEXT_COL (for deleted cells) or INCLUDE_AND_NEXT_COL (for non-deleted cells).
However the optimize method is changing the returned value to a SKIP instead, based on next index key (We're completely not sure of the optimize logic yet). This is causing the delete marker to be skipped, and then an older version of the same cell to be passed to the transaction co-processor. Since the older version also passes the visibility check for the transaction, the older version gets a INCLUDE_AND_NEXT_COL. Thus deletes are becoming visible again.
We should safeguard against such issues in the transaction co-processor by remembering the last skipped column, and then returning SKIP to all further versions of the same cell. But this will mean that we would process more versions of a cell than necessary, and is not optimal. But, it is good to have this safeguard nevertheless.
We'll follow up with an HBase ticket.
Attachments
Issue Links
- relates to
-
TEPHRA-170 Use SEEK_NEXT_USING_HINT vs NEXT_COL in Transaction Visibility filter
-
- Open
-
- links to