[PHOENIX-7231] Delete from table for transformed table is failing

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 5.3.0
• Assignee: Sanjeet Malhotra
• Reporter: Sanjeet Malhotra
• Created: 23/Feb/24 07:06
• Updated: 16/Apr/24 05:53

Description

Steps to reproduce:

1. Create a table.
2. Upsert one row into it.
3. Change column encoding scheme to transform table.
4. Run transform tool and make sure cutover is successful.
5. Upsert one more row to original Phoenix table which now points to new physical table.
6. Run delete from <table name>

Following error is thrown:

org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed to build index for unexpected reason!
        at org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:208)
        at org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:467)
        at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:997)
        at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$28.call(RegionCoprocessorHost.java:994)
        at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558)
        at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631)
        at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:994)
        at org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.prepareMiniBatchOperations(HRegion.java:3790)
        at org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4508)
        at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4446)
        at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4368)
        at org.apache.hadoop.hbase.regionserver.HRegion.lambda$batchMutate$10(HRegion.java:4381)
        at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
        at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4380)
        at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4376)
        at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:277)
        at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatchWithRetries(UngroupedAggregateRegionObserver.java:240)
        at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commit(UngroupedAggregateRegionObserver.java:504)
        at org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.annotateAndCommit(UngroupedAggregateRegionScanner.java:691)
        at org.apache.phoenix.coprocessor.UngroupedAggregateRegionScanner.next(UngroupedAggregateRegionScanner.java:642)
        at org.apache.phoenix.coprocessor.BaseRegionScanner.nextRaw(BaseRegionScanner.java:56)
        at org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)
        at org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:79)
        at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:254)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3389)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3655)
        at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:415)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
        at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
        at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82)
Caused by: java.lang.NullPointerException
        at org.apache.phoenix.index.IndexMaintainer.buildRowDeleteMutation(IndexMaintainer.java:1425)
        at org.apache.phoenix.hbase.index.IndexRegionObserver.prepareIndexMutations(IndexRegionObserver.java:934)
        at org.apache.phoenix.hbase.index.IndexRegionObserver.preparePreIndexMutations(IndexRegionObserver.java:963)
        at org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutateWithExceptions(IndexRegionObserver.java:1181)
        at org.apache.phoenix.hbase.index.IndexRegionObserver.preBatchMutate(IndexRegionObserver.java:464)
        ... 29 more

Debugging analysis:

Seems like its related to instance variable inheritance and visibility scope of instance variables in TransformMaintainer class. As per error, its seems like TransformMaintainer#buildRowDeleteMutation was called. buildRowDeleteMutation is defined in parent class: IndexMaintainer and is trying to access to access private instance variable: emptyKeyValueCFPtr. As the instance variable is declared in both the parent and child classes and is private in both so, IndexMaintainer.emptyKeyValueCFPtr gets used which is null while TransformMaintainer.emptyKeyValueCFPtr is not non-null.

 

This Jira aims to fix following:

• Fix delete from table for transformed tables.
• Fix TransformTool related tests which are currently broken and not running on master pipeline as Category annotations are missing.
• Handle physical and logical table name mismatch in general like for table level max lookback age.