[PHOENIX-5996] IndexRebuildRegionScanner.prepareIndexMutationsForRebuild may incorrectly delete index row when a delete and put mutation with the same timestamp

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.1.0, 4.16.0
• Fix Version/s: 5.1.0, 4.16.0
• Assignee: chenglei
• Reporter: chenglei
• Created: 09/Jul/20 06:50
• Updated: 14/Jul/20 03:01
• Resolved: 14/Jul/20 02:51

Description

With PHOENIX-5748, IndexRebuildRegionScanner.prepareIndexMutationsForRebuild is responsible for generating index table mutations for rebuild.
In the processing of data table mutations list, there can be a delete and put mutation with the same timestamp. If so, the delete and put are processed together in one iteration. First, the delete mutation is applied on the put mutation and current row state, and then the modified put mutation is processed.
But when the modified put mutation is empty , even the current row state is not empty after the delete mutation is applied, the whole index row is deleted, just as following line 1191 in IndexRebuildRegionScanner.prepareIndexMutationsForRebuild:

1189              } else {
1190                    if (currentDataRowState != null) {
1191                        Mutation del = indexMaintainer.buildRowDeleteMutation(indexRowKeyForCurrentDataRow,
1192                                IndexMaintainer.DeleteType.ALL_VERSIONS, ts);
1193                        indexMutations.add(del);
1194                        // For the next iteration of the for loop
1195                        currentDataRowState = null;
1196                        indexRowKeyForCurrentDataRow = null;
1197                    }
1198              }

I think above logical is wrong, when the current row state is not empty after the delete mutation is applied, we can not delete the whole index row, but instead we should reuse the logical of applying a delete mutation on current row state. I wrote a unit test in PrepareIndexMutationsForRebuildTest to produce the case:

    @Test
    public void testPutDeleteOnSameTimeStampAndPutNullifiedByDelete() throws Exception {
        SetupInfo info = setup(
                TABLE_NAME,
                INDEX_NAME,
                "ROW_KEY VARCHAR, CF1.C1 VARCHAR, CF2.C2 VARCHAR",
                "CF2.C2",
                "ROW_KEY",
                "");

        Put dataPut = new Put(Bytes.toBytes(ROW_KEY));
        addCellToPutMutation(
                dataPut,
                Bytes.toBytes("CF2"),
                Bytes.toBytes("C2"),
                1,
                Bytes.toBytes("v2"));
        addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 1);
        
        addCellToPutMutation(
                dataPut,
                Bytes.toBytes("CF1"),
                Bytes.toBytes("C1"),
                2,
                Bytes.toBytes("v1"));
        addEmptyColumnToDataPutMutation(dataPut, info.pDataTable, 2);

        Delete dataDel = new Delete(Bytes.toBytes(ROW_KEY));
        addCellToDelMutation(
                dataDel,
                Bytes.toBytes("CF1"),
                null,
                2,
                KeyValue.Type.DeleteFamily);

        List<Mutation> actualIndexMutations = IndexRebuildRegionScanner.prepareIndexMutationsForRebuild(
                info.indexMaintainer,
                dataPut,
                dataDel);

        List<Mutation> expectedIndexMutations = new ArrayList<>();
        byte[] idxKeyBytes = generateIndexRowKey("v2");

        
        Put idxPut1 = new Put(idxKeyBytes);
        addEmptyColumnToIndexPutMutation(idxPut1, info.indexMaintainer, 1);
        expectedIndexMutations.add(idxPut1);
        
        Put idxPut2 = new Put(idxKeyBytes);
        addEmptyColumnToIndexPutMutation(idxPut2, info.indexMaintainer, 2);
        expectedIndexMutations.add(idxPut2);

        assertEqualMutationList(expectedIndexMutations, actualIndexMutations);
    }

Issue Links

• links to: Github PR#826