[HBASE-17529] MergeTableRegionsProcedure failed due to ArrayIndexOutOfBoundsException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Cannot Reproduce
• Labels: rpc
• Assignee: Unassigned
• Reporter: Ted Yu
• Created: 25/Jan/17 03:31
• Updated: 08/Jun/17 18:20
• Resolved: 08/Jun/17 18:20

Description

I built tar ball using master branch based on commit 616f4801b06a8427a03ceca9fb8345700ce1ad71.

Was running the following command:

hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList -DinMemoryCompaction=BASIC Loop 4 6 1000000 /tmp/hbase-biglinkedlist-verify 6 --monkey slowDeterministic

Here was related snippet:

2017-01-24 21:29:00,107 DEBUG [RpcServer.deafult.FPBQ.Fifo.handler=0,queue=0,port=16000] procedure2.ProcedureExecutor: Stored MergeTableRegionsProcedure (table=IntegrationTestBigLinkedList regions=[IntegrationTestBigLinkedList,,1485292220242.4c5ea240e86ef22ec7264b1153dd557d., IntegrationTestBigLinkedList,\x0E8\xE3\x8E8\xE3\x8E8,1485292220242.6cdb98dfed41ea689b3cd66478c2c580. ] forcible=false), procId=12, owner=hbase, state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE
2017-01-24 21:29:00,108 DEBUG [ProcedureExecutorWorker-14] wal.WALProcedureStore: Set running procedure count=1, slots=24
2017-01-24 21:29:00,127 ERROR [ProcedureExecutorWorker-14] procedure2.ProcedureExecutor: CODE-BUG: Uncatched runtime exception for procedure: MergeTableRegionsProcedure (table=IntegrationTestBigLinkedList regions=[IntegrationTestBigLinkedList,,1485292220242.4c5ea240e86ef22ec7264b1153dd557d., IntegrationTestBigLinkedList,\x0E8\xE3\x8E8\xE3\x8E8,1485292220242.6cdb98dfed41ea689b3cd66478c2c580. ] forcible=false), procId=12, owner=hbase, state=RUNNABLE:MERGE_TABLE_REGIONS_MOVE_REGION_TO_SAME_RS
java.lang.ArrayIndexOutOfBoundsException
        at org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1024)
        at org.apache.hadoop.hbase.nio.MultiByteBuff.get(MultiByteBuff.java:628)
        at org.apache.hadoop.hbase.ipc.RpcServer$ByteBuffByteInput.read(RpcServer.java:1483)
        at org.apache.hadoop.hbase.shaded.com.google.protobuf.ByteInputByteString.copyToInternal(ByteInputByteString.java:105)
        at org.apache.hadoop.hbase.shaded.com.google.protobuf.ByteString.toByteArray(ByteString.java:651)
        at org.apache.hadoop.hbase.RegionLoad.getName(RegionLoad.java:50)
        at org.apache.hadoop.hbase.ServerLoad.getRegionsLoad(ServerLoad.java:236)
        at org.apache.hadoop.hbase.master.procedure.MergeTableRegionsProcedure.getRegionLoad(MergeTableRegionsProcedure.java:774)
        at org.apache.hadoop.hbase.master.procedure.MergeTableRegionsProcedure.MoveRegionsToSameRS(MergeTableRegionsProcedure.java:461)
        at org.apache.hadoop.hbase.master.procedure.MergeTableRegionsProcedure.executeFromState(MergeTableRegionsProcedure.java:142)
        at org.apache.hadoop.hbase.master.procedure.MergeTableRegionsProcedure.executeFromState(MergeTableRegionsProcedure.java:72)
        at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:154)
        at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:708)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1332)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1133)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:76)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1588)

Master log to be attached.