[FLINK-19633] Fix ArrayIndexOutOfBoundsException in RetractableTopNFunction

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Table SQL / Planner
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: dalongliu
• Created: 14/Oct/20 06:00
• Updated: 10/Dec/20 14:31
• Resolved: 10/Dec/20 14:31

Description

This is reported in Dingtalk group by user. Like FLINK-17625

We should check list is not empty before get the element in RetractableTopNFunction#processElementWithoutRowNumber.

java.lang.ArrayIndexOutOfBoundsException: -1
	at java.util.ArrayList.elementData(ArrayList.java:422) ~[?:1.8.0_232]
	at java.util.ArrayList.get(ArrayList.java:435) ~[?:1.8.0_232]
	at org.apache.flink.table.runtime.operators.rank.RetractableTopNFunction.retractRecordWithoutRowNumber(RetractableTopNFunction.java:410) ~[flink-table-blink_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.table.runtime.operators.rank.RetractableTopNFunction.processElement(RetractableTopNFunction.java:165) ~[flink-table-blink_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.table.runtime.operators.rank.RetractableTopNFunction.processElement(RetractableTopNFunction.java:57) ~[flink-table-blink_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:85) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:161) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.processElement(StreamTaskNetworkInput.java:178) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:153) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:67) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:345) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxStep(MailboxProcessor.java:191) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:181) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:558) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:530) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_232]
.text-only,.text-card-text{white-space: pre;}.rich-text-paragraph{min-height: 15px;}2020-10-14 11:52:03,624 INFO  org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - Calculating tasks to restart to recover the failed task 90bea66de1c231edf33913ecd54406c1_1.
2020-10-14 11:52:03,625 INFO  org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - 9 tasks should be restarted to recover the failed task 90bea66de1c231edf33913ecd54406c1_1. 
2020-10-14 11:52:03,626 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - Job default: select t2.*
        from 
        (
          select user_id
                ,level_id
                ,unit_id
                ,correct_status,utime
                ,row_number() over(partition by user_id,level_id,unit_id order by utime desc) as rn
            from flink_test.binlog_homework_homework_math
        )t2
        where t2.rn=1 (c44be2cec7a1da1d9007df32aed97be3) switched from state RUNNING to FAILING.
org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
	at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:116) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:78) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:192) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:185) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:179) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:503) ~[flink-dist_2.11-1.11.1.jar:1.11.1]
	at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:386) ~[flink-dist_2.11-1.11.1.jar:1.11.1]

Issue Links

• is fixed by: FLINK-20271 Fix ArrayIndexOutOfBounds for RetractableTopNFunction
• links to: GitHub Pull Request #13840