Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6663

ArrayIndexOutOfBoundsException in CaptureChangeMySQL

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Trivial
    • Resolution: Unresolved
    • Affects Version/s: 1.9.2
    • Fix Version/s: None
    • Labels:
      None

      Description

      • Current implementaion of CaptureChangeMySQL assumesĀ binlog_row_image=FULL
        https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-cdc-mysql-nar/1.9.2/org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQL/
      • But CDC Processor throws ArrayIndexOutOfBoundsException when binlog_row_image is set to Minimal at
        https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors/src/main/java/org/apache/nifi/cdc/mysql/event/io/UpdateRowsWriter.java#L87
        2019-06-11 18:01:09,465 ERROR [Timer-Driven Process Thread-8] o.a.n.c.m.processors.CaptureChangeMySQL CaptureChangeMySQL[id=016b1000-dfcd-1189-97d7-6bebf706ec18] CaptureChangeMySQL[id=016b1000-dfcd-1189-97d7-
        6bebf706ec18] failed to process session due to java.lang.ArrayIndexOutOfBoundsException: 5; Processor Administratively Yielded for 1 sec: java.lang.ArrayIndexOutOfBoundsException: 5
        java.lang.ArrayIndexOutOfBoundsException: 5
                at org.apache.nifi.cdc.mysql.event.io.UpdateRowsWriter.writeRow(UpdateRowsWriter.java:87)
                at org.apache.nifi.cdc.mysql.event.io.UpdateRowsWriter.lambda$writeEvent$0(UpdateRowsWriter.java:57)
                at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2627)
                at org.apache.nifi.cdc.mysql.event.io.UpdateRowsWriter.writeEvent(UpdateRowsWriter.java:51)
                at org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQL.outputEvents(CaptureChangeMySQL.java:973)
                at org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQL.onTrigger(CaptureChangeMySQL.java:619)
                at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
                at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
                at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
                at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748) 
      • Since not everyone who uses this processor is aware of underlying config & details, I was thinking of following fixes
        1. Include this in CaptureChangeMySQL processor description
        2. try to throw a meaningful error message
        3. Fix current implementation to support other binlog_row_image types

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rahulbpatil Rahul Patil
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: