Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-15549

Integer overflow in SpillingResettableMutableObjectIterator

    XMLWordPrintableJSON

    Details

      Description

      The SpillingResettableMutableObjectIterator has a data overflow problem if the number of elements in a single input exceeds Integer.MAX_VALUE.

      The reason is inside the SpillingResettableMutableObjectIterator, it track the total number of elements and the number of elements currently read with two int type fileds (elementCount and currentElementNum), and if the number of elements exceeds Integer.MAX_VALUE, it will overflow.

      If there is an overflow, then in the next iteration, after reset the input , the data will not be read or only part of the data will be read.

      Therefore, we should changing the type of these two fields of SpillingResettableIterator* from int to long, and we also need a pre-check mechanism before such numerical.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                caojian0613 caojian0613
                Reporter:
                caojian0613 caojian0613
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m