Uploaded image for project: 'Commons Math'
  1. Commons Math
  2. MATH-1068

KendallsCorrelation suffers from integer overflow for large arrays.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.3
    • Fix Version/s: 3.3
    • Labels:

      Description

      For large array size (say, over 5,000), numPairs > 10 million.
      in line 258, (numPairs - tiedXPairs) * (numPairs - tiedYPairs) possibly > 100 billion, which will cause an integer overflow, resulting in a negative number, which will result in the end result in a NaN since the square-root of that number is calculated.
      This can easily be solved by changing line 163 to
      final long numPairs = ((long)n) * (n - 1) / 2; // to avoid overflow

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Terran-Ghost Gal Lalouche
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1m
                1m
                Remaining:
                Remaining Estimate - 1m
                1m
                Logged:
                Time Spent - Not Specified
                Not Specified