Commons Math
  1. Commons Math
  2. MATH-1068

KendallsCorrelation suffers from integer overflow for large arrays.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.3
    • Fix Version/s: 3.3
    • Labels:

      Description

      For large array size (say, over 5,000), numPairs > 10 million.
      in line 258, (numPairs - tiedXPairs) * (numPairs - tiedYPairs) possibly > 100 billion, which will cause an integer overflow, resulting in a negative number, which will result in the end result in a NaN since the square-root of that number is calculated.
      This can easily be solved by changing line 163 to
      final long numPairs = ((long)n) * (n - 1) / 2; // to avoid overflow

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Gal Lalouche
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 1m
              1m
              Remaining:
              Remaining Estimate - 1m
              1m
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development