Uploaded image for project: 'Commons Math'
  1. Commons Math
  2. MATH-238

MathUtils.gcd(u, v) fails when u and v both contain a high power of 2

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0
    • Fix Version/s: None
    • Labels:
      None

      Description

      The test at the beginning of MathUtils.gcd(u, v) for arguments equal to zero fails when u and v contain high enough powers of 2 so that their product overflows to zero.

      assertEquals(3 * (1<<15), MathUtils.gcd(3 * (1<<20), 9 * (1<<15)));

      Fix: Replace the test at the start of MathUtils.gcd()

      if (u * v == 0) {

      by

      if (u == 0 || v == 0) {

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              chsemrau Christian Semrau
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: