Uploaded image for project: 'Commons Math'
  1. Commons Math
  2. MATH-1070

Incorrect rounding of float

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 3.2
    • 3.3
    • None
    • None
    • Windows 7, IntelliJ IDEA 10.5.4.

    Description

      package org.apache.commons.math3.util
      example of usage of round functions of Precision class:

      Precision.round(0.0f, 2, BigDecimal.ROUND_UP) = 0.01
      Precision.round((float)0.0, 2, BigDecimal.ROUND_UP) = 0.01
      Precision.round((float) 0.0, 2) = 0.0
      Precision.round(0.0, 2, BigDecimal.ROUND_UP) = 0.0

      Seems the reason is usage of extending float to double inside round functions and getting influence of memory trash as value.

      I think, same problem will be found at usage of other round modes.

      Attachments

        Activity

          People

            Unassigned Unassigned
            lvovlink Oleksandr Muliarevych
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: