[NUMBERS-185] Precision.compareTo with ulps cannot be used for sorting

• Type: Bug
• Status: Closed
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 1.0
• Fix Version/s: 1.1
• Component/s: core
• Assignee: Unassigned
• Reporter: Alex Herbert
• Created: 16/Feb/22 12:41
• Updated: 01/Nov/22 11:00
• Resolved: 16/Feb/22 13:47

Description

The Precision.compareTo class was fixed in NUMBERS-154 to allow correct handling of NaN values. This fixed was applied to the compareTo with a delta but not compareTo using a max ULP argument.

This can be fixed by using Double.compare (as is done in compareTo(double, double, double):

    public static int compareTo(final double x, final double y, final int maxUlps) {
        if (equals(x, y, maxUlps)) {
            return 0;
        } else if (x < y) {
            return -1;
        } else if (x > y) {
            return 1;
        }
        // NaN input.
        return Double.compare(x, y);
    } 

 

Issue Links

• is related to: NUMBERS-154 Consistent sorting using "Precision" class