Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-32485

RecordBinaryComparatorSuite test failures on big-endian systems

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0
    • 3.1.0
    • Tests

    Description

      The fix for SPARK-29918 broke two tests on big-endian systems:

      • testBinaryComparatorWhenSubtractionIsDivisibleByMaxIntValue
      • testBinaryComparatorWhenSubtractionCanOverflowLongValue

      These tests date from a time where subtraction was being used to do multi-byte comparisons. They try to trigger old bugs by feeding specific values into the comparison. However the fix for SPARK-29918 modified the order in which bytes are compared when comparing 8 bytes at a time on little-endian systems (to match the normal byte-by-byte comparison). This fix did not affect big-endian systems. However the expected output of the tests was modified for all systems regardless of endianness. So the tests broke on big-endian systems.

      It is also not clear that the values compared in the tests match the original intent of the tests now that the bytes in those values are compared in order (equivalent to the bytes in the values being reversed).

      Attachments

        Activity

          People

            mundaym Michael Munday
            mundaym Michael Munday
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: