Uploaded image for project: 'Apache Sedona'
  1. Apache Sedona
  2. SEDONA-434

Improve reliability by resolve the nondeterministic of the order of the Map

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 1.5.1

    Description

        1. Problem :
           
          when the program reading from the newRdd, it assumes the order of the data to read into the Map https://github.com/apache/sedona/blob/a64972027aa4c0fea354898da67a2dd4e6cde46b/spark/common/src/test/java/org/apache/sedona/core/formatMapper/GeoJsonIOTest.java#L106
           
          However, when it write the data, it did not assume the order according to the [Oracle's official document](https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html#:~:text=This%20class%20makes%20no%20guarantees%20as%20to%20the%20order%20of%20the%20map)
           
           
          https://github.com/apache/sedona/blob/a64972027aa4c0fea354898da67a2dd4e6cde46b/spark/common/src/main/java/org/apache/sedona/core/spatialRDD/SpatialRDD.java#L571-L572
           
          So it will cause the ERROR when the environment has been changed
           
          ```
          org.junit.ComparisonFailure: expected:<0[1]> but was:<0[23]>
           
          at org.junit.Assert.assertEquals(Assert.java:117)
          at org.junit.Assert.assertEquals(Assert.java:146)
          at org.apache.sedona.core.formatMapper.GeoJsonIOTest.testReadWriteGeoJson(GeoJsonIOTest.java:115)
          ```
           
      1. Solution
         
        So we can change the Map to use the LinkedHashMap to guarantee the order to solve the problem 
         

      Here is the proposed change : https://github.com/MyEnthusiastic/sedona/pull/1

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              hannahchi huiyang chi
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m