Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25829 remove duplicated map keys with last wins policy
  3. SPARK-25831

should apply "earlier entry wins" in hive map value converter

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • 2.4.0
    • None
    • SQL
    • None

    Description

      scala> sql("CREATE TABLE t AS SELECT map(1,2,1,3) as map")
      res11: org.apache.spark.sql.DataFrame = []
      
      scala> sql("select * from t").show
      +--------+
      |     map|
      +--------+
      |[1 -> 3]|
      +--------+
      

      We mistakenly apply "later entry wins"

      Attachments

        Activity

          People

            Unassigned Unassigned
            cloud_fan Wenchen Fan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: