Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31990

Streaming's state store compatibility is broken

    XMLWordPrintableJSON

Details

    Description

      This line of https://github.com/apache/spark/pull/28062 changed the order of groupCols in dropDuplicates(). Thus, the executor JVM could probably crash, throw a random exception or even return a wrong answer when using the checkpoint written by the previous version. 

      Attachments

        Issue Links

          Activity

            People

              maropu Takeshi Yamamuro
              smilegator Xiao Li
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: