Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35659

Avoid write null to StateStore

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.2, 3.1.2, 3.2.0
    • Fix Version/s: 3.0.3, 3.2.0, 3.1.3
    • Component/s: Structured Streaming
    • Labels:
      None

      Description

      According to get method doc in StateStore API, it returns non-null row if the key exists. So basically we should avoid write null to StateStore. You cannot distinguish if the returned null row is because the key doesn't exist, or the value is actually null. And due to the defined behavior of get, it is quite easy to cause NPE error if the caller doesn't expect to get a null if the caller believes the key exists.

        Attachments

          Activity

            People

            • Assignee:
              viirya L. C. Hsieh
              Reporter:
              viirya L. C. Hsieh
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: