Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20376

Make StateStoreProvider plugable

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.3.0
    • Component/s: Structured Streaming
    • Labels:
      None
    • Target Version/s:

      Description

      Currently only option available for StateStore and StateStoreProvider is HDFSBackedStateStoreProvider. There is no way to provide your own custom implementation of StateStoreProvider for users of structured streaming. We have built a performant in memory StateStore which we would like to plugin for our use cases.

        Attachments

          Activity

            People

            • Assignee:
              tdas Tathagata Das
              Reporter:
              ymahajan Yogesh Mahajan
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: