Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22644

Make ML testsuite support StructuredStreaming test

    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.2.0
    • 2.3.0
    • ML
    • None

    Description

      We need to add some helper code to make testing ML transformers & models easier with streaming data. These tests might help us catch any remaining issues and we could encourage future PRs to use these tests to prevent new Models & Transformers from having issues.

      Attachments

        Issue Links

          blocks

          Test - A new unit, integration or system test. SPARK-22881 ML test for StructuredStreaming: spark.ml.regression

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. SPARK-22882 ML test for StructuredStreaming: spark.ml.classification

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. SPARK-22883 ML test for StructuredStreaming: spark.ml.feature, A-M

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. SPARK-22884 ML test for StructuredStreaming: spark.ml.clustering

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. SPARK-22885 ML test for StructuredStreaming: spark.ml.tuning

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. SPARK-22886 ML test for StructuredStreaming: spark.ml.recommendation

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. SPARK-22887 ML test for StructuredStreaming: spark.ml.fpm

          • Major - Major loss of function.
          • Resolved

          Test - A new unit, integration or system test. SPARK-22915 ML test for StructuredStreaming: spark.ml.feature, N-Z

          • Major - Major loss of function.
          • Resolved
          Is contained by

          Umbrella - An overarching type made of sub-tasks SPARK-21926 Compatibility between ML Transformers and Structured Streaming

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-22346 Update VectorAssembler to work with Structured Streaming

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Pull request #19843 [Github] Pull Request #19843 (WeichenXu123)

          Pull request #19979 [Github] Pull Request #19979 (WeichenXu123)

          Activity

            People

              weichenxu123 Weichen Xu
              weichenxu123 Weichen Xu
              Joseph K. Bradley Joseph K. Bradley
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: