Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28199

Move Trigger implementations to Triggers.scala and avoid exposing these to the end users

    XMLWordPrintableJSON

    Details

    • Docs Text:
      Hide
      In Spark 3.0, the deprecated class org.apache.spark.sql.streaming.ProcessingTime has been removed. Use org.apache.spark.sql.streaming.Trigger.ProcessingTime() instead. Likewise, org.apache.spark.sql.execution.streaming.continuous.ContinuousTrigger has been removed in favor of Trigger.Continuous(), and org.apache.spark.sql.execution.streaming.OneTimeTrigger has been hidden in favor of Trigger.Once().
      Show
      In Spark 3.0, the deprecated class org.apache.spark.sql.streaming.ProcessingTime has been removed. Use org.apache.spark.sql.streaming.Trigger.ProcessingTime() instead. Likewise, org.apache.spark.sql.execution.streaming.continuous.ContinuousTrigger has been removed in favor of Trigger.Continuous(), and org.apache.spark.sql.execution.streaming.OneTimeTrigger has been hidden in favor of Trigger.Once().

      Description

      EvenĀ ProcessingTime is deprecated in 2.2.0, it's being used in Spark codebase, and actually the alternative Spark proposes use deprecated methods which feels like circular - never be able to remove usage.

      In fact, ProcessingTime is deprecated because we want to only expose Trigger.xxx instead of exposing actual implementations, and I think we miss some other implementations as well.

      This issue targets to move all Trigger implementations to Triggers.scala, and hide from end users.

        Attachments

          Activity

            People

            • Assignee:
              kabhwan Jungtaek Lim
              Reporter:
              kabhwan Jungtaek Lim
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: