Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28199

Move Trigger implementations to Triggers.scala and avoid exposing these to the end users

    XMLWordPrintableJSON

Details

    • Hide
      In Spark 3.0, the deprecated class org.apache.spark.sql.streaming.ProcessingTime has been removed. Use org.apache.spark.sql.streaming.Trigger.ProcessingTime() instead. Likewise, org.apache.spark.sql.execution.streaming.continuous.ContinuousTrigger has been removed in favor of Trigger.Continuous(), and org.apache.spark.sql.execution.streaming.OneTimeTrigger has been hidden in favor of Trigger.Once().
      Show
      In Spark 3.0, the deprecated class org.apache.spark.sql.streaming.ProcessingTime has been removed. Use org.apache.spark.sql.streaming.Trigger.ProcessingTime() instead. Likewise, org.apache.spark.sql.execution.streaming.continuous.ContinuousTrigger has been removed in favor of Trigger.Continuous(), and org.apache.spark.sql.execution.streaming.OneTimeTrigger has been hidden in favor of Trigger.Once().

    Description

      EvenĀ ProcessingTime is deprecated in 2.2.0, it's being used in Spark codebase, and actually the alternative Spark proposes use deprecated methods which feels like circular - never be able to remove usage.

      In fact, ProcessingTime is deprecated because we want to only expose Trigger.xxx instead of exposing actual implementations, and I think we miss some other implementations as well.

      This issue targets to move all Trigger implementations to Triggers.scala, and hide from end users.

      Attachments

        Activity

          People

            kabhwan Jungtaek Lim
            kabhwan Jungtaek Lim
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: