Details

    • Type: Umbrella
    • Status: Resolved
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: Spark Core
    • Labels:
    • Target Version/s:

      Description

      This is an umbrella ticket to track things we are deprecating and removing in Spark 2.0.

        Issue Links

        1.
        Remove support for Hadoop < 2.2 (i.e. Hadoop 1 and 2.0) Sub-task Resolved Reynold Xin
         
        2.
        Remove Bagel Sub-task Resolved Reynold Xin
         
        3.
        Remove Bagel Sub-task Resolved Reynold Xin
         
        4.
        Remove Guava's "Optional" from public API Sub-task Resolved Sean Owen
         
        5.
        SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T] Sub-task Resolved Reynold Xin
         
        6.
        Remove Mesos fine-grained mode subject to discussions Sub-task Resolved Michael Gummelt
         
        7.
        Remove YARN Client / ClientArguments Sub-task Resolved Saisai Shao
         
        8.
        Remove history serving functionality from standalone Master Sub-task Resolved Bryan Cutler
         
        9.
        Remove closure serializer Sub-task Resolved Sean Owen
         
        10.
        Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator Sub-task Resolved Sean Owen
         
        11.
        Remove usage of Hadoop deprecated APIs and reflection that supported 1.x Sub-task Resolved Sean Owen
         
        12.
        Rename "dynamic allocation" to "elastic scaling" Sub-task Closed Andrew Or
         
        13.
        Remove JobLogger Sub-task Resolved Reynold Xin
         
        14.
        Remove HTTPBroadcast Sub-task Resolved Reynold Xin
         
        15.
        Remove TTL-based metadata cleaning (spark.cleaner.ttl) Sub-task Resolved Josh Rosen
         
        16.
        Confusing behavior of fold function of RDD in pyspark Sub-task Resolved Sean Owen
         
        17.
        Remove deprecated methods in SQL / DataFrames Sub-task Resolved Reynold Xin
         
        18.
        Java count(AprroxDistinct)ByKey methods return Scala Long not Java Sub-task Resolved Sean Owen
         
        19.
        Remove some deprecated APIs in RDD/SparkContext Sub-task Resolved Reynold Xin
         
        20.
        Remove SparkContext preferred locations constructor Sub-task Resolved Reynold Xin
         
        21.
        Clean up build warnings: 2.0.0 edition Sub-task Resolved Sean Owen
         
        22.
        Remove unused code related to Hadoop 0.23 Sub-task Resolved Kousuke Saruta
         
        23.
        Remove Vector, VectorSuite and GraphKryoRegistrator which are deprecated and no longer used Sub-task Resolved Kousuke Saruta
         
        24.
        Remove block manager's internal "external block store" API Sub-task Resolved Reynold Xin
         
        25.
        Remove submit python/R scripts through pyspark/sparkR Sub-task Resolved Jeff Zhang
         
        26.
        Remove HistoryServer old multiple files format Sub-task Resolved Felix Cheung
         
        27.
        Deprecate TaskContext.isRunningLocally() Sub-task Resolved Josh Rosen
         
        28.
        Change default build to Hadoop 2.7 Sub-task Resolved Unassigned
         
        29.
        Follow up SPARK-12707, Update documentation and other related code Sub-task Resolved Unassigned
         
        30.
        Spark on Yarn remove deprecated configs for 2.0 Sub-task Closed Boyang Jerry Peng
         
        31.
        Remove usage of ADD_FILES in pyspark Sub-task Resolved Jeff Zhang
         
        32.
        Remove deprecate APIs from GraphX Sub-task Resolved Takeshi Yamamuro
         
        33.
        Deprecate "yarn-client" and "yarn-cluster" Sub-task Resolved Saisai Shao
         
        34.
        Logging: Make it either stable or private[spark] Sub-task Resolved Sean Owen
         
        35.
        Move UserDefinedFunction into sql.expressions package Sub-task Resolved Reynold Xin
         
        36.
        Upgrade to Jersey 2 Sub-task Resolved Matt Cheah
         
        37.
        Remove SparkContext.metricsSystem Sub-task Resolved Reynold Xin
         
        38.
        Remove the support of SIMR cluster manager Sub-task Resolved Saisai Shao
         
        39.
        Remove reference to Tachyon in cluster & release script Sub-task Resolved Reynold Xin
         
        40.
        Move network/* modules into common/network-* Sub-task Resolved Reynold Xin
         
        41.
        Move tags and unsafe modules into common Sub-task Resolved Reynold Xin
         
        42.
        Move docker, extras modules into external Sub-task Resolved Sean Owen
         
        43.
        Move misc top-level build files into appropriate subdirs Sub-task Resolved Sean Owen
         
        44.
        remove typed operations (map, flatMap, mapPartitions) from Python DataFrame Sub-task Resolved Wenchen Fan
         
        45.
        Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages Sub-task Resolved Shixiong Zhu
         
        46.
        Move docs of streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages Sub-task Resolved Shixiong Zhu
         
        47.
        Consider removing Kafka modules from Spark / Spark Streaming Sub-task Closed Unassigned
         
        48.
        Remove Streaming Akka docs from Spark Sub-task Resolved Shixiong Zhu
         
        49.
        Move streaming-flume back to Spark Sub-task Resolved Shixiong Zhu
         
        50.
        Remove straggler references to Tachyon Sub-task Resolved Liwei Lin(Inactive)
         
        51.
        Require Java 8 for Spark 2.x Sub-task Closed Unassigned
         
        52.
        Change SparkListener from a trait to an abstract class, and remove JavaSparkListener Sub-task Resolved Reynold Xin
         
        53.
        Remove HashShuffleManager Sub-task Resolved Reynold Xin
         
        54.
        Put examples packaging on a diet Sub-task Resolved Marcelo Vanzin
         
        55.
        Java sampleByKey methods take ju.Map but with Scala Double values; results in type Object Sub-task Resolved Sean Owen
         
        56.
        Remove support for "zk://" master URL Sub-task Resolved Sean Owen
         
        57.
        Add a deprecation warning for Python 2.6 Sub-task Resolved Unassigned
         

          Activity

          Hide
          rxin Reynold Xin added a comment -

          No not yet. I don't know how many we want to deprecate though.

          Show
          rxin Reynold Xin added a comment - No not yet. I don't know how many we want to deprecate though.
          Hide
          tgraves Thomas Graves added a comment -

          I added a task to remove the deprecated yarn configs, especially the old env variables.

          Do we have anything around this for core in general?

          Show
          tgraves Thomas Graves added a comment - I added a task to remove the deprecated yarn configs, especially the old env variables. Do we have anything around this for core in general?

            People

            • Assignee:
              rxin Reynold Xin
              Reporter:
              rxin Reynold Xin
            • Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development