Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11806

Spark 2.0 deprecations and removals

    Details

    • Type: Umbrella
    • Status: Resolved
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: Spark Core
    • Labels:
    • Target Version/s:

      Description

      This is an umbrella ticket to track things we are deprecating and removing in Spark 2.0.

        Attachments

          Issue Links

          1.
          Remove support for Hadoop < 2.2 (i.e. Hadoop 1 and 2.0) Sub-task Resolved Reynold Xin
          2.
          Remove Bagel Sub-task Resolved Reynold Xin
          3.
          Remove Bagel Sub-task Resolved Reynold Xin
          4.
          Remove Guava's "Optional" from public API Sub-task Resolved Sean Owen
          5.
          SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T] Sub-task Resolved Reynold Xin
          6.
          Remove Mesos fine-grained mode subject to discussions Sub-task Resolved Michael Gummelt
          7.
          Remove YARN Client / ClientArguments Sub-task Resolved Saisai Shao
          8.
          Remove history serving functionality from standalone Master Sub-task Resolved Bryan Cutler
          9.
          Remove closure serializer Sub-task Resolved Sean Owen
          10.
          Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator Sub-task Resolved Sean Owen
          11.
          Remove usage of Hadoop deprecated APIs and reflection that supported 1.x Sub-task Resolved Sean Owen
          12.
          Rename "dynamic allocation" to "elastic scaling" Sub-task Closed Andrew Or
          13.
          Remove JobLogger Sub-task Resolved Reynold Xin
          14.
          Remove HTTPBroadcast Sub-task Resolved Reynold Xin
          15.
          Remove TTL-based metadata cleaning (spark.cleaner.ttl) Sub-task Resolved Josh Rosen
          16.
          Confusing behavior of fold function of RDD in pyspark Sub-task Resolved Sean Owen
          17.
          Remove deprecated methods in SQL / DataFrames Sub-task Resolved Reynold Xin
          18.
          Java count(AprroxDistinct)ByKey methods return Scala Long not Java Sub-task Resolved Sean Owen
          19.
          Remove some deprecated APIs in RDD/SparkContext Sub-task Resolved Reynold Xin
          20.
          Remove SparkContext preferred locations constructor Sub-task Resolved Reynold Xin
          21.
          Clean up build warnings: 2.0.0 edition Sub-task Resolved Sean Owen
          22.
          Remove unused code related to Hadoop 0.23 Sub-task Resolved Kousuke Saruta
          23.
          Remove Vector, VectorSuite and GraphKryoRegistrator which are deprecated and no longer used Sub-task Resolved Kousuke Saruta
          24.
          Remove block manager's internal "external block store" API Sub-task Resolved Reynold Xin
          25.
          Remove submit python/R scripts through pyspark/sparkR Sub-task Resolved Jeff Zhang
          26.
          Remove HistoryServer old multiple files format Sub-task Resolved Felix Cheung
          27.
          Deprecate TaskContext.isRunningLocally() Sub-task Resolved Josh Rosen
          28.
          Change default build to Hadoop 2.7 Sub-task Resolved Unassigned
          29.
          Follow up SPARK-12707, Update documentation and other related code Sub-task Resolved Unassigned
          30.
          Spark on Yarn remove deprecated configs for 2.0 Sub-task Closed Boyang Jerry Peng
          31.
          Remove usage of ADD_FILES in pyspark Sub-task Resolved Jeff Zhang
          32.
          Remove deprecate APIs from GraphX Sub-task Resolved Takeshi Yamamuro
          33.
          Deprecate "yarn-client" and "yarn-cluster" Sub-task Resolved Saisai Shao
          34.
          Logging: Make it either stable or private[spark] Sub-task Resolved Sean Owen
          35.
          Move UserDefinedFunction into sql.expressions package Sub-task Resolved Reynold Xin
          36.
          Upgrade to Jersey 2 Sub-task Resolved Matt Cheah
          37.
          Remove SparkContext.metricsSystem Sub-task Resolved Reynold Xin
          38.
          Remove the support of SIMR cluster manager Sub-task Resolved Saisai Shao
          39.
          Remove reference to Tachyon in cluster & release script Sub-task Resolved Reynold Xin
          40.
          Move network/* modules into common/network-* Sub-task Resolved Reynold Xin
          41.
          Move tags and unsafe modules into common Sub-task Resolved Reynold Xin
          42.
          Move docker, extras modules into external Sub-task Resolved Sean Owen
          43.
          Move misc top-level build files into appropriate subdirs Sub-task Resolved Sean Owen
          44.
          remove typed operations (map, flatMap, mapPartitions) from Python DataFrame Sub-task Resolved Wenchen Fan
          45.
          Move streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages Sub-task Resolved Shixiong Zhu
          46.
          Move docs of streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages Sub-task Resolved Shixiong Zhu
          47.
          Consider removing Kafka modules from Spark / Spark Streaming Sub-task Closed Unassigned
          48.
          Remove Streaming Akka docs from Spark Sub-task Resolved Shixiong Zhu
          49.
          Move streaming-flume back to Spark Sub-task Resolved Shixiong Zhu
          50.
          Remove straggler references to Tachyon Sub-task Resolved Liwei Lin(Inactive)
          51.
          Require Java 8 for Spark 2.x Sub-task Closed Unassigned
          52.
          Change SparkListener from a trait to an abstract class, and remove JavaSparkListener Sub-task Resolved Reynold Xin
          53.
          Remove HashShuffleManager Sub-task Resolved Reynold Xin
          54.
          Put examples packaging on a diet Sub-task Resolved Marcelo Vanzin
          55.
          Java sampleByKey methods take ju.Map but with Scala Double values; results in type Object Sub-task Resolved Sean Owen
          56.
          Remove support for "zk://" master URL Sub-task Resolved Sean Owen
          57.
          Add a deprecation warning for Python 2.6 Sub-task Resolved Unassigned

            Activity

              People

              • Assignee:
                rxin Reynold Xin
                Reporter:
                rxin Reynold Xin
              • Votes:
                0 Vote for this issue
                Watchers:
                18 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: