Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5166

Stabilize Spark SQL APIs

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersStop watchingWatchersCreate sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Component/s: SQL
    • Labels:
    • Target Version/s:

      Description

      Before we take Spark SQL out of alpha, we need to audit the APIs and stabilize them.

      As a general rule, everything under org.apache.spark.sql.catalyst should not be exposed.

        Attachments

        Issue Links

        1.
        Stabilize Spark SQL data type API Sub-task Resolved Reynold Xin Actions
        2.
        Move Row into sql package and make it usable for Java Sub-task Resolved Reynold Xin Actions
        3.
        Adding data frame APIs to SchemaRDD Sub-task Resolved Reynold Xin Actions
        4.
        Make SQLConf a field rather than mixin in SQLContext Sub-task Resolved Reynold Xin Actions
        5.
        Native Date type for SQL92 Date Sub-task Resolved Adrian Wang Actions
        6.
        [SQL] Public API in SQLContext to list tables Sub-task Resolved Bill Bejeck Actions
        7.
        Make Spark SQL API usable in Java and remove the Java-specific API Sub-task Resolved Reynold Xin Actions
        8.
        Move Decimal from types.decimal to types package Sub-task Resolved Adrian Wang Actions
        9.
        Enable javadoc/scaladoc for public classes in catalyst project Sub-task Resolved Michael Armbrust Actions
        10.
        Determine serializability of SQLContext Sub-task Resolved Reynold Xin Actions
        11.
        Clean up exposed classes in sql.hive package Sub-task Resolved Reynold Xin Actions
        12.
        Stabilize UDFRegistration API Sub-task Resolved Reynold Xin Actions
        13.
        Use java.math.BigDecimal as the exposed Decimal type Sub-task Resolved Reynold Xin Actions
        14.
        Update SQL programming guide for 1.3 Sub-task Resolved Unassigned Actions
        15.
        Row shouldn't extend Seq Sub-task Resolved Reynold Xin Actions
        16.
        Add defaultSizeOf to every data type Sub-task Resolved Yin Huai Actions
        17.
        Cross-langauge load/store functions for creating and saving DataFrames Sub-task Resolved Yin Huai Actions
        18.
        Make sure DataFrame expressions are usable in Java Sub-task Resolved Reynold Xin Actions
        19.
        Replace reference to SchemaRDD with DataFrame Sub-task Resolved Reynold Xin Actions
        20.
        Make CacheManager a concrete class and field in SQLContext Sub-task Resolved Reynold Xin Actions
        21.
        Remove Python LocalHiveContext Sub-task Resolved Reynold Xin Actions
        22.
        Break sql.py into multiple files Sub-task Resolved Davies Liu Actions
        23.
        SQLContext.createDataFrame shouldn't be an implicit function Sub-task Closed Reynold Xin Actions
        24.
        collect should call executeCollect Sub-task Resolved Reynold Xin Actions
        25.
        Error messages for plans with invalid AttributeReferences Sub-task Resolved Michael Armbrust Actions
        26.
        Create type alias for SchemaRDD for source backward compatibility Sub-task Resolved Reynold Xin Actions
        27.
        Support explode in DataFrame DSL Sub-task Resolved Michael Armbrust Actions
        28.
        Add more tests and docs for DataFrame Python API Sub-task Resolved Davies Liu Actions
        29.
        Create a convenient way for Python users to register SQL UDFs Sub-task Resolved Davies Liu Actions
        30.
        Provide a convenient way for Scala users to use UDFs Sub-task Resolved Reynold Xin Actions
        31.
        Provide support for project using SQL expression Sub-task Resolved Reynold Xin Actions
        32.
        support select/filter by SQL expression for Python DataFrame Sub-task Resolved Davies Liu Actions
        33.
        Better support for creating DataFrame from local data collection Sub-task Resolved Reynold Xin Actions
        34.
        Allow using String to specify colum name in DSL aggregate functions Sub-task Resolved Reynold Xin Actions
        35.
        Move DataFrame implicit functions into SQLContext.implicits Sub-task Resolved Reynold Xin Actions
        36.
        Add a config flag to disable eager analysis of DataFrames Sub-task Resolved Reynold Xin Actions
        37.
        Support DataFrame.renameColumn Sub-task Resolved Reynold Xin Actions
        38.
        Add a show method to print the content of a DataFrame in columnar format Sub-task Resolved Reynold Xin Actions
        39.
        XyzType companion object should subclass XyzType Sub-task Resolved Reynold Xin Actions
        40.
        Python DataFrame API remaining tasks Sub-task Resolved Davies Liu Actions
        41.
        DataFrame.to_pandas Sub-task Resolved Davies Liu Actions
        42.
        Allow short names for built-in data sources Sub-task Resolved Reynold Xin Actions
        43.
        createDataFrame replace applySchema/inferSchema Sub-task Resolved Davies Liu Actions
        44.
        Allow creating a DataFrame from local Python data Sub-task Resolved Davies Liu Actions
        45.
        Don't implicitly convert RDDs directly to DataFrames Sub-task Resolved Reynold Xin Actions
        46.
        Schema support in Row Sub-task Resolved Reynold Xin Actions
        47.
        fix Data Frame Python API Sub-task Resolved Davies Liu Actions
        48.
        DataFrame methods with varargs do not work in Java Sub-task Resolved Reynold Xin Actions
        49.
        sortBy -> orderBy in Python Sub-task Resolved Reynold Xin Actions
        50.
        Python DataFrame documentation fixes Sub-task Resolved Davies Liu Actions

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment