Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12144

Support more external data source API in SparkR

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.5.2
    • 1.6.0
    • SparkR
    • None

    Description

      Currently we support read.df, write.df, jsonFile, parquetFile in SQLContext, we should support more external data source API such as read.json, read.parquet, read.orc, read.jdbc, read.csv and so on.
      Some of the exist API is deprecated and will remove at Spark 2.0, we should also deprecate them at SparkR.
      Note: we should refer the DataFrameReader and DataFrameWriter API at Spark SQL but defined with R-like style.
      DataFrameReader API: http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameReader
      DataFrameWriter API: http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameWriter

      Attachments

        Issue Links

          incorporates

          Bug - A problem which impairs or prevents the functions of the product. SPARK-12224 R support for JDBC source

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue SPARK-7738 DataFrame reader/writer API in Python

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          There are no Sub-Tasks for this issue.

          Activity

            People

              yanboliang Yanbo Liang
              sunrui Sun Rui
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: