Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-6632

Migrate connectors to Spark-3

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • connectors-6.0.0
    • connectors-6.0.0
    • spark-connector
    • None
    • Phoenix connectors now includes a connector for Phoenix 5 and Spark 3.

    Description

      With Spark-3, the DatasourceV2 API has had major changes, where a new TableProvider Interface has been introduced. These new changes bring in more control to the data source developer and better integration with spark-optimizer.

      Attachments

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6667 Spark3 connector requires that all columns are specified when writing

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6668 Spark3 connector cannot distinguish column name cases

          • Major - Major loss of function.
          • Resolved
          causes

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6772 org.apache.phoenix.spark.DataSourceApiIT.DataSourceApiIT is missing ASF header

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. PHOENIX-6031 Update phoenix-spark to use Spark 3.0.0

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. PHOENIX-6354 Update to spark 3.0

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6268 NoSuchMethodError when writing from Spark Dataframe to Phoenix with phoenix-spark connector

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. PHOENIX-6657 Port Phoenix-Spark connector tests to the current API

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #69

          Web Link GitHub Pull Request #71

          Activity

            People

              stoty Istvan Toth
              ashwinb1998 Ashwin Balasubramani
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: