Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-6632

Migrate connectors to Spark-3

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • connectors-6.0.0
    • connectors-6.0.0
    • spark-connector
    • None
    • Phoenix connectors now includes a connector for Phoenix 5 and Spark 3.

    Description

      With Spark-3, the DatasourceV2 API has had major changes, where a new TableProvider Interface has been introduced. These new changes bring in more control to the data source developer and better integration with spark-optimizer.

      Attachments

        Issue Links

        breaks

        Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6667 Spark3 connector requires that all columns are specified when writing

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6668 Spark3 connector cannot distinguish column name cases

        • Major - Major loss of function.
        • Resolved
        causes

        Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6772 org.apache.phoenix.spark.DataSourceApiIT.DataSourceApiIT is missing ASF header

        • Major - Major loss of function.
        • Resolved
        is duplicated by

        Improvement - An improvement or enhancement to an existing feature or task. PHOENIX-6031 Update phoenix-spark to use Spark 3.0.0

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. PHOENIX-6354 Update to spark 3.0

        • Major - Major loss of function.
        • Resolved
        relates to

        Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6268 NoSuchMethodError when writing from Spark Dataframe to Phoenix with phoenix-spark connector

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. PHOENIX-6657 Port Phoenix-Spark connector tests to the current API

        • Major - Major loss of function.
        • Resolved
        links to

        Web Link GitHub Pull Request #69

        Web Link GitHub Pull Request #71

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stoty Istvan Toth
            ashwinb1998 Ashwin Balasubramani
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment