Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-6668

Spark3 connector cannot distinguish column name cases

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • connectors-6.0.0
    • None
    • None

    Description

      The Spark2 connector handled lowercase and mixed case column names correctly in DataFrame definitions.

      Spark3 only does case-insensitive column resolving, and even spark.sql.caseSensitive  doesn't seem to do anything, neither backquouting.

      Again, this is not something that can likely be fixes from the Phoenix side without changes in Spark, and this ticket is mainy for documenting this regression.

      Attachments

        Issue Links

          is broken by

          Improvement - An improvement or enhancement to an existing feature or task. PHOENIX-6632 Migrate connectors to Spark-3

          • Major - Major loss of function.
          • Resolved
          is fixed by

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-6667 Spark3 connector requires that all columns are specified when writing

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #94

          Activity

            People

              attilapiros Attila Zsolt Piros
              stoty Istvan Toth
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: