Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33160

Allow saving/loading INT96 in parquet w/o rebasing

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.1.0
    • Fix Version/s: 3.1.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Currently, Spark always performs rebasing of INT96 columns in Parquet datasource but this is not required by parquet spec. This tickets aims to allow users to turn off rebasing via SQL config.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                maxgekk Maxim Gekk
                Reporter:
                maxgekk Maxim Gekk
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: