Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34440

Allow saving/loading datetime in ORC w/o rebasing

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.2.0
    • None
    • SQL
    • None

    Description

      Currently, Spark always performs rebasing of date/timestamp columns in ORC datasource but this is not required by parquet spec. This tickets aims to allow users to turn off rebasing via SQL configs or DS options.

      Attachments

        1. 000000_0
          0.3 kB
          Wei-Che Wei

        Issue Links

          is a clone of

          New Feature - A new feature of the product, which has yet to be developed. SPARK-33160 Allow saving/loading INT96 in parquet w/o rebasing

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              maxgekk Max Gekk
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: