Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26797

Start using the new logical types API of Parquet 1.11.0 instead of the deprecated one

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete CommentsDelete
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 3.0.0
    • None
    • SQL
    • None

    Description

      The 1.11.0 release of parquet-mr will deprecate its logical type API in favour of a newly introduced one. The new API also introduces new subtypes for different timestamp semantics, support for which should be added to Spark in order to read those types correctly.

      At this point only a release candidate of parquet-mr 1.11.0 is available, but that already allows implementing and reviewing this change.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            zi Zoltan Ivanfi

            Dates

              Created:
              Updated:

              Slack

                Issue deployment