Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-50262

Forbid change complex types during altering collation

Rank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 4.0.0
    • 4.0.0
    • SQL

    Description

      SPARK-48413 has brought ability to change collation on table.
      But it seems to be too wide, because people couldn't change the whole StructType/MapType/ArrayType by altering before.
      Right now there is found problem with altering collation when table has metadata. Altering will be failed because of a metadata mismatch between user's type (that doesn't have metadata) and existing schema.
      So I suggest to shrink the feature to only leaf string types.
      People will be able to change any collation still, but they need to use full column path:

      ALTER TABLE table ALTER COLUMN full.column.path TYPE STRING COLLATE UNICODE

      Attachments

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            alexxxey Aleksei Shishkin
            alexxxey Aleksei Shishkin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment