Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-37627

Add sorted column in BucketTransform

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete CommentsDelete
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0
    • SQL
    • None

    Description

      In V1, we can create table with sorted bucket like the following:

            sql("CREATE TABLE tbl(a INT, b INT) USING parquet " +
              "CLUSTERED BY (a) SORTED BY (b) INTO 5 BUCKETS")
      

      However, creating table with sorted bucket in V2 failed with Exception

      org.apache.spark.sql.AnalysisException: Cannot convert bucketing with sort columns to a transform.
      

      We should be able to create table with sorted bucket in V2.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            huaxingao Huaxin Gao Assign to me
            huaxingao Huaxin Gao
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment