Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-8556

Trim the number of columns to generate col stats out of the box

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete CommentsClose IssueResolve Issue
    XMLWordPrintableJSON

Details

    Description

      As of now, out of the box, we generate col stats for all top level fields. This could be prohibitively expensive for wider tables having 1000 columns. So, we should trim it down to say first 32 to level columns for a good out of the box performance. 

      Users will anyway have an option to override if need be. 

       

      Lets add a config to drive this, and out of the box we can set it to 32. 

       

      Attachments

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jonvex Jonathan Vexler Assign to me
            shivnarayan sivabalan narayanan
            sivabalan narayanan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Agile

                Active Sprint:
                Hudi 1.0 Blockers+Bugs Sprint ends 19/Nov/24
                View on Board

                Slack

                  Issue deployment