Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11881

DataFrame subpartitioning order is incorrect

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      Currently we've defined

      Nothing() < Index([i]) < Index([i,j]) < .. < Index() < Singleton()

      s.t. Singleton is a subpartitoning of Index, is a subpartitioning of Index([i,j]), but this is incorrect. The order should be

      Singleton() < Index([i]) < Index([i,j]) < .. < Index() < Nothing()

      s.t. every other partitioning is a subpartitioning of Singleton. This is logical, since Singleton will collect the largest amount of data on a single node, partitioning by a single index will be alittle more distributed, and partitioning by the full Index() will be the most distribtued.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              bhulette Brian Hulette
              Reporter:
              bhulette Brian Hulette

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 7h 20m
                7h 20m

                  Issue deployment