Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-574

Add a sort-based physical executor for column partition store

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Physical Operator
    • Labels:
      None

      Description

      ColumnPartitionStoreExec keeps numerous open files while it is storing all data. In addition, it's random write gives burden to HDFS namenode.

      To solve this problem, I would like to propose a sort-based physical executor for column partition store. It assumes that input tuples are sorted in an ascending or descending order of partition keys. It means that it needs extra sort operation. But, it opens only one file simultaneously. It writes all data sequentially. In many cases, it would be the best choice for column partition store.

        Attachments

        1. TAJO-574.patch
          51 kB
          Hyunsik Choi

          Activity

            People

            • Assignee:
              hyunsik Hyunsik Choi
              Reporter:
              hyunsik Hyunsik Choi
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: