[TAJO-1952] Implement PartitionFileFragment

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Fix Version/s: 0.12.0
• Component/s: Planner/Optimizer, Storage
• Assignee: JaeHwa Jung
• Reporter: JaeHwa Jung
• Created: 28/Oct/15 07:42
• Updated: 04/Dec/17 04:36
• Resolved: 04/Dec/17 04:36

Description

Currently, PartitionedTableScanNode contains the list of partitions and it seems to me that the list has some problems as following:

1. Duplicate Informs: Task contains Fragment which specify target directory or target file for scanning. A path of partition lists already would write to Fragment.

2. Network Resource: When scanning lost of partition, it will occupy network resource, for example, several hundred kilobytes or more. It looks like an unnecessary resource because Fragment already has the path of partitions.

I want to improve above problems by implementing new Fragment called PartitionedFileFragment. Currently, I'm planning the implementation as following:

• PartitionedFileFragment will borrow FileFragment and it contains the partition path and the partition key values.
• Remove the path array of partitions from PartitionedTableScanNode.
• Implement a method for getting filtered partition directories in FileTableSpace.
• Implement a method for making PartitionedFileFragment array.
• Before making splits, call above method and use it for making splits.

Issue Links

• blocks: TAJO-1891 Allow arbitrary partition path
• is depended upon by: TAJO-2101 Support add multiple partitions in a single ALTER TABLE statement
• relates to: TAJO-1493 Make partition pruning based on catalog informations
• relates to: TAJO-1927 PartitionedTableRewriter should use a local variable to remember the volume of partitions.
• links to: GitHub Pull Request #1020
• links to: Pull Request