[DERBY-1676] Dynamically change the max merge runs in a merge phase during external sorting.

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 10.0.2.0, 10.0.2.1, 10.1.1.0, 10.1.2.1, 10.1.3.1, 10.2.1.6
• Component/s: Store
• Assignee: Unassigned
• Reporter: Sunitha Kambhampati
• Created: 11/Aug/06 19:08
• Updated: 11/Aug/06 20:29

Description

Please see solution #3 proposed in DERBY-1661 notes.
http://issues.apache.org/jira/secure/attachment/12338618/1661_Notes.txt

Currently, the max number of open files for a merge phase is 512. Worst case, the sort module can keep 512 open files per connection during the merge phase. It seems that we should be able to dynamically change the max merge runs.

Even though the 512 limit is there, it is very much possible to hit the too many file descriptor limit given other files open on the system. One interesting solution would be to catch the FileNotFoundException that gets thrown when during the scan , when an error happens on opening the merge run file and dynamically change the max_merge_runs to accommadate this.

This jira is to investigate this issue.

More improvements are possible in the sort module, see derby-1661

Issue Links

• is related to: DERBY-1677 Investigate increasing the memory limit that is used to calculate the sortbuffer size for sorting.
• relates to: DERBY-1661 Create index on large tables fails with too many open files , FileNotFoundException.
• relates to: DERBY-1679 Improve usage of file resources when sorting