[DRILL-1681] select with limit on directory with csv files takes quite long to terminate

• Type: Bug
• Status: Reopened
• Priority: Minor
• Resolution: Unresolved
• Fix Version/s: Future
• Component/s: Storage - Text & CSV
• Assignee: Unassigned
• Reporter: Suresh Ollala
• Created: 10/Nov/14 23:53
• Updated: 24/Jun/16 21:50

Description

query like select * from `/drill/data` limit 100 takes quite long to terminate, about 20+ seconds.

/drill/data includes overall 1100 csv files, all in single directory.

select * from `/drill/data/d2.csv` limit 100; terminates in 0.2 seconds.

Issue Links

• duplicates: DRILL-1072 Drill is very slow when we have a large number of text files
• is related to: DRILL-3722 LIMIT 1 query on top of a dir with 50K files takes ~150 seconds
• relates to: DRILL-1682 Avoid using UnionExchange for read workloads