[DRILL-3370] FLATTEN error with a where clause

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.1.0
• Component/s: Functions - Drill
• Assignee: Jason Altekruse
• Reporter: Joanlynn LIN
• Created: 25/Jun/15 06:21
• Updated: 30/Jun/15 01:00
• Resolved: 30/Jun/15 01:00

Description

I've got a JSON file which contains 150 JSON strings all like this:

{"arr": [94]} {"arr": [39]} {"arr": [180]}

I was trying to Flatten() the arrays and filter the values using such an SQL query:
select flatten(arr) as a from dfs.`/data/test/jsonarray.150.json` where a > 100;
However, it returned no result. Then I modified my expression like this:
select a from (select flatten(arr) as a from dfs.`/data/test/jsonarray.150.json`) where a > 100;
It then failed:
Error: SYSTEM ERROR: org.apache.drill.exec.exception.SchemaChangeException: Failure while trying to materialize incoming schema. Errors:
Error in expression at index 1. Error: Missing function implementation: [flatten(BIGINT-REPEATED)]. Full expression: --UNKNOWN EXPRESSION-..
Fragment 0:0
[Error Id: 1d71bf0e-48da-43f8-8b36-6a513120d7e0 on slave2:31010] (state=,code=0)
After a lot of attempts, I finally got it work:
select a from (select flatten(arr) as a from dfs.`/data/test/jsonarray.150.json` limit 10000000) where a > 100;
See, I just added a "limit 10000000" in this query and I am wondering if this is a bug or what in Drill?
Looking forward to your attention and help. Many thanks.