[DRILL-7960] Column metadata DECIMAL precision can exceed max supported value

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.20.0
• Fix Version/s: 1.20.2
• Component/s: Client - HTTP

Environment

Drill in Docker

Parquet Files with VARDECIMAL(29,6)

 

• Assignee: James Turton
• Reporter: Nick Stenroos-Dam
• Created: 01/Jul/21 10:34
• Updated: 21/Jul/22 08:28
• Resolved: 21/Jul/22 08:28

Description

We are using a lot of decimals in our data and keep running into issues.

The latest issue is this error message:

SYSTEM ERROR: Drill Remote Exception

Please, refer to logs for more information.

 (java.lang.IllegalArgumentException) VARDECIMAL(40, 6) exceeds maximum suppored precision of 38
 org.apache.drill.exec.record.metadata.MetadataUtils.newDecimal():228
 org.apache.drill.exec.record.metadata.MetadataUtils.fromField():66
 org.apache.drill.exec.physical.resultSet.model.single.SingleSchemaInference.inferVector():86
 org.apache.drill.exec.physical.resultSet.model.single.SingleSchemaInference.infer():64
 org.apache.drill.exec.physical.rowSet.DirectRowSet.fromContainer():80
 org.apache.drill.exec.physical.resultSet.impl.PushResultSetReaderImpl.createRowSet():100
 org.apache.drill.exec.physical.resultSet.impl.PushResultSetReaderImpl.start():85
 org.apache.drill.exec.server.rest.stream.StreamingHttpConnection.sendData():116
 org.apache.drill.exec.ops.AccountingUserConnection.sendData():43
 org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():106
 org.apache.drill.exec.physical.impl.BaseRootExec.next():93
 org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():323
 org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():310
 .......():0
 org.apache.hadoop.security.UserGroupInformation.doAs():1762
 org.apache.drill.exec.work.fragment.FragmentExecutor.run():310
 org.apache.drill.common.SelfCleaningRunnable.run():38
 .......():0

I have now been able to reproduce the issue.
Apparently the issue only occurs when using the Web API (query.json) and not the Web interface.

To reproduce the issue run

curl -X POST http://localhost:8047/query.json -d '{ "queryType": "SQL", "query": "SELECT CAST(10 AS DECIMAL(38, 4)) AS `Col1`\nUNION ALL \nSELECT CAST(22 AS DECIMAL(29, 6)) AS `Col1`", "autoLimit": null, "defaultSchema": "data" }' -H 'Content-Type: application/json'