[HOP-3933] Round decimal value to fit in scale in Numeric field in Monetdb Bulk Loader

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Labels: pull-request-available
• Assignee: Sergio Ramazzina
• Reporter: Enrico Maria Carmona
• Created: 06/May/22 10:18
• Updated: 30/May/22 07:17
• Resolved: 30/May/22 07:17

Description

With Monetdb Bulk Loader, inserting a number with more decimals than the scale raise an error:

2022/05/06 11:52:05 - MonetDB bulk loader.0 - ERROR: Error in transform, asking everyone to stop because of:
2022/05/06 11:52:05 - MonetDB bulk loader.0 - ERROR: org.apache.hop.core.exception.HopException: 
2022/05/06 11:52:05 - MonetDB bulk loader.0 - An error occurred writing data to the MonetDB API (MAPI) process
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 
2022/05/06 11:52:05 - MonetDB bulk loader.0 - Error loading data: 42000!Failed to import table 'test_decimal', line 1 field column1 'decimal(18,3)' expected in '12345.1234567'
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 	at org.apache.hop.pipeline.transforms.monetdbbulkloader.MonetDbBulkLoader.writeBufferToMonetDB(MonetDbBulkLoader.java:508)
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 	at org.apache.hop.pipeline.transforms.monetdbbulkloader.MonetDbBulkLoader.processRow(MonetDbBulkLoader.java:151)
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 	at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:58)
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 	at java.base/java.lang.Thread.run(Thread.java:834)
2022/05/06 11:52:05 - MonetDB bulk loader.0 - Caused by: org.apache.hop.core.exception.HopException: 
2022/05/06 11:52:05 - MonetDB bulk loader.0 - Error loading data: 42000!Failed to import table 'test_decimal', line 1 field column1 'decimal(18,3)' expected in '12345.1234567'
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 	at org.apache.hop.pipeline.transforms.monetdbbulkloader.MonetDbBulkLoader.writeBufferToMonetDB(MonetDbBulkLoader.java:486)
2022/05/06 11:52:05 - MonetDB bulk loader.0 - 	... 3 more

This is because the COPY INTO command, instead the SQL INSERT rounds it.

We can force the transform to adhere to the scale of the numeric field table rounding the input value when the "Field format OK" = False.

Issue Links

• links to: GitHub Pull Request #1505