[ARROW-10492] [Java][JDBC] Allow users to config the mapping between SQL types and Arrow types

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Java
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/26465
• Assignee: Liya Fan
• Reporter: Liya Fan
• Created: 04/Nov/20 06:38
• Updated: 11/Jan/23 08:13
• Resolved: 04/Jan/21 22:30

Description

According to the current implementation of JDBC adapter, the conversion between SQL types and Arrow types is hard-coded. This will cause some problems in practice:

1. The appropriate conversion may vary for different databases. For example, for SQL Server, type real corresponds to 4 byte floating point values (https://docs.microsoft.com/en-us/sql/t-sql/data-types/float-and-real-transact-sql?view=sql-server-ver15), whereas for SQLite, real corresponds to 8 byte floating point values (https://www.sqlitetutorial.net/sqlite-data-types/). If the maping is not right, some extra conversion would happen, which can impact performance severely. 
2. Our current implementation determines the type conversion solely based on the type ID. However, the appropriate conversion may also depend some other information, like precision and scale. For example, for FLOAT( n ), it should correspond to 4 byte floating point values, if n <= 24, otherwise, it should correspond to 8 byte floating point values.  

To address the problems, we should allow users to customize the conversion between SQL and Arrow types. 

Issue Links

• blocks: ARROW-10762 [Java] Configuration does not provide a mapping for array column
• links to: GitHub Pull Request #8597