[PARQUET-1285] [Java] SchemaConverter should not convert from TimeUnit.SECOND AND TimeUnit.NANOSECOND of Arrow

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: parquet-mr
• Assignee: Masayuki Takahashi
• Reporter: Masayuki Takahashi
• Due: 29/Apr/18
• Created: 29/Apr/18 09:17
• Updated: 02/Aug/18 13:04
• Resolved: 07/May/18 08:12

Description

Arrow's 'Time' definition is below:

{ "name" : "time", "unit" : "SECOND|MILLISECOND|MICROSECOND|NANOSECOND", "bitWidth": /* integer: 32 or 64 */ }

http://arrow.apache.org/docs/metadata.html

 

But Parquet only supports 'TIME_MILLIS' and 'TIME_MICROS'.
https://github.com/Apache/parquet-format/blob/master/LogicalTypes.md

Therefore SchemaConverter should not convert from TimeUnit.SECOND AND TimeUnit.NANOSECOND of Arrow to Parquet.
 

Issue Links

• is cloned by: PARQUET-1297 [Java] SchemaConverter should not convert from Timestamp(TimeUnit.SECOND) and Timestamp(TimeUnit.NANOSECOND) of Arrow
• links to: GitHub Pull Request #469