[PHOENIX-2288] Phoenix-Spark: PDecimal precision and scale aren't carried through to Spark DataFrame

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.5.2
• Fix Version/s: 4.7.0
• Assignee: Josh Mahonin
• Reporter: Josh Mahonin
• Created: 23/Sep/15 18:06
• Updated: 01/Aug/23 12:49
• Resolved: 13/Nov/15 17:47

Description

When loading a Spark dataframe from a Phoenix table with a 'DECIMAL' type, the underlying precision and scale aren't carried forward to Spark.

The Spark catalyst schema converter should load these from the underlying column. These appear to be exposed in the ResultSetMetaData, but if there was a way to expose these somehow through ColumnInfo, it would be cleaner.

I'm not sure if Pig has the same issues or not, but I suspect it may.

Issue Links

• links to: GitHub Pull Request #124