[ARROW-1359] [Python] Add Parquet writer option to normalize field names for use in Spark - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.7.0
Component/s: Python
Labels:
None

External issue URL:
https://github.com/apache/arrow/issues/17390

Description

Spaces and a few other characters are not permitted in field names in Spark: https://github.com/apache/spark/blob/cba826d00173a945b0c9a7629c66e36fa73b723e/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala#L565

Attachments

Activity

People

Assignee:: Wes McKinney

Reporter:: Wes McKinney

Votes:: 1 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 16/Aug/17 16:03

Updated:: 11/Jan/23 07:14

Resolved:: 10/Sep/17 06:35