[SPARK-22273] Fix key/value schema field names in HashMapGenerators.

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.1, 2.2.0
• Fix Version/s: 2.1.3, 2.2.1, 2.3.0
• Component/s: SQL
• Assignee: Takuya Ueshin
• Reporter: Takuya Ueshin
• Created: 13/Oct/17 09:03
• Updated: 14/Oct/17 06:26
• Resolved: 14/Oct/17 06:26

Description

When fixing schema field names using escape characters with addReferenceMinorObj() at SPARK-18952, double-quotes around the names were remained and the names become something like "((java.lang.String) references[1])".

/* 055 */     private int maxSteps = 2;
/* 056 */     private int numRows = 0;
/* 057 */     private org.apache.spark.sql.types.StructType keySchema = new org.apache.spark.sql.types.StructType().add("((java.lang.String) references[1])", org.apache.spark.sql.types.DataTypes.StringType);
/* 058 */     private org.apache.spark.sql.types.StructType valueSchema = new org.apache.spark.sql.types.StructType().add("((java.lang.String) references[2])", org.apache.spark.sql.types.DataTypes.LongType);
/* 059 */     private Object emptyVBase;

We should remove the double-quotes to refer the values in references properly:

/* 055 */     private int maxSteps = 2;
/* 056 */     private int numRows = 0;
/* 057 */     private org.apache.spark.sql.types.StructType keySchema = new org.apache.spark.sql.types.StructType().add(((java.lang.String) references[1]), org.apache.spark.sql.types.DataTypes.StringType);
/* 058 */     private org.apache.spark.sql.types.StructType valueSchema = new org.apache.spark.sql.types.StructType().add(((java.lang.String) references[2]), org.apache.spark.sql.types.DataTypes.LongType);
/* 059 */     private Object emptyVBase;

Issue Links

• links to: [Github] Pull Request #19491 (ueshin)