[SQOOP-1685] HCatalog integration is not working on JDK8

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.5
• Fix Version/s: 1.4.6
• Assignee: Jarek Jarcec Cecho
• Reporter: Jarek Jarcec Cecho
• Created: 06/Nov/14 02:31
• Updated: 06/Nov/14 06:14
• Resolved: 06/Nov/14 05:17

Description

I was looking into JDK8 compatibility in Sqoop 1 and I found out that HCatalog tests are broken, failing on exceptions such as this one:

14/11/05 16:29:06 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Projected column id not in list of columns from database
	at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.initDBColumnInfo(SqoopHCatUtilities.java:483)
	at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:314)
	at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:753)

Looking deeper into our HCatalog code, we do have class LCKeyMap that is suppose to store all it's keys in lower case form and provide case independent lookup. Indeed we're overriding methods put and get and lowercasing the key parameter to achieve desired behavior.

Now to the actual problem. We are in the code calling method putAll to bulk insert values from another Map. This worked in JDK7 as the underlaying implementation of putAll method is calling put method that we're overriding. It seems that this has changed in JDK8 where the putAll method is no longer calling put and hence when we call putAll all the keys are stored with their original case and subsequent lookup fails

Issue Links

• links to: Review board