Details
Description
On a one-box windows setup, do the following from a powershell prompt:
cmd /c curl.exe -s `
-d user.name=hadoop `
-d arg=-useHCatalog `
-d execute="emp = load '/data/emp/emp_0.dat'; dump emp;" `
-d statusdir="/tmp/webhcat.output01" `
'http://localhost:50111/templeton/v1/pig' -v
The job fails with error code 7, but it should run.
I traced this down to the following. In the job configuration for the TempletonJobController, we have templeton.args set to
cmd,/c,call,C:\\hadoop\\\\pig-0.11.0.1.3.0.0-0846/bin/pig.cmd,-D_WEBHCAT_TOKEN_FILE_LOCATION_="-useHCatalog",-execute,"emp = load '/data/emp/emp_0.dat'; dump emp;"
Notice the = sign before "-useHCatalog". I think this should be a comma.
The bad string D_WEBHCAT_TOKEN_FILE_LOCATION_="-useHCatalog" gets created in org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows().
It happens at line 434:
} else { if (i < args.length - 1) { prop += "=" + args[++i]; // RIGHT HERE! at iterations i = 37, 38 } }
Bug is here:
if (prop != null) { if (prop.contains("=")) { // -D__WEBHCAT_TOKEN_FILE_LOCATION__ does not contain equal, so else branch is run and appends ="-useHCatalog", // everything good } else { if (i < args.length - 1) { prop += "=" + args[++i]; } } newArgs.add(prop); }
One possible fix is to change the string constant org.apache.hcatalog.templeton.tool.TempletonControllerJob.TOKEN_FILE_ARG_PLACEHOLDER to have an "=" sign in it. Or, preProcessForWindows() itself could be changed.