[OOZIE-2819] Make Oozie REST API accept multibyte characters for script Actions

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.0.0b1
• Assignee: Attila Sasvári
• Reporter: Attila Sasvári
• Created: 10/Mar/17 13:26
• Updated: 25/Jan/18 21:00
• Resolved: 16/Mar/17 01:38

Description

Submitted Pig action with client side xml failed via proxy submission when it contained multibyte characters.

curl -i  -X POST -d @/tmp/pig.xml -H 'Content-Type: application/XML; charset=UTF-8' 'http://'localhost':11000/oozie/v1/jobs?jobtype=pig&action=start'

Where

$ hdfs dfs -cat /tmp/encoding/input.txt
松
林檎
松

$ cat /tmp/pig.xml 
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:8020/</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>localhost:8032</value>
</property>
<property>
<name>user.name</name>
<value>hdfs</value>
</property>
<property>
<name>oozie.pig.script</name>
<value><![CDATA[
lines = LOAD 'hdfs:///tmp/encoding/input.txt' USING PigStorage('\n') AS line;
test = FILTER lines BY line == '松';
STORE test INTO 'hdfs:///tmp/encoding/output' USING PigStorage('\n');
]]></value>
</property>
<property>
<name>oozie.pig.script.params.size</name>
<value>0</value>
</property>
<property>
<name>oozie.pig.script.options.size</name>
<value>0</value>
</property>
<property>
<name>oozie.libpath</name>
<value>hdfs:///user/oozie/share/lib</value>
</property>
<property>
<name>oozie.use.system.libpath</name>
<value>true</value>
</property>
<property>
<name>oozie.proxysubmission</name>
<value>true</value>
</property>
</configuration>

In the Oozie launcher log, I could see

lines = LOAD 'hdfs:///tmp/encoding/input.txt' USING PigStorage('\n') AS line;test = FILTER lines BY line == '~';STORE test INTO 'hdfs:///tmp/encoding/output' USING PigStorage('\n');

was used instead of the intended 松