[HADOOP-3256] JobHistory file on HDFS should not use the 'job name'

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.17.0
• Fix Version/s: 0.17.0
• Hadoop Flags: Reviewed
• Assignee: Arun Murthy
• Reporter: Arun Murthy
• Created: 15/Apr/08 00:59
• Updated: 08/Jul/09 16:52
• Resolved: 16/Apr/08 16:55

Description

HADOOP-2178 introduced the feature of saving jobhistory logs on HDFS.

Unfortunately the following code:

        // setup the history log file for this job
        String logFileName = jobUniqueString +  
                             "_" + user+ "_" + jobName;
        if (logFileName.length() > MAX_FILENAME_SIZE) {
          logFileName = logFileName.substring(0, MAX_FILENAME_SIZE-1);
        }

is vulnerable to user-provided job names.

Specifically I ran into 'URISyntaxException' with jobs whose names include a ":".

The easy fix is to ensure that we do not use the human-friendly job names and only the jobid.

The long term fix is to ensure that Path handles filenames with any characters.

Issue Links

• blocks: PIG-198 Integration with Hadoop 0.17
• is related to: HADOOP-3257 Path should handle all characters
• relates to: HDFS-13 filenames with ':' colon throws java.lang.IllegalArgumentException