Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3256

JobHistory file on HDFS should not use the 'job name'

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.17.0
    • 0.17.0
    • None
    • None
    • Reviewed

    Description

      HADOOP-2178 introduced the feature of saving jobhistory logs on HDFS.

      Unfortunately the following code:

              // setup the history log file for this job
              String logFileName = jobUniqueString +  
                                   "_" + user+ "_" + jobName;
              if (logFileName.length() > MAX_FILENAME_SIZE) {
                logFileName = logFileName.substring(0, MAX_FILENAME_SIZE-1);
              }
      

      is vulnerable to user-provided job names.

      Specifically I ran into 'URISyntaxException' with jobs whose names include a ":".

      The easy fix is to ensure that we do not use the human-friendly job names and only the jobid.

      The long term fix is to ensure that Path handles filenames with any characters.

      Attachments

        1. HADOOP-3256_2_20080415.patch
          21 kB
          Arun Murthy
        2. HADOOP-3256_2_20080415.patch
          21 kB
          Arun Murthy
        3. HADOOP-3256_2_20080415.patch
          22 kB
          Arun Murthy
        4. HADOOP-3256_2_20080415.patch
          22 kB
          Arun Murthy
        5. HADOOP-3256_1_20080415.patch
          5 kB
          Arun Murthy
        6. HADOOP-3256_0_20080414.patch
          2 kB
          Arun Murthy

        Issue Links

          Activity

            People

              acmurthy Arun Murthy
              acmurthy Arun Murthy
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: