Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3203

TaskTracker::localizeJob doesn't provide the correct size to LocalDirAllocator

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.15.3
    • 0.18.0
    • None
    • None
    • Reviewed

    Description

      In TaskTracker::localizeJob:

          // Get sizes of JobFile and JarFile
          // sizes are -1 if they are not present.
          FileSystem fileSystem = FileSystem.get(fConf);
          FileStatus status[] = fileSystem.listStatus(new Path(jobFile).getParent());
          long jarFileSize = -1;
          long jobFileSize = -1;
          for(FileStatus stat : status) {
            if (stat.getPath().toString().contains("job.xml")) {
              jobFileSize = stat.getLen();
            } else {
              jobFileSize = -1;
            }
            if (stat.getPath().toString().contains("job.jar")) {
              jarFileSize = stat.getLen();
            } else {
              jarFileSize = -1;
            }
          }
      

      One or both of jobFileSize and jarFileSize will be -1.

      Attachments

        1. patch-3203.txt
          1 kB
          Amareshwari Sriramadasu
        2. patch-3203.txt
          3 kB
          Amareshwari Sriramadasu

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            amareshwari Amareshwari Sriramadasu
            cdouglas Christopher Douglas
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment