Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5960

JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0, 2.4.1
    • Fix Version/s: 2.6.0
    • Component/s: client
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      $ mapred job -submit myjob.xml 
      2014-07-06 15:06:34.731 java[12120:1903] Unable to load realm info from SCDynamicStore
      14/07/06 15:06:34 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      14/07/06 15:06:34 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
      14/07/06 15:06:37 WARN conf.Configuration: bad conf file: element not <property>
      14/07/06 15:06:37 WARN conf.Configuration: bad conf file: element not <property>
      14/07/06 15:06:37 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
      14/07/06 15:11:11 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/gshegalov/.staging/job_1404679996131_0004
      Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: viewfs:/user/gshegalov/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar, expected: file:///
      	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:643)
      	at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:80)
      	at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:525)
      	at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:739)
      	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:520)
      	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
      	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
      	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1902)
      	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1870)
      	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1835)
      	at org.apache.hadoop.mapreduce.JobSubmitter.copyJar(JobSubmitter.java:286)
      	at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:254)
      	at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
      	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
      	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
      	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
      	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
      	at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:260)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
      	at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1239)
      
      1. MAPREDUCE-5960.v01.patch
        2 kB
        Gera Shegalov
      2. MAPREDUCE-5960.v02.patch
        10 kB
        Gera Shegalov
      3. MAPREDUCE-5960.v03.patch
        9 kB
        Gera Shegalov
      4. MAPREDUCE-5960.v04.patch
        13 kB
        Gera Shegalov

        Issue Links

          Activity

          Jason Lowe made changes -
          Link This issue breaks HADOOP-11409 [ HADOOP-11409 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Jason Lowe made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.6.0 [ 12327180 ]
          Resolution Fixed [ 1 ]
          Gera Shegalov made changes -
          Attachment MAPREDUCE-5960.v04.patch [ 12679705 ]
          Gera Shegalov made changes -
          Attachment MAPREDUCE-5960.v03.patch [ 12679429 ]
          Gera Shegalov made changes -
          Attachment MAPREDUCE-5960.v02.patch [ 12674646 ]
          Gera Shegalov made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Gera Shegalov made changes -
          Field Original Value New Value
          Attachment MAPREDUCE-5960.v01.patch [ 12654237 ]
          Gera Shegalov created issue -

            People

            • Assignee:
              Gera Shegalov
              Reporter:
              Gera Shegalov
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development