Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-959

JobConf::setWorkingDirectory requires that the default FileSystem is reachable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: client, test
    • Labels:
      None

      Description

      If mapred.working.dir is not set, JobConf::setWorkingDirectory will attempt to obtain the default working directory for the default FileSystem. In trunk at least, if the default fs is HDFS and not reachable, set will fail:

      java.net.UnknownHostException: unknown host: notahost
      java.lang.RuntimeException: java.net.UnknownHostException: unknown host: notahost
        at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:541)
        at org.apache.hadoop.mapred.JobConf.setWorkingDirectory(JobConf.java:522)
        at org.apache.hadoop.conf.TestJobConf.testSetWorkingDir(TestJobConf.java:67)
      Caused by: java.net.UnknownHostException: unknown host: notahost
        at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:216)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:876)
        at org.apache.hadoop.ipc.Client.call(Client.java:746)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:223)
        at $Proxy4.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:366)
        at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:169)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:276)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:235)
        at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:83)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1430)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:69)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1458)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1446)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:190)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:98)
        at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:537)
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              chris.douglas Chris Douglas
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: