Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-959

JobConf::setWorkingDirectory requires that the default FileSystem is reachable

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: client, test
    • Labels:
      None

      Description

      If mapred.working.dir is not set, JobConf::setWorkingDirectory will attempt to obtain the default working directory for the default FileSystem. In trunk at least, if the default fs is HDFS and not reachable, set will fail:

      java.net.UnknownHostException: unknown host: notahost
      java.lang.RuntimeException: java.net.UnknownHostException: unknown host: notahost
        at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:541)
        at org.apache.hadoop.mapred.JobConf.setWorkingDirectory(JobConf.java:522)
        at org.apache.hadoop.conf.TestJobConf.testSetWorkingDir(TestJobConf.java:67)
      Caused by: java.net.UnknownHostException: unknown host: notahost
        at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:216)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:876)
        at org.apache.hadoop.ipc.Client.call(Client.java:746)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:223)
        at $Proxy4.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:366)
        at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:169)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:276)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:235)
        at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:83)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1430)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:69)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1458)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1446)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:190)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:98)
        at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:537)
      
      1. M959-test.patch
        2 kB
        Chris Douglas

        Activity

        Chris Douglas created issue -
        Chris Douglas made changes -
        Field Original Value New Value
        Attachment M959-test.patch [ 12418858 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Chris Douglas
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development