• Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Environment:

      Windows + Cygwin


      When I try to run a datanode on Windows, I get the following exception: Expecting a line not the end of stream
      	at org.apache.hadoop.fs.DU.parseExecResult(
      	at org.apache.hadoop.util.Shell.runCommand(
      	at org.apache.hadoop.fs.DU.<init>(
      	at org.apache.hadoop.fs.DU.<init>(
      	at org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolume.<init>(
      	at org.apache.hadoop.hdfs.server.datanode.FSDataset.<init>(
      	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(
      	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(
      	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(
      	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(

      This is because Hadoop execs "du -sk C:\tmp\hadoop-SYSTEM\dfs\data" with a Windows path representation, which cygwin du doesn't understand.

      C:\hudson>du -sk C:\tmp\hadoop-SYSTEM\dfs\data
      du -sk C:\tmp\hadoop-SYSTEM\dfs\data
      du: cannot access `C:\\tmp\\hadoop-SYSTEM\\dfs\\data': No such file or directory

      For this to work correctly, Hadoop would have to run cygpath first to get a Unix path representation, then to call DU.

      Also, I had to use the debugger to get this information. Shell.runCommand should catch IOException from parseExecResult and add the buffered stderr to simplify the error diagnostics.

      1. HADOOP-5486
        0.6 kB
        Kohsuke Kawaguchi

        Issue Links


          Kohsuke Kawaguchi created issue -
          Doug Cutting made changes -
          Field Original Value New Value
          Assignee Kohsuke Kawaguchi [ kohsuke ]
          Kohsuke Kawaguchi made changes -
          Attachment HADOOP-5486 [ 12402304 ]
          Steve Loughran made changes -
          Link This issue is related to HADOOP-5875 [ HADOOP-5875 ]
          Owen O'Malley made changes -
          Project Hadoop Common [ 12310240 ] HDFS [ 12310942 ]
          Key HADOOP-5496 HDFS-197
          Affects Version/s 0.19.1 [ 12313473 ]
          Component/s dfs [ 12310710 ]
          Harsh J made changes -
          Assignee Kohsuke Kawaguchi [ kohsuke ] Harsh J [ qwertymaniac ]
          Harsh J made changes -
          Assignee Harsh J [ qwertymaniac ]
          Allen Wittenauer made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]


            • Assignee:
              Kohsuke Kawaguchi
            • Votes:
              0 Vote for this issue
              5 Start watching this issue


              • Created: