Hadoop Common
  1. Hadoop Common
  2. HADOOP-836

unit tests fail on windows (/C:/cygwin/... is invalid)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:
      None

      Description

      Under windows, I get the following exception from some of the unit tests:

      java.io.IOException: Invalid file name: /C:/cygwin/home/hadoopwin/owen/hadoop/build/test/mapre
      d/local/50062_0/taskTracker/jobcache/job_0001/task_0001_m_000002_0/.split.dta.crc
      at org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:416)
      at org.apache.hadoop.dfs.NameNode.create(NameNode.java:238)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:585)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:337)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:538)

      1. dos-dfs.patch
        3 kB
        Owen O'Malley

        Activity

        Owen O'Malley created issue -
        Owen O'Malley made changes -
        Field Original Value New Value
        Description Under windows, I get the following exception from some of the unit tests:

        java.io.IOException: Invalid file name: /C:/cygwin/home/hadoopwin/owen/hadoop/build/test/mapre
        d/local/50062_0/taskTracker/jobcache/job_0001/task_0001_m_000002_0/.split.dta.crc
                at org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:416)
                at org.apache.hadoop.dfs.NameNode.create(NameNode.java:238)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:2
        5)
                at java.lang.reflect.Method.invoke(Method.java:585)
                at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:337)
                at org.apache.hadoop.ipc.Server$Handler.run(Server.java:538)

        Under windows, I get the following exception from some of the unit tests:

        java.io.IOException: Invalid file name: /C:/cygwin/home/hadoopwin/owen/hadoop/build/test/mapre
        d/local/50062_0/taskTracker/jobcache/job_0001/task_0001_m_000002_0/.split.dta.crc
                at org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:416)
                at org.apache.hadoop.dfs.NameNode.create(NameNode.java:238)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                at java.lang.reflect.Method.invoke(Method.java:585)
                at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:337)
                at org.apache.hadoop.ipc.Server$Handler.run(Server.java:538)

        Hide
        Owen O'Malley added a comment -

        Paths without schemes were being used in MapTask.java. So the local split file was being written into dfs by mistake. This patch fixes that problem, makes the dfs client reject illegal names immediately, and adds a new static method to get the local file system.

        Show
        Owen O'Malley added a comment - Paths without schemes were being used in MapTask.java. So the local split file was being written into dfs by mistake. This patch fixes that problem, makes the dfs client reject illegal names immediately, and adds a new static method to get the local file system.
        Owen O'Malley made changes -
        Attachment dos-dfs.patch [ 12347526 ]
        Owen O'Malley made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        +1, because http://issues.apache.org/jira/secure/attachment/12347526/dos-dfs.patch applied and successfully tested against trunk revision r488444.

        Show
        Hadoop QA added a comment - +1, because http://issues.apache.org/jira/secure/attachment/12347526/dos-dfs.patch applied and successfully tested against trunk revision r488444.
        Hide
        Doug Cutting added a comment -

        I just committed this. I made two minor changes. I used LocalFileSystem.NAME instead of new URI("file:///") to name the LocalFileSystem, and I replaced calls to FileSystem.getNamed("local",...) with FileSystem.getLocal(...).

        Thanks, Owen!

        Show
        Doug Cutting added a comment - I just committed this. I made two minor changes. I used LocalFileSystem.NAME instead of new URI("file:///") to name the LocalFileSystem, and I replaced calls to FileSystem.getNamed("local",...) with FileSystem.getLocal(...). Thanks, Owen!
        Doug Cutting made changes -
        Resolution Fixed [ 1 ]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Doug Cutting made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Owen O'Malley made changes -
        Component/s mapred [ 12310690 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        57m 19s 1 Owen O'Malley 20/Dec/06 00:30
        Patch Available Patch Available Resolved Resolved
        19h 24m 1 Doug Cutting 20/Dec/06 19:54
        Resolved Resolved Closed Closed
        16d 3h 28m 1 Doug Cutting 05/Jan/07 23:22

          People

          • Assignee:
            Owen O'Malley
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development