Avro
  1. Avro
  2. AVRO-912

Mapreduce tether test fails on Windows

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: java
    • Labels:
      None

      Description

      The problems are:
      1. The executable filename is passed around as a URL. Windows filenames are valid URLs.
      2. Typical windows user's home directory is c:\Documents and Settings\username. Maven puts the downloaded jar files under $HOME/.m2. So the classpath has several directories with space in their names. Splitting command line arguments using space generates invalid classpath.
      3. Hadoop's TaskLog.captureOutAndError() generates command line for unix systems using bash.

      1. AVRO-912.patch
        6 kB
        Thiruvalluvan M. G.

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        21h 31m 1 Thiruvalluvan M. G. 07/Oct/11 08:21
        Patch Available Patch Available Resolved Resolved
        45s 1 Thiruvalluvan M. G. 07/Oct/11 08:21
        Resolved Resolved Closed Closed
        25d 11h 14m 1 Doug Cutting 01/Nov/11 18:36
        Doug Cutting made changes -
        Component/s java [ 12312780 ]
        Doug Cutting made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Doug Cutting made changes -
        Fix Version/s 1.6.0 [ 12316198 ]
        Thiruvalluvan M. G. made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Thiruvalluvan M. G. added a comment -

        Committed revision 1179946.

        Thank you Doug for reviewing.

        Show
        Thiruvalluvan M. G. added a comment - Committed revision 1179946. Thank you Doug for reviewing.
        Thiruvalluvan M. G. made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Doug Cutting added a comment -

        +1 Changes look good to me and tests pass on Linux.

        Show
        Doug Cutting added a comment - +1 Changes look good to me and tests pass on Linux.
        Thiruvalluvan M. G. made changes -
        Field Original Value New Value
        Attachment AVRO-912.patch [ 12497970 ]
        Hide
        Thiruvalluvan M. G. added a comment -

        The attached patch addresses problems 1 and 2 by:
        1. Using File instead of URL for passing executable filename.
        2. Uses a List<String> instead of String to pass command arguments. While serializing the commands arguments into JobConf, it uses \n as delimiter.

        The patch does not address the problem 3. Java 6's ProcessBuilder does not have a mechanism to redirect output or error. (ProcessBuilder in Java 7 has a platform independent solution, though). But the current thing using bash works if the windows machine has cygwin.

        Show
        Thiruvalluvan M. G. added a comment - The attached patch addresses problems 1 and 2 by: 1. Using File instead of URL for passing executable filename. 2. Uses a List<String> instead of String to pass command arguments. While serializing the commands arguments into JobConf , it uses \n as delimiter. The patch does not address the problem 3. Java 6's ProcessBuilder does not have a mechanism to redirect output or error. ( ProcessBuilder in Java 7 has a platform independent solution, though). But the current thing using bash works if the windows machine has cygwin .
        Thiruvalluvan M. G. created issue -

          People

          • Assignee:
            Thiruvalluvan M. G.
            Reporter:
            Thiruvalluvan M. G.
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development