Avro
  1. Avro
  2. AVRO-912

Mapreduce tether test fails on Windows

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: java
    • Labels:
      None

      Description

      The problems are:
      1. The executable filename is passed around as a URL. Windows filenames are valid URLs.
      2. Typical windows user's home directory is c:\Documents and Settings\username. Maven puts the downloaded jar files under $HOME/.m2. So the classpath has several directories with space in their names. Splitting command line arguments using space generates invalid classpath.
      3. Hadoop's TaskLog.captureOutAndError() generates command line for unix systems using bash.

      1. AVRO-912.patch
        6 kB
        Thiruvalluvan M. G.

        Activity

        Hide
        Thiruvalluvan M. G. added a comment -

        The attached patch addresses problems 1 and 2 by:
        1. Using File instead of URL for passing executable filename.
        2. Uses a List<String> instead of String to pass command arguments. While serializing the commands arguments into JobConf, it uses \n as delimiter.

        The patch does not address the problem 3. Java 6's ProcessBuilder does not have a mechanism to redirect output or error. (ProcessBuilder in Java 7 has a platform independent solution, though). But the current thing using bash works if the windows machine has cygwin.

        Show
        Thiruvalluvan M. G. added a comment - The attached patch addresses problems 1 and 2 by: 1. Using File instead of URL for passing executable filename. 2. Uses a List<String> instead of String to pass command arguments. While serializing the commands arguments into JobConf , it uses \n as delimiter. The patch does not address the problem 3. Java 6's ProcessBuilder does not have a mechanism to redirect output or error. ( ProcessBuilder in Java 7 has a platform independent solution, though). But the current thing using bash works if the windows machine has cygwin .
        Hide
        Doug Cutting added a comment -

        +1 Changes look good to me and tests pass on Linux.

        Show
        Doug Cutting added a comment - +1 Changes look good to me and tests pass on Linux.
        Hide
        Thiruvalluvan M. G. added a comment -

        Committed revision 1179946.

        Thank you Doug for reviewing.

        Show
        Thiruvalluvan M. G. added a comment - Committed revision 1179946. Thank you Doug for reviewing.

          People

          • Assignee:
            Thiruvalluvan M. G.
            Reporter:
            Thiruvalluvan M. G.
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development