Avro
  1. Avro
  2. AVRO-338

ant test-tools does not work under Cygwin

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: java
    • Labels:
      None

      Description

      The target "test-tools" invokes a shell script using "exec" task. Since the shell scripts cannot be invoked directly by the Windows, it is required to have the "exec" task with executable="sh" and then pass the script name as an argument.

      This patch does just that.

        Activity

        Hide
        Thiruvalluvan M. G. added a comment -

        The interaction between Java and Cygwin tools are too broken. For example, by default Java installation sets JAVA_HOME=c:\Program Files\Java\.... The space in JAVA_HOME causes trouble in shell scripts. Another example is that Cygwin's "find" needs Cygwin paths, but Java requires Windows paths, so the output of find cannot be passed to Java. One way to resolve the path problems is to use relative paths.

        These require too many fixes.

        Show
        Thiruvalluvan M. G. added a comment - The interaction between Java and Cygwin tools are too broken. For example, by default Java installation sets JAVA_HOME=c:\Program Files\Java\.... The space in JAVA_HOME causes trouble in shell scripts. Another example is that Cygwin's "find" needs Cygwin paths, but Java requires Windows paths, so the output of find cannot be passed to Java. One way to resolve the path problems is to use relative paths. These require too many fixes.
        Hide
        Patrick Hunt added a comment -

        Thiru you might want to reconsider - check out cygpath, it's used for handling this issue: http://www.cygwin.com/cygwin-ug-net/using-utils.html#cygpath

        You might also look at the zookeeper shell scripts (*.sh in bin directory), the current zk trunk has patches to make things work
        with cygwin, std unix shell, and macosx.

        Show
        Patrick Hunt added a comment - Thiru you might want to reconsider - check out cygpath, it's used for handling this issue: http://www.cygwin.com/cygwin-ug-net/using-utils.html#cygpath You might also look at the zookeeper shell scripts (*.sh in bin directory), the current zk trunk has patches to make things work with cygwin, std unix shell, and macosx.
        Hide
        Doug Cutting added a comment -

        It would be great if the build works on cygwin, but this has proven tough to maintain for Hadoop, since most developers build only on Linux and/or MacOS.

        Show
        Doug Cutting added a comment - It would be great if the build works on cygwin, but this has proven tough to maintain for Hadoop, since most developers build only on Linux and/or MacOS.
        Hide
        Thiruvalluvan M. G. added a comment -

        It looks like )Cygwin + Shell-script + Java) is a difficult combination. We have just one shell script test_tools.sh in the Java part of Avro. Is there a specific reason why it is a shell script. What happens if we replace its functionality with ant?

        Show
        Thiruvalluvan M. G. added a comment - It looks like )Cygwin + Shell-script + Java) is a difficult combination. We have just one shell script test_tools.sh in the Java part of Avro. Is there a specific reason why it is a shell script. What happens if we replace its functionality with ant?
        Hide
        Philip Zeyliger added a comment -

        test_tools.sh is a shell script because I hate writing shell scripts in ant . (The script invokes avro-tools with various arguments, and uses diff/cmp to make sure it gets expected results.) For what it does, I'd prefer python to ant, but if I won't get in the way if you'd prefer ant.

        Show
        Philip Zeyliger added a comment - test_tools.sh is a shell script because I hate writing shell scripts in ant . (The script invokes avro-tools with various arguments, and uses diff/cmp to make sure it gets expected results.) For what it does, I'd prefer python to ant, but if I won't get in the way if you'd prefer ant.
        Hide
        Patrick Hunt added a comment -

        I hate writing shell scripts in general. Maintaining the ones we have for ZooKeeper is an annoying job. If I could
        do over again I'd probably use ant whenever possible. I did something similar to this when I created the
        avro quickstart example.

        Every so often I see posts like this one http://perlmeister.com/shell_yapc2008.html from ppl I respect (like Mike).

        ie "why not use X instead of shell scripts" etc... where X is often python/perl.

        any thoughts on this? I love python and it's available for pretty much every platform right? Why not switch to say
        python for stuff like this?

        Show
        Patrick Hunt added a comment - I hate writing shell scripts in general. Maintaining the ones we have for ZooKeeper is an annoying job. If I could do over again I'd probably use ant whenever possible. I did something similar to this when I created the avro quickstart example. Every so often I see posts like this one http://perlmeister.com/shell_yapc2008.html from ppl I respect (like Mike). ie "why not use X instead of shell scripts" etc... where X is often python/perl. any thoughts on this? I love python and it's available for pretty much every platform right? Why not switch to say python for stuff like this?
        Hide
        Philip Zeyliger added a comment -

        I'm totally cool with switching to python. (It's available everywhere, though sometimes which version varies, and depending on things not in the stdlib is a pain.) I find that for short things, I don't mind shell, and I try to do as little XML as possible

        I suspect this is all preference-based.

        Show
        Philip Zeyliger added a comment - I'm totally cool with switching to python. (It's available everywhere, though sometimes which version varies, and depending on things not in the stdlib is a pain.) I find that for short things, I don't mind shell, and I try to do as little XML as possible I suspect this is all preference-based.
        Hide
        Thiruvalluvan M. G. added a comment -

        Yes, in general, Python will be a much better alternative to shell.

        But the specific problem here is that Java which is completely unaware of Cygwin, works with Windows paths and any Cygwin based scripting engine will understand only Cygwin paths and so they don't interact well. Another minor irritation is that under Windows Java's classpath separator is semicolon and under Unix, it is colon. There are the following options:

        • Use relative paths always. Then the only difference between Cygwin and Windows paths is forward- versus back-slash. Windows API is fortunately okay with forward slash.
        • Use cygpath Cygwin's path translation tool. This will lead to some kind "if (cygwin)" constructs in the scripts.
        • Use ant. Since ant is a Java utility, it does the right thing for Windows.

        None of these is a pretty solution. I'm leaning toward ant because, we already need ant (no new dependency) and we have (at least for now) just one small script to translate.

        Show
        Thiruvalluvan M. G. added a comment - Yes, in general, Python will be a much better alternative to shell. But the specific problem here is that Java which is completely unaware of Cygwin, works with Windows paths and any Cygwin based scripting engine will understand only Cygwin paths and so they don't interact well. Another minor irritation is that under Windows Java's classpath separator is semicolon and under Unix, it is colon. There are the following options: Use relative paths always. Then the only difference between Cygwin and Windows paths is forward- versus back-slash. Windows API is fortunately okay with forward slash. Use cygpath Cygwin's path translation tool. This will lead to some kind "if (cygwin)" constructs in the scripts. Use ant. Since ant is a Java utility, it does the right thing for Windows. None of these is a pretty solution. I'm leaning toward ant because, we already need ant (no new dependency) and we have (at least for now) just one small script to translate.

          People

          • Assignee:
            Unassigned
            Reporter:
            Thiruvalluvan M. G.
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development