Pig
  1. Pig
  2. PIG-3752

Fix e2e Parallel test for Windows

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: e2e harness
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Parallel e2e test does not work for Windows (either activePerl or strawberryPerl). This is because IPC::Run module has limitation under Win32 (http://search.cpan.org/~toddr/IPC-Run-0.92/lib/IPC/Run.pm#Win32_LIMITATIONS). It will run into crash or deadlock. Fortunately cygwin perl does not use Win32 subsystem, and it does not have this issue. So we want to solve this by using cygwin perl.

        Activity

        Daniel Dai created issue -
        Hide
        Daniel Dai added a comment -

        Instruction for test environment:
        1. Install cygwin 1.7.28+ (earlier version has a bug), minimum core + perl, dev module
        2. Install perl module (Parallel::ForkManager, IPC::Run)
        To install it, open a cygwin terminal, use "cpan Parallel::ForkManager" etc to install.
        3. Need to put $CYGWIN_HOME/bin in PATH, after GnuWin32. Test will pick perl from cygwin, every other utilities still from GnuWin32.

        Show
        Daniel Dai added a comment - Instruction for test environment: 1. Install cygwin 1.7.28+ (earlier version has a bug), minimum core + perl, dev module 2. Install perl module (Parallel::ForkManager, IPC::Run) To install it, open a cygwin terminal, use "cpan Parallel::ForkManager" etc to install. 3. Need to put $CYGWIN_HOME/bin in PATH, after GnuWin32. Test will pick perl from cygwin, every other utilities still from GnuWin32.
        Daniel Dai made changes -
        Field Original Value New Value
        Attachment PIG-3752-1.patch [ 12627569 ]
        Hide
        Arpit Gupta added a comment -

        +1 we have been running tests in parallel on windows with this patch and have found no regressions in the framework.

        Show
        Arpit Gupta added a comment - +1 we have been running tests in parallel on windows with this patch and have found no regressions in the framework.
        Hide
        Rohini Palaniswamy added a comment -

        +1

        Show
        Rohini Palaniswamy added a comment - +1
        Hide
        Daniel Dai added a comment -

        Patch committed to both 0.13 branch and trunk. Thanks Arpit, Rohini for review!

        Show
        Daniel Dai added a comment - Patch committed to both 0.13 branch and trunk. Thanks Arpit, Rohini for review!
        Daniel Dai made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Daniel Dai made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        110d 10h 52m 1 Daniel Dai 28/May/14 19:23
        Resolved Resolved Closed Closed
        39d 23h 44m 1 Daniel Dai 07/Jul/14 19:07

          People

          • Assignee:
            Daniel Dai
            Reporter:
            Daniel Dai
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development