Pig
  1. Pig
  2. PIG-3248

Upgrade hadoop-2.0.0-alpha to hadoop-2.0.3-alpha

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Activity

      Hide
      Daniel Dai added a comment -

      All unit tests pass for both hadoop1/hadoop2.

      Show
      Daniel Dai added a comment - All unit tests pass for both hadoop1/hadoop2.
      Hide
      Thejas M Nair added a comment -

      Daniel, Is the following change relevant for the hadoop 2 version upgrade ?

      ===================================================================
      --- test/org/apache/pig/test/TestPigSplit.java	(revision 1456106)
      +++ test/org/apache/pig/test/TestPigSplit.java	(working copy)
      @@ -108,7 +108,7 @@
               createInput(new String[] { "0\ta" });
       
               pigServer.registerQuery("a = load '" + inputFileName + "';");
      -        for (int i = 0; i < 500; i++) {
      +        for (int i = 0; i < 200; i++) {
                   pigServer.registerQuery("a = filter a by $0 == '1';");
               }
               Iterator<Tuple> iter = pigServer.openIterator("a");
      
      Show
      Thejas M Nair added a comment - Daniel, Is the following change relevant for the hadoop 2 version upgrade ? =================================================================== --- test/org/apache/pig/test/TestPigSplit.java (revision 1456106) +++ test/org/apache/pig/test/TestPigSplit.java (working copy) @@ -108,7 +108,7 @@ createInput( new String [] { "0\ta" }); pigServer.registerQuery( "a = load '" + inputFileName + "';" ); - for ( int i = 0; i < 500; i++) { + for ( int i = 0; i < 200; i++) { pigServer.registerQuery( "a = filter a by $0 == '1';" ); } Iterator<Tuple> iter = pigServer.openIterator( "a" );
      Hide
      Daniel Dai added a comment -

      This is actually a different issue. TestPigSplit always fail on my test machine due to stack overflow. It is Ok if you want me open a separate Jira for this issue.

      Show
      Daniel Dai added a comment - This is actually a different issue. TestPigSplit always fail on my test machine due to stack overflow. It is Ok if you want me open a separate Jira for this issue.
      Hide
      Thejas M Nair added a comment -

      This is actually a different issue. TestPigSplit always fail on my test machine due to stack overflow. It is Ok if you want me open a separate Jira for this issue.

      As it is a very minor change to a test case, i think its fine to include it here. 200 is large enough for number of statements, so i think so many levels instead of 500 should be ok, if it is causing issues with some jvm configs.

      Show
      Thejas M Nair added a comment - This is actually a different issue. TestPigSplit always fail on my test machine due to stack overflow. It is Ok if you want me open a separate Jira for this issue. As it is a very minor change to a test case, i think its fine to include it here. 200 is large enough for number of statements, so i think so many levels instead of 500 should be ok, if it is causing issues with some jvm configs.
      Hide
      Daniel Dai added a comment -

      Yes, otherwise I need to add "-Xss" option to increase the stack size.

      Show
      Daniel Dai added a comment - Yes, otherwise I need to add "-Xss" option to increase the stack size.
      Hide
      Thejas M Nair added a comment -

      +1

      Show
      Thejas M Nair added a comment - +1
      Hide
      Daniel Dai added a comment -

      Committed to trunk.

      Show
      Daniel Dai added a comment - Committed to trunk.

        People

        • Assignee:
          Daniel Dai
          Reporter:
          Daniel Dai
        • Votes:
          0 Vote for this issue
          Watchers:
          2 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development