Pig
  1. Pig
  2. PIG-2405

svn tags/release-0.9.1: some unit test case failed with open JDK

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.1
    • Fix Version/s: 0.11, 0.12.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      ant-1.8.2
      open jdk: 1.6

      Description

      [junit] Test org.apache.pig.test.TestDataModel FAILED
      Testcase: testTupleToString took 0.004 sec
      FAILED
      toString expected:<...ad a little lamb)},[hello#world,goodbye#all],42,5000000000,3.14...> but was:<...ad a little lamb)},[goodbye#all,hello#world],42,5000000000,3.14...>
      junit.framework.ComparisonFailure: toString expected:<...ad a little lamb)},[hello#world,goodbye#all],42,5000000000,3.14...> but was:<...ad a little lamb)},[goodbye#all,hello#world],42,5000000000,3.14...>
      at org.apache.pig.test.TestDataModel.testTupleToString(TestDataModel.java:269

      [junit] Test org.apache.pig.test.TestHBaseStorage FAILED
      Tests run: 18, Failures: 0, Errors: 12, Time elapsed: 188.612 sec

      Testcase: testHeterogeneousScans took 0.018 sec
      Caused an ERROR
      java.io.FileNotFoundException: /root/pigtest/conf/hadoop-site.xml (Too many open files)
      java.lang.RuntimeException: java.io.FileNotFoundException: /root/pigtest/conf/hadoop-site.xml (Too many open files)
      at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1162)
      at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1035)
      at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:980)
      at org.apache.hadoop.conf.Configuration.get(Configuration.java:436)
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:271)
      at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:155)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:167)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:130)
      at org.apache.pig.test.TestHBaseStorage.prepareTable(TestHBaseStorage.java:809)
      at org.apache.pig.test.TestHBaseStorage.testHeterogeneousScans(TestHBaseStorage.java:741)
      Caused by: java.io.FileNotFoundException: /root/pigtest/conf/hadoop-site.xml (Too many open files)
      at java.io.FileInputStream.<init>(FileInputStream.java:112)
      at java.io.FileInputStream.<init>(FileInputStream.java:72)
      at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
      at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
      at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
      at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
      at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
      at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
      at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
      at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
      at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
      at javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
      at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1079)

      Caused an ERROR
      Could not resolve the DNS name of hostname:39611
      java.lang.IllegalArgumentException: Could not resolve the DNS name of hostname:39611
      at org.apache.hadoop.hbase.HServerAddress.checkBindAddressCanBeResolved(HServerAddress.java:105)
      at org.apache.hadoop.hbase.HServerAddress.<init>(HServerAddress.java:66)
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:755)
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:590)
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
      at org.apache.pig.test.TestHBaseStorage.deleteAllRows(TestHBaseStorage.java:120)
      at org.apache.pig.test.TestHBaseStorage.tearDown(TestHBaseStorage.java:112)

      [junit] Test org.apache.pig.test.TestMRCompiler FAILED
      Testcase: testSortUDF1 took 0.045 sec
      FAILED
      null expected:<...---MapReduce(20,SUM,[COUNT,TestMRCompiler$WeirdComparator]) - -18:

      ...> but was:<...---MapReduce(20,SUM,[TestMRCompiler$WeirdComparator,COUNT]) - -18:
      ...>
      junit.framework.ComparisonFailure: null expected:<...---MapReduce(20,SUM,[COUNT,TestMRCompiler$WeirdComparator]) - -18:
      ...> but was:<...---MapReduce(20,SUM,[TestMRCompiler$WeirdComparator,COUNT]) - -18:
      ...>
      at org.apache.pig.test.TestMRCompiler.run(TestMRCompiler.java:1080)
      at org.apache.pig.test.TestMRCompiler.testSortUDF1(TestMRCompiler.java:791

      [junit] Test org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor FAILED
      (1)
      Testcase: testSimplePlan took 0.675 sec
      FAILED
      expected:<class org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject> but was:<class org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.ConstantExpression>
      junit.framework.AssertionFailedError: expected:<class org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject> but was:<class org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.ConstantExpression>
      at org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor.testSimplePlan(TestNewPlanLogToPhyTranslationVisitor.java:127)
      (2)
      Testcase: testJoinPlan took 0.064 sec
      FAILED
      expected:<0> but was:<1>
      junit.framework.AssertionFailedError: expected:<0> but was:<1>
      at org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor.testJoinPlan(TestNewPlanLogToPhyTranslationVisitor.java:158)
      (3)
      Testcase: testMultiStore took 0.041 sec
      FAILED
      expected:<0> but was:<1>
      junit.framework.AssertionFailedError: expected:<0> but was:<1>
      at org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor.testMultiStore(TestNewPlanLogToPhyTranslationVisitor.java:239)

      [junit] Test org.apache.pig.test.TestPruneColumn FAILED
      (1)
      Testcase: testMapKey2 took 7.1 sec
      FAILED
      null
      junit.framework.AssertionFailedError: null
      at org.apache.pig.test.TestPruneColumn.testMapKey2(TestPruneColumn.java:1206)
      (2)
      Testcase: testMapKey3 took 7.088 sec
      FAILED
      null
      junit.framework.AssertionFailedError: null
      at org.apache.pig.test.TestPruneColumn.testMapKey3(TestPruneColumn.java:1222)
      (3)
      Testcase: testMapKeyInSplit1 took 7.1 sec
      FAILED
      null
      junit.framework.AssertionFailedError: null
      at org.apache.pig.test.TestPruneColumn.testMapKeyInSplit1(TestPruneColumn.java:1296)
      (4)
      Testcase: testSharedSchemaObject took 7.1 sec
      FAILED
      null
      junit.framework.AssertionFailedError: null
      at org.apache.pig.test.TestPruneColumn.testSharedSchemaObject(TestPruneColumn.java:1619)

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          281d 7h 28m 1 Richard Ding 13/Sep/12 21:30
          Patch Available Patch Available Resolved Resolved
          55d 8h 18m 1 Cheolsoo Park 08/Nov/12 04:48
          Resolved Resolved Closed Closed
          106d 5m 1 Bill Graham 22/Feb/13 04:53
          Bill Graham made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Cheolsoo Park made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.12 [ 12323380 ]
          Resolution Fixed [ 1 ]
          Hide
          Cheolsoo Park added a comment -

          Committed to trunk/0.11. Thank you for your contribution, Fagnfang!

          Show
          Cheolsoo Park added a comment - Committed to trunk/0.11. Thank you for your contribution, Fagnfang!
          Hide
          fang fang chen added a comment -

          Hi Cheolsoo,

          Sorry for missing fix. Updated the patch based on your comments.
          Thanks for your comments.

          Thanks

          Show
          fang fang chen added a comment - Hi Cheolsoo, Sorry for missing fix. Updated the patch based on your comments. Thanks for your comments. Thanks
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552602 ]
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552590 ]
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552590 ]
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552261 ]
          Hide
          Cheolsoo Park added a comment -

          Hi Fangfang,

          I was running tests with your new patch in the RB to commit it, but I found that TestPruneColumn.testStream2 fails. I believe that you omitted the following change in your new patch.

          TestPruneColumn.java
          -        assertTrue(checkLogFileMessage(new String[]{"Map key required for event_serve: $0->[key4, key3]", 
          -                "Map key required for cm_data_raw: $0->[key4, key3, key5]"}));
          +        assertTrue(checkLogFileMessage(new String[]{"Map key required for event_serve: $0->[key3, key4]", 
          +                "Map key required for cm_data_raw: $0->[key3, key4, key5]"}));
          

          Can you please upload a new patch to this JIRA? I will commit it as soon as you upload a new patch.

          I also have a super minor comment. This is just a suggestion, so I won't insist. I found that you replaced assertEquals with assertTrue:

          TestPruneColumn.java
          -        assertEquals("([2#1,1#1])", t.toString());
          +        assertTrue(TestHelper.sortString("\\[(.*)\\]", t.toString(), ",")
          +                .equals("([1#1, 2#1])"));
          

          Can you please not change assertEquals to assertTrue? We made a good amount of effort to modernize test code in PIG-3006, and this was one of patterns that we fixed.

          Thanks for your patience!

          Show
          Cheolsoo Park added a comment - Hi Fangfang, I was running tests with your new patch in the RB to commit it, but I found that TestPruneColumn.testStream2 fails. I believe that you omitted the following change in your new patch. TestPruneColumn.java - assertTrue(checkLogFileMessage( new String []{ "Map key required for event_serve: $0->[key4, key3]" , - "Map key required for cm_data_raw: $0->[key4, key3, key5]" })); + assertTrue(checkLogFileMessage( new String []{ "Map key required for event_serve: $0->[key3, key4]" , + "Map key required for cm_data_raw: $0->[key3, key4, key5]" })); Can you please upload a new patch to this JIRA? I will commit it as soon as you upload a new patch. I also have a super minor comment. This is just a suggestion, so I won't insist. I found that you replaced assertEquals with assertTrue : TestPruneColumn.java - assertEquals( "([2#1,1#1])" , t.toString()); + assertTrue(TestHelper.sortString( "\\[(.*)\\]" , t.toString(), "," ) + .equals( "([1#1, 2#1])" )); Can you please not change assertEquals to assertTrue? We made a good amount of effort to modernize test code in PIG-3006 , and this was one of patterns that we fixed. Thanks for your patience!
          Hide
          Rohini Palaniswamy added a comment -

          +1 from me. Actually did not realize that TestMRCompiler case was more complicated until I saw Cheolsoo's comment that he does not have a better suggestion. After taking a deeper look, I couldn't think of one too without changing src code. This one is a pretty good solution. Thanks for fixing this Fang. I will let Cheolsoo take a final look and commit because he has been working with you so far.

          Show
          Rohini Palaniswamy added a comment - +1 from me. Actually did not realize that TestMRCompiler case was more complicated until I saw Cheolsoo's comment that he does not have a better suggestion. After taking a deeper look, I couldn't think of one too without changing src code. This one is a pretty good solution. Thanks for fixing this Fang. I will let Cheolsoo take a final look and commit because he has been working with you so far.
          Richard Ding made changes -
          Fix Version/s 0.11 [ 12318878 ]
          Hide
          fang fang chen added a comment -

          Loaded the patch to RB:
          https://reviews.apache.org/r/7898/

          Show
          fang fang chen added a comment - Loaded the patch to RB: https://reviews.apache.org/r/7898/
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552261 ]
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552260 ]
          Hide
          fang fang chen added a comment -

          Hi Rohini and Cheolsoo,

          TestMRCompiler is fixed by sorting UDFs in this class. Remove all the LinkedHash... change set now. This patch works for both 0.11 and trunk branch.
          Current change set should be OK. Please help review.

          Thanks

          Show
          fang fang chen added a comment - Hi Rohini and Cheolsoo, TestMRCompiler is fixed by sorting UDFs in this class. Remove all the LinkedHash... change set now. This patch works for both 0.11 and trunk branch. Current change set should be OK. Please help review. Thanks
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552260 ]
          fang fang chen made changes -
          Attachment PIG-2405-0.11.patch [ 12552246 ]
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552247 ]
          Hide
          fang fang chen added a comment -

          Update patches: remove tabs.

          Show
          fang fang chen added a comment - Update patches: remove tabs.
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552247 ]
          fang fang chen made changes -
          Attachment PIG-2405-0.11.patch [ 12552246 ]
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552225 ]
          fang fang chen made changes -
          Attachment PIG-2405-0.11.patch [ 12552224 ]
          Hide
          fang fang chen added a comment -

          Yes, you are right. We can not sort the output
          Maybe we can generate two kinds of golden files. Any concern?

          Show
          fang fang chen added a comment - Yes, you are right. We can not sort the output Maybe we can generate two kinds of golden files. Any concern?
          Hide
          fang fang chen added a comment -

          Agree with you and Rohini that we should avoid modifying core code.
          Will generate new patches based via sorting outputs.

          Show
          fang fang chen added a comment - Agree with you and Rohini that we should avoid modifying core code. Will generate new patches based via sorting outputs.
          Hide
          Cheolsoo Park added a comment -
          • RB is review board: https://reviews.apache.org
          • I still see tabs in your patch. Please use 4 spaces instead.
          • OK, let's sort log messages then.

          But can you please comment on Rohini's comment? She doesn't seem to agree with replacing HashMap with LinkedHashMap in MapReduceOper.java and OperatorPlan.java. I understand that you can't sort the result to fix TestMRCompiler. But I also agree with Rohini that we should avoid modifying code if possible. Unfortunately, I don't really have a better idea.

          Show
          Cheolsoo Park added a comment - RB is review board: https://reviews.apache.org I still see tabs in your patch. Please use 4 spaces instead. OK, let's sort log messages then. But can you please comment on Rohini's comment? She doesn't seem to agree with replacing HashMap with LinkedHashMap in MapReduceOper.java and OperatorPlan.java . I understand that you can't sort the result to fix TestMRCompiler. But I also agree with Rohini that we should avoid modifying code if possible. Unfortunately, I don't really have a better idea.
          Hide
          fang fang chen added a comment -

          BTW, following is the test-patch result, and the failure is not caused by the patches. There are [javadoc] 38 warnings in original trunk branch. I will open a JIRA for this.
          [exec] -1 overall.
          [exec]
          [exec] +1 @author. The patch does not contain any @author tags.
          [exec]
          [exec] +1 tests included. The patch appears to include 12 new or modified tests.
          [exec]
          [exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages.
          [exec]
          [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
          [exec]
          [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
          [exec]
          [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.

          Show
          fang fang chen added a comment - BTW, following is the test-patch result, and the failure is not caused by the patches. There are [javadoc] 38 warnings in original trunk branch. I will open a JIRA for this. [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 12 new or modified tests. [exec] [exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
          Hide
          fang fang chen added a comment -

          Hi Cheolsoo,

          Thanks for your review.

          • You are right, the sortLogMessages() method is sorting the output with some spacial order.
          • Actually, I though about to fix by using reg. exp. as which is much more simpler. The reason of not use it is we need to do the same steps to all related test cases(current 4 tests). In future, if we need to add new tests, the same action would also be needed.
          • Besides, the reg. exp. will be too complex when there are too many items in [], like:
            "Map key required for raw: $0->[cm_serve_id, cm_serve_timestamp_ms, p_url, source, type]"

          So I insist the original fix. Do you agree? Please let me know if I miss anything. Thanks.
          I regenerated the patch files by following "how to contribute" steps, and attach them at "Attachments". Could you please tell what is "RB"? Please help load them to RB and let me know if modifications are needed. Thanks

          Show
          fang fang chen added a comment - Hi Cheolsoo, Thanks for your review. You are right, the sortLogMessages() method is sorting the output with some spacial order. Actually, I though about to fix by using reg. exp. as which is much more simpler. The reason of not use it is we need to do the same steps to all related test cases(current 4 tests). In future, if we need to add new tests, the same action would also be needed. Besides, the reg. exp. will be too complex when there are too many items in [], like: "Map key required for raw: $0-> [cm_serve_id, cm_serve_timestamp_ms, p_url, source, type] " So I insist the original fix. Do you agree? Please let me know if I miss anything. Thanks. I regenerated the patch files by following "how to contribute" steps, and attach them at "Attachments". Could you please tell what is "RB"? Please help load them to RB and let me know if modifications are needed. Thanks
          fang fang chen made changes -
          Attachment PIG-2405-trunk.patch [ 12552225 ]
          Hide
          fang fang chen added a comment -

          For trunk branch.

          Show
          fang fang chen added a comment - For trunk branch.
          fang fang chen made changes -
          Attachment PIG-2405-0.11.patch [ 12552224 ]
          Hide
          fang fang chen added a comment -

          For 0.11 branch.

          Show
          fang fang chen added a comment - For 0.11 branch.
          fang fang chen made changes -
          Attachment PIG-2405-0.11.patch [ 12552055 ]
          fang fang chen made changes -
          Attachment 2405_2.patch [ 12544961 ]
          fang fang chen made changes -
          Attachment 2405_1.patch [ 12544960 ]
          Hide
          Cheolsoo Park added a comment -

          Thanks Rohini for your comment. Sure, agreed. But I don't really have a better suggestion on how to fix TestMRCompiler otherwise. I am open to better suggestions.

          Show
          Cheolsoo Park added a comment - Thanks Rohini for your comment. Sure, agreed. But I don't really have a better suggestion on how to fix TestMRCompiler otherwise. I am open to better suggestions.
          Hide
          Rohini Palaniswamy added a comment -

          I think we should avoid changing to LinkedHashMap. Recently, for a different project for test result comparison where the order was required we moved from LinkedHashMap to TreeMap because the order of insertion into LinkedHashMap was different in jdk7 and LinkedHashMap did not work because of that to compare golden files. That might not be the case here because we insert iterating over an array. But would still prefer sorting the test results when comparing instead of changing code to use LinkedHashMap or TreeMap.

          Show
          Rohini Palaniswamy added a comment - I think we should avoid changing to LinkedHashMap. Recently, for a different project for test result comparison where the order was required we moved from LinkedHashMap to TreeMap because the order of insertion into LinkedHashMap was different in jdk7 and LinkedHashMap did not work because of that to compare golden files. That might not be the case here because we insert iterating over an array. But would still prefer sorting the test results when comparing instead of changing code to use LinkedHashMap or TreeMap.
          Hide
          Cheolsoo Park added a comment -

          Hi Fangfang,

          Thank for the clarification. Your changes seem reasonable, but I have a few comments as below.

          • Please remove tabs.
          • Please use Unix newline chars (i.e. no CRs).
          • Regarding TestPruneColumn, can't we use reg. exp. instead of sorting log messages? If I am not mistaken, the sortLogMessages() method is added to sort entries inside the "[ ]". But the same can be done by reg. exp. For example, the following pattern will match all [key1,key2], [key2,key1], [key1, key2], and [key2, key1]. I think that this is simpler. Do you agree?
            ".*\\[((key2,[ ]?key1)|(key1,[ ]?key2))\\]"
            
          • Regarding TestMRCompiler, I would prefer to fix the test rather than changing the code like Rohini commented in PIG-2908. But I don't have a better suggestion, so I won't disagree with your solution.

            Fang has solved it by changing it to a LinkedHashMap in MapReduceOper.java and OperatorPlan.java. I have just modified tests to sort the entrySet results and then assert. If LinkedHashMap approach is preferred, I can remove TestNewPlanLogToPhyTranslationVisitor from this patch. But in general, would prefer not changing code for tests to work.

          Can you please update the patch and upload it to the RB?

          Thanks!

          Show
          Cheolsoo Park added a comment - Hi Fangfang, Thank for the clarification. Your changes seem reasonable, but I have a few comments as below. Please remove tabs. Please use Unix newline chars (i.e. no CRs). Regarding TestPruneColumn, can't we use reg. exp. instead of sorting log messages? If I am not mistaken, the sortLogMessages() method is added to sort entries inside the "[ ]". But the same can be done by reg. exp. For example, the following pattern will match all [key1,key2] , [key2,key1] , [key1, key2] , and [key2, key1] . I think that this is simpler. Do you agree? ".*\\[((key2,[ ]?key1)|(key1,[ ]?key2))\\]" Regarding TestMRCompiler, I would prefer to fix the test rather than changing the code like Rohini commented in PIG-2908 . But I don't have a better suggestion, so I won't disagree with your solution. Fang has solved it by changing it to a LinkedHashMap in MapReduceOper.java and OperatorPlan.java. I have just modified tests to sort the entrySet results and then assert. If LinkedHashMap approach is preferred, I can remove TestNewPlanLogToPhyTranslationVisitor from this patch. But in general, would prefer not changing code for tests to work. Can you please update the patch and upload it to the RB? Thanks!
          Hide
          fang fang chen added a comment -

          Hi Cheolsoo Park,

          With current trunk/0.11 branch, I still encounter output order problem to TestDataModel, TestMRCompiler, and TestPruneColumn with OpenSource JDK, and did not with TestNewPlanLogToPhyTranslationVisitor. Seems this UT has been modified a lot.

          Thanks

          Show
          fang fang chen added a comment - Hi Cheolsoo Park, With current trunk/0.11 branch, I still encounter output order problem to TestDataModel, TestMRCompiler, and TestPruneColumn with OpenSource JDK, and did not with TestNewPlanLogToPhyTranslationVisitor. Seems this UT has been modified a lot. Thanks
          Hide
          Cheolsoo Park added a comment -

          Hi Fangfang,

          I ran the test cases that you mentioned (TestDataModel, TestNewPlanLogToPhyTranslationVisitor, TestMRCompiler, and TestPruneColumn) with trunk using OpenJDK 1.6. But I cannot reproduce the failures. Can you verify that you still see these failures with the current trunk, or is there anything that I have to do to reproduce them?

          I tested with

          • revision 1405587.
          • java version "1.6.0_24"
            OpenJDK Runtime Environment (IcedTea6 1.11.5) (6b24-1.11.5-0ubuntu1~12.04.1)
            OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

          Thanks!

          Show
          Cheolsoo Park added a comment - Hi Fangfang, I ran the test cases that you mentioned (TestDataModel, TestNewPlanLogToPhyTranslationVisitor, TestMRCompiler, and TestPruneColumn) with trunk using OpenJDK 1.6. But I cannot reproduce the failures. Can you verify that you still see these failures with the current trunk, or is there anything that I have to do to reproduce them? I tested with revision 1405587 . java version "1.6.0_24" OpenJDK Runtime Environment (IcedTea6 1.11.5) (6b24-1.11.5-0ubuntu1~12.04.1) OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode) Thanks!
          fang fang chen made changes -
          Attachment PIG-2405-0.11.patch [ 12552055 ]
          Hide
          fang fang chen added a comment -

          This patch is for branch 0.11.

          Show
          fang fang chen added a comment - This patch is for branch 0.11.
          Hide
          Leonardo Rangel Augusto added a comment -

          in TestDataModel, the underlying problem is using an order-independent structure in an order-dependent test. What about keeping it simple and removing the HashMap from the Tuple, instead of replacing it with a LinkedHashMap?

          Show
          Leonardo Rangel Augusto added a comment - in TestDataModel, the underlying problem is using an order-independent structure in an order-dependent test. What about keeping it simple and removing the HashMap from the Tuple, instead of replacing it with a LinkedHashMap?
          Rohini Palaniswamy made changes -
          Link This issue relates to PIG-2908 [ PIG-2908 ]
          Richard Ding made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          fang fang chen added a comment -

          Patch files are based on pig-trunk.

          Show
          fang fang chen added a comment - Patch files are based on pig-trunk.
          fang fang chen made changes -
          Attachment 2405_1.patch [ 12544960 ]
          Attachment 2405_2.patch [ 12544961 ]
          Hide
          fang fang chen added a comment -

          Current all UT passed.
          2405_1.patch is for TestDataModel, TestNewPlanLogToPhyTranslationVisitor, TestMRCompiler
          2405_2.patch is for TestPruneColumn

          Show
          fang fang chen added a comment - Current all UT passed. 2405_1.patch is for TestDataModel, TestNewPlanLogToPhyTranslationVisitor, TestMRCompiler 2405_2.patch is for TestPruneColumn
          Hide
          fang fang chen added a comment -

          Will try my best to fix this when available.

          Show
          fang fang chen added a comment - Will try my best to fix this when available.
          Hide
          Daniel Dai added a comment -

          I am sure this is the cause for some unit test failures for OpenJDK and Sun JDK 1.7. Would you like to submit a patch?

          Show
          Daniel Dai added a comment - I am sure this is the cause for some unit test failures for OpenJDK and Sun JDK 1.7. Would you like to submit a patch?
          Hide
          fang fang chen added a comment -

          The left failures are caused by HashMap. The outputs in open source JDK are also correct.

          Show
          fang fang chen added a comment - The left failures are caused by HashMap. The outputs in open source JDK are also correct.
          Hide
          fang fang chen added a comment -

          Before testing pig UT, we should increase "max user processes" and "open files" variables by "ulimit". This is workaround for "Too many files" error.

          Show
          fang fang chen added a comment - Before testing pig UT, we should increase "max user processes" and "open files" variables by "ulimit". This is workaround for "Too many files" error.
          Daniel Dai made changes -
          Field Original Value New Value
          Assignee fang fang chen [ fang fang chen ]
          fang fang chen created issue -

            People

            • Assignee:
              fang fang chen
              Reporter:
              fang fang chen
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development