Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Resolves the test failure by modifying the test to base it on spill counters rather than on bytes read/written. It also introduces a new configuration parameter "mapred.job.shuffle.input.buffer.percent" to provide finer grained control on the memory limit to be used during shuffle.
      Show
      Resolves the test failure by modifying the test to base it on spill counters rather than on bytes read/written. It also introduces a new configuration parameter "mapred.job.shuffle.input.buffer.percent" to provide finer grained control on the memory limit to be used during shuffle.

      Description

      Testcase: testReduceFromMem took 23.625 sec
      	FAILED
      Non-zero read from local: 83
      junit.framework.AssertionFailedError: Non-zero read from local: 83
      	at org.apache.hadoop.mapred.TestReduceFetch.testReduceFromMem(TestReduceFetch.java:289)
      	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      	at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      	at junit.extensions.TestSetup.run(TestSetup.java:27)
      

      Ran TestReduceFetch a few times on a clean trunk. It failed consistently.

      1. M433-1y20.patch
        5 kB
        Chris Douglas
      2. M433-1v20.patch
        5 kB
        Chris Douglas
      3. M433-v20.another.patch
        5 kB
        Devaraj Das
      4. M433-0v20.patch
        5 kB
        Chris Douglas
      5. 6029-2.patch
        6 kB
        Chris Douglas
      6. 6029-1.patch
        5 kB
        Chris Douglas
      7. 6029-0.patch
        5 kB
        Chris Douglas
      8. testReduceFromMem.txt
        287 kB
        Tsz Wo Nicholas Sze
      9. TEST-org.apache.hadoop.mapred.TestReduceFetch.txt
        288 kB
        Tsz Wo Nicholas Sze
      10. TEST-org.apache.hadoop.mapred.TestReduceFetch.txt
        302 kB
        Devaraj Das
      11. FAILING-PARTIALMEM-TEST-org.apache.hadoop.mapred.TestReduceFetch.txt
        288 kB
        Devaraj Das

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Chris Douglas
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development