Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Resolves the test failure by modifying the test to base it on spill counters rather than on bytes read/written. It also introduces a new configuration parameter "mapred.job.shuffle.input.buffer.percent" to provide finer grained control on the memory limit to be used during shuffle.
      Show
      Resolves the test failure by modifying the test to base it on spill counters rather than on bytes read/written. It also introduces a new configuration parameter "mapred.job.shuffle.input.buffer.percent" to provide finer grained control on the memory limit to be used during shuffle.

      Description

      Testcase: testReduceFromMem took 23.625 sec
      	FAILED
      Non-zero read from local: 83
      junit.framework.AssertionFailedError: Non-zero read from local: 83
      	at org.apache.hadoop.mapred.TestReduceFetch.testReduceFromMem(TestReduceFetch.java:289)
      	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      	at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      	at junit.extensions.TestSetup.run(TestSetup.java:27)
      

      Ran TestReduceFetch a few times on a clean trunk. It failed consistently.

        Attachments

        1. 6029-0.patch
          5 kB
          Chris Douglas
        2. 6029-1.patch
          5 kB
          Chris Douglas
        3. 6029-2.patch
          6 kB
          Chris Douglas
        4. FAILING-PARTIALMEM-TEST-org.apache.hadoop.mapred.TestReduceFetch.txt
          288 kB
          Devaraj Das
        5. M433-0v20.patch
          5 kB
          Chris Douglas
        6. M433-1v20.patch
          5 kB
          Chris Douglas
        7. M433-1y20.patch
          5 kB
          Chris Douglas
        8. M433-v20.another.patch
          5 kB
          Devaraj Das
        9. TEST-org.apache.hadoop.mapred.TestReduceFetch.txt
          288 kB
          Tsz Wo Nicholas Sze
        10. TEST-org.apache.hadoop.mapred.TestReduceFetch.txt
          302 kB
          Devaraj Das
        11. testReduceFromMem.txt
          287 kB
          Tsz Wo Nicholas Sze

          Issue Links

            Activity

              People

              • Assignee:
                chris.douglas Chris Douglas
                Reporter:
                szetszwo Tsz Wo Nicholas Sze
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: