Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Resolves the test failure by modifying the test to base it on spill counters rather than on bytes read/written. It also introduces a new configuration parameter "mapred.job.shuffle.input.buffer.percent" to provide finer grained control on the memory limit to be used during shuffle.
      Show
      Resolves the test failure by modifying the test to base it on spill counters rather than on bytes read/written. It also introduces a new configuration parameter "mapred.job.shuffle.input.buffer.percent" to provide finer grained control on the memory limit to be used during shuffle.

      Description

      Testcase: testReduceFromMem took 23.625 sec
      	FAILED
      Non-zero read from local: 83
      junit.framework.AssertionFailedError: Non-zero read from local: 83
      	at org.apache.hadoop.mapred.TestReduceFetch.testReduceFromMem(TestReduceFetch.java:289)
      	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      	at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      	at junit.extensions.TestSetup.run(TestSetup.java:27)
      

      Ran TestReduceFetch a few times on a clean trunk. It failed consistently.

        Attachments

        1. FAILING-PARTIALMEM-TEST-org.apache.hadoop.mapred.TestReduceFetch.txt
          288 kB
          Devaraj Das
        2. TEST-org.apache.hadoop.mapred.TestReduceFetch.txt
          302 kB
          Devaraj Das
        3. TEST-org.apache.hadoop.mapred.TestReduceFetch.txt
          288 kB
          Tsz Wo Nicholas Sze
        4. testReduceFromMem.txt
          287 kB
          Tsz Wo Nicholas Sze
        5. 6029-0.patch
          5 kB
          Chris Douglas
        6. 6029-1.patch
          5 kB
          Chris Douglas
        7. 6029-2.patch
          6 kB
          Chris Douglas
        8. M433-0v20.patch
          5 kB
          Chris Douglas
        9. M433-v20.another.patch
          5 kB
          Devaraj Das
        10. M433-1v20.patch
          5 kB
          Chris Douglas
        11. M433-1y20.patch
          5 kB
          Chris Douglas

          Issue Links

            Activity

              People

              • Assignee:
                chris.douglas Chris Douglas
                Reporter:
                szetszwo Tsz Wo Nicholas Sze
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: