Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
0.2.0, 0.2.1, 0.2.2
-
None
Description
Still working to reproduce this condition, if others are seeing something similar please describe it here so that we can narrow in on and hopefully reliably reproduce it...
In a m/r job, the majority of reducers are fairly quickly successful and complete. Then a small subset slow to an absolute crawl. There are some overflow rows - but not an abnormal amount as compared to other successful reducers. It indexes locally, and I notice in the main index directory there are around 14k files which I imagine is an important clue.
I changed two things at once to get the job to complete successfully in a reasonable amount of time. I limited the row size to max of 250k records and added a multiplier of 4.