[MAHOUT-834] rowsimilarityjob doesn't clean it's temp dir, and fails when seeing it again

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.6, 0.7
• Fix Version/s: 0.7
• Component/s: classic
• Assignee: Suneel Marthi
• Reporter: Dan Brickley
• Created: 09/Oct/11 13:37
• Updated: 31/Jan/24 22:17
• Resolved: 07/May/12 14:16

Description

If I do this:

mahout rowsimilarity --input matrixified/matrix --output sims/ --numberOfColumns 27684 --similarityClassname SIMILARITY_LOGLIKELIHOOD --excludeSelfSimilarity

then clean my output and rerun,

rm -rf sims/ # (though this step doesn't even seem needed)

then try again:

mahout rowsimilarity --input matrixified/matrix --output sims/ --numberOfColumns 27684 --similarityClassname SIMILARITY_LOGLIKELIHOOD --excludeSelfSimilarity

The temp files left from the first run make a re-run impossible - we get: "Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory temp/weights already exists".

Manually deleting the temp directory fixes this.

I get same behaviour if I explicitly pass in a --tempdir path, e.g.:

mahout rowsimilarity --input matrixified/matrix --output sims/ --numberOfColumns 27684 --similarityClassname SIMILARITY_LOGLIKELIHOOD --excludeSelfSimilarity --tempDir tmp2/

Presumably something like HadoopUtil.delete(getConf(),tempDirPath) is needed somewhere? (and maybe --overwrite too ?)