[MAHOUT-1545] Creating holdout sets with seq2sparse and split

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Later
• Affects Version/s: 0.9
• Fix Version/s: 0.10.0
• Component/s: Classification, CLI, Examples
• Assignee: Unassigned
• Reporter: Andrew Palumbo
• Created: 05/May/14 01:05
• Updated: 13/Apr/15 10:21
• Resolved: 18/May/14 06:35

Description

The current method for vectorizing data using seq2sparse and then "split" allows for a large amount of information to spill over from the training sets to the test sets- especially in the case of TF-IDF transformations. The IDF transform provides alot of information on the holdout set to the training set if calculated previous to splitting them up.

I'm not sure if given the current seq2sparse implementation's status as Legacy and the relatively minor advantages that it might give whether or not its worth adding something like a "split" option to SparseVectorsFromSequenceFiles.java. But i know that i saw a new implementation being discussed and and think that it would be worth it to have an option like this built in.

I think that this issue may have been raised before, but i wanted to bring it up again in light of the current move away from MapReduce and the new implementations of Mahout tools that will be coming along.