[FLINK-30730] StringIndexer cannot handle null values correctly

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: ml-2.1.0
• Fix Version/s: ml-2.2.0
• Component/s: Library / Machine Learning
• Labels: pull-request-available
• Assignee: Fan Hong
• Reporter: Fan Hong
• Created: 18/Jan/23 07:58
• Updated: 08/Feb/23 02:39
• Resolved: 07/Feb/23 11:32

Description

When training data contains null values, StringIndexer throws a exception. The reason is this method [1]: null values are neither String type nor Number type.

In StringIndexerModel, null values are also not handled correctly when performing transformation.

 

[1] https://github.com/apache/flink-ml/blob/966cedd7bbab4e12d8d8b37dbd582146714e68a6/flink-ml-lib/src/main/java/org/apache/flink/ml/feature/stringindexer/StringIndexer.java#L164

Issue Links

• links to: GitHub Pull Request #205
• links to: GitHub Pull Request #208