[SPARK-7514] Add MinMaxScaler to feature transformation

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: MLlib
• Target Version/s: 1.5.0
• Assignee: yuhao yang
• Reporter: yuhao yang
• Created: 10/May/15 08:59
• Updated: 12/Aug/15 20:24
• Resolved: 30/Jun/15 19:45

Description

Add a popular scaling method to feature component, which is commonly known as min-max normalization or Rescaling.

Core function is,
Normalized( x ) = (x - min) / (max - min) * scale + newBase

where newBase and scale are parameters of the VectorTransformer. newBase is the new minimum number for the feature, and scale controls the range after transformation. This is a little complicated than the basic MinMax normalization, yet it provides flexibility so that users can control the range more specifically. like [0.1, 0.9] in some NN application.

for case that max == min, 0.5 is used as the raw value.

reference:
http://en.wikipedia.org/wiki/Feature_scaling
http://stn.spotfire.com/spotfire_client_help/index.htm#norm/norm_scale_between_0_and_1.htm

Issue Links

• relates to: SPARK-8529 Set metadata for MinMaxScaler
• relates to: SPARK-8530 Add Python API for MinMaxScaler
• relates to: SPARK-8531 Update ML user guide for MinMaxScaler
• relates to: SPARK-9891 User guide for MinMaxScaler
• links to: [Github] Pull Request #6039 (hhbyyh)