[HIVEMALL-260] Remove dependencies to Scala library in xgboost classifier

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.5.2
• Fix Version/s: 0.6.0
• Assignee: Makoto Yui
• Reporter: Makoto Yui
• Created: 18/Jul/19 09:10
• Updated: 31/Oct/19 08:50
• Resolved: 29/Oct/19 06:38

Description

Since LabeledPoint is implemented in Scala, java library of xgboost requires Scala library.

https://github.com/dmlc/xgboost/blob/master/jvm-packages/xgboost4j/src/main/scala/ml/dmlc/xgboost4j/LabeledPoint.scala

So, replace ml/dmlc/xgboost4j/LabeledPoint.scala with ml/dmlc/xgboost4j/LabeledPoint.java in
https://github.com/myui/build-xgboost-jvm

java.lang.RuntimeException: java.lang.NoClassDefFoundError: scala/Product$class
 at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:257)
 at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
 at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
 at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:414)
 at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
 at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
 at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: scala/Product$class
 at ml.dmlc.xgboost4j.LabeledPoint.<init>(LabeledPoint.scala:35)
 at ml.dmlc.xgboost4j.LabeledPoint.<init>(LabeledPoint.scala:41)
 at hivemall.xgboost.XGBoostUtils.parseFeatures(XGBoostUtils.java:54)
 at hivemall.xgboost.XGBoostUDTF.process(XGBoostUDTF.java:294)
 at org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116)
 at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
 at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
 at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:236)
 ... 11 more
Caused by: java.lang.ClassNotFoundException: scala.Product$class
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 ... 19 more

Issue Links

• is a child of: HIVEMALL-276 [Umbrella] official support for xgboost v0.90
• links to: GitHub Pull Request #205