Uploaded image for project: 'Hivemall'
  1. Hivemall
  2. HIVEMALL-260

Remove dependencies to Scala library in xgboost classifier

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.5.2
    • Fix Version/s: 0.6.0
    • Labels:
      None

      Description

      Since LabeledPoint is implemented in Scala, java library of xgboost requires Scala library.

      https://github.com/dmlc/xgboost/blob/master/jvm-packages/xgboost4j/src/main/scala/ml/dmlc/xgboost4j/LabeledPoint.scala

      So, replace ml/dmlc/xgboost4j/LabeledPoint.scala with ml/dmlc/xgboost4j/LabeledPoint.java in
      https://github.com/myui/build-xgboost-jvm

      java.lang.RuntimeException: java.lang.NoClassDefFoundError: scala/Product$class
       at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:257)
       at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
       at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
       at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:414)
       at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
       at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
       at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.NoClassDefFoundError: scala/Product$class
       at ml.dmlc.xgboost4j.LabeledPoint.<init>(LabeledPoint.scala:35)
       at ml.dmlc.xgboost4j.LabeledPoint.<init>(LabeledPoint.scala:41)
       at hivemall.xgboost.XGBoostUtils.parseFeatures(XGBoostUtils.java:54)
       at hivemall.xgboost.XGBoostUDTF.process(XGBoostUDTF.java:294)
       at org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116)
       at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
       at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
       at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:236)
       ... 11 more
      Caused by: java.lang.ClassNotFoundException: scala.Product$class
       at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
       at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
       ... 19 more

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                myui Makoto Yui
                Reporter:
                myui Makoto Yui
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: