Uploaded image for project: 'Spot'
  1. Spot
  2. SPOT-236

[ML] Spot-ml cannot locate top-1m.csv when executing proxy

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Bug
    • Affects Version/s: 1.0
    • Fix Version/s: None
    • Labels:
    • Environment:
      cloudera 5.12.1

      Description

      when executing ml_op.sh

      ./ml_ops.sh 20171010 proxy -1 500
      

      I get error below.

      [Stage 2:> (0 + 0) / 4]
      [Stage 2:> (0 + 3) / 4]
      [Stage 2:============================================> (3 + 0) / 4]
      [Stage 2:============================================> (3 + 1) / 4]
      17/10/10 18:21:52 INFO SuspiciousConnectsAnalysis: Read source data 17/10/10 18:21:52 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.ExceptionInInitializerError java.lang.ExceptionInInitializerError at org.apache.spot.proxy.ProxySuspiciousConnectsModel$.ipWordCountFromDF(ProxySuspiciousConnectsModel.scala:196) at org.apache.spot.proxy.ProxySuspiciousConnectsModel$.getIPWordCounts(ProxySuspiciousConnectsModel.scala:186) at org.apache.spot.proxy.ProxySuspiciousConnectsModel$.trainModel(ProxySuspiciousConnectsModel.scala:151) at org.apache.spot.proxy.ProxySuspiciousConnectsAnalysis$.run(ProxySuspiciousConnectsAnalysis.scala:106) at org.apache.spot.SuspiciousConnects$.main(SuspiciousConnects.scala:78) at org.apache.spot.SuspiciousConnects.main(SuspiciousConnects.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:645)

      Caused by: java.io.FileNotFoundException: top-1m/top-1m.csv (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at scala.io.Source$.fromFile(Source.scala:91) at scala.io.Source$.fromFile(Source.scala:76) at scala.io.Source$.fromFile(Source.scala:54) at org.apache.spot.utilities.TopDomains$.<init>(TopDomains.scala:29) at org.apache.spot.utilities.TopDomains$.<clinit>(TopDomains.scala) ... 11 more

        Attachments

        1. Screen Shot 2017-10-11 at 10.58.33 AM.png
          34 kB
          Ricardo Barona
        2. spot-236-desktop.png
          20 kB
          hubert dulay

          Activity

            People

            • Assignee:
              rabarona Ricardo Barona
              Reporter:
              hdulay hubert dulay
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: