Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-582

Nimbus Halt with FileNotFoundException: '../nimbus/../stormconf.ser' dose not exists

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.2-incubating
    • Fix Version/s: None
    • Component/s: storm-core
    • Labels:
      None

      Description

      To notice, it is different from STORM-130. It is nimbus to halt. We ran into this problem several times, every time it happens is after several days of stable running. Here is the stacktrace
      ======================================================
      2014-11-03 14:30:56 b.s.d.nimbus [INFO] Cleaning inbox ... deleted: stormjar-c1c856f0-cf8b-4299-9c20-712f169802b7.jar
      2014-11-12 19:32:33 b.s.d.nimbus [ERROR] Error when processing event
      java.io.FileNotFoundException: File '/tmp/storm-0.9.3/nimbus/stormdist/DNSAnalyse-7-1414992073/stormconf.ser' does not exist
      at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299) ~[commons-io-2.4.jar:2.4]
      at org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:1763) ~[commons-io-2.4.jar:2.4]
      at backtype.storm.daemon.nimbus$read_storm_conf.invoke(nimbus.clj:89) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
      at backtype.storm.daemon.nimbus$read_topology_details.invoke(nimbus.clj:324) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
      at backtype.storm.daemon.nimbus$mk_assignments$iter_31003104$fn_3105.invoke(nimbus.clj:649) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
      at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
      at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
      at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
      at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
      at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) ~[clojure-1.5.1.jar:na]
      at clojure.core.protocols$fn__6026.invoke(protocols.clj:54) ~[clojure-1.5.1.jar:na]
      at clojure.core.protocols$fn_5979$G5974_5992.invoke(protocols.clj:13) ~[clojure-1.5.1.jar:na]
      at clojure.core$reduce.invoke(core.clj:6177) ~[clojure-1.5.1.jar:na]
      at clojure.core$into.invoke(core.clj:6229) ~[clojure-1.5.1.jar:na]
      at backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:648) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
      at clojure.lang.RestFn.invoke(RestFn.java:410) ~[clojure-1.5.1.jar:na]
      at backtype.storm.daemon.nimbus$fn_3281$exec_fn1205auto__3282$fn3287$fn_3288.invoke(nimbus.clj:907) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
      at backtype.storm.daemon.nimbus$fn_3281$exec_fn1205auto__3282$fn_3287.invoke(nimbus.clj:906) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
      at backtype.storm.timer$schedule_recurring$this__2169.invoke(timer.clj:99) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
      at backtype.storm.timer$mk_timer$fn_2152$fn_2153.invoke(timer.clj:50) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
      at backtype.storm.timer$mk_timer$fn__2152.invoke(timer.clj:42) [storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
      at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
      at java.lang.Thread.run(Thread.java:679) [na:1.6.0_22]
      2014-11-12 19:32:33 b.s.util [INFO] Halting process: ("Error when processing an event")
      2014-11-12 19:32:33 b.s.d.nimbus [INFO] Shutting down master
      =====================================================
      To notice, after stable running for 9 days, without printing "Clean up

      {storm-id}

      " logs, nimbus halt.

        Attachments

          Activity

            People

            • Assignee:
              sriharsha Sriharsha Chintalapani
              Reporter:
              Gvain Jiahong Li
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: