Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-554

Attempting to open a Stub Source!" when starting a collector sink

    XMLWordPrintableJSON

    Details

      Description

      I'm always getting this error with a fresh instalation using a
      collectorSink which writes to HDFS. The steps to reproduce it are
      1.- start master (flume master)
      2.- configure a collector sink:
      exec config collector02.data data-flow autoCollectorSource
      'collectorSink("hdfs://name.host/user/flume/input", "data-", 30000)'
      on the flume shell
      *collector02.data is already spawned to the physical node properly.
      3.- start node (flume node).
      Once the node is started I get this error:
      2011-03-09 16:51:03,604 [logicalNode collector02.data-26] INFO
      agent.LogicalNode: Connector started: LazyOpenSource |
      LazyOpenDecorator
      2011-03-09 16:51:03,606 [logicalNode collector02.data-26] ERROR
      connector.DirectDriver: Driving src/sink failed! LazyOpenSource |
      LazyOpenDecorator because Attempting to open a Stub Source!
      java.io.IOException: Attempting to open a Stub Source!
      at com.cloudera.flume.core.EventSource
      $StubSource.open(EventSource.java:75)
      at
      com.cloudera.flume.handlers.debug.LazyOpenSource.next(LazyOpenSource.java:
      53)
      at com.cloudera.flume.core.connector.DirectDriver
      $PumperThread.run(DirectDriver.java:89)
      2011-03-09 16:51:03,608 [logicalNode collector02.data-26] INFO
      agent.LogicalNode: Connector collector02.data exited with error
      Attempting to open a Stub Source!
      2011-03-09 16:51:03,608 [logicalNode collector02.data-26] ERROR
      agent.LogicalNode: Error closing collector02.data source: Attempting
      to close a Stub Source!
      2011-03-09 16:51:03,608 [logicalNode collector02.data-26] ERROR
      agent.LogicalNode: Driver on collector02.data closed LazyOpenSource |
      LazyOpenDecorator because of Attempting to open a Stub Source!
      java.io.IOException: Attempting to open a Stub Source!
      at com.cloudera.flume.core.EventSource
      $StubSource.open(EventSource.java:75)
      at
      com.cloudera.flume.handlers.debug.LazyOpenSource.next(LazyOpenSource.java:
      53)
      at com.cloudera.flume.core.connector.DirectDriver
      $PumperThread.run(DirectDriver.java:89)
      If i restart collector02.data (exec refresh collector02.data) the
      error apears again when refreshig, but, if I refresh again the error
      disapears (so refreshing twice the error does not apear anymore). The
      output looks like:
      2011-03-10 12:26:01,254 [SpawningLogicalNode collector02.data] INFO
      rolling.RollSink: Created RollSink: trigger=[TimeTrigger: maxAge=30000
      tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@28a50da4]
      checkPeriodMs = 250 spec='escapedCustomDfs("hdfs://name.host/user/
      flume/input","data-%

      {rolltag}

      " )'
      2011-03-10 12:26:01,255 [SpawningLogicalNode collector02.data] INFO
      thrift.ThriftEventSource: Closed server on port 35853...
      2011-03-10 12:26:01,256 [SpawningLogicalNode collector02.data] INFO
      thrift.ThriftEventSource: Queue still has 0 elements ...
      2011-03-10 12:26:01,285 [logicalNode collector02.data-37] INFO
      thrift.ThriftEventSource: Closed server on port 35853...
      2011-03-10 12:26:01,285 [logicalNode collector02.data-37] INFO
      thrift.ThriftEventSource: Queue still has 0 elements ...
      2011-03-10 12:26:01,285 [logicalNode collector02.data-37] WARN
      debug.LazyOpenDecorator: Closing a lazy sink that was not logically
      opened
      2011-03-10 12:26:01,285 [logicalNode collector02.data-37] INFO
      agent.LogicalNode: collector02.data: Connector stopped: LazyOpenSource

      LazyOpenDecorator
      2011-03-10 12:26:01,285 [SpawningLogicalNode collector02.data] INFO
      agent.LogicalNode: Node config successfully set to
      com.cloudera.flume.conf.FlumeConfigData@41f6321
      2011-03-10 12:26:01,287 [logicalNode collector02.data-48] INFO
      agent.LogicalNode: Connector started: LazyOpenSource

      LazyOpenDecorator
      2011-03-10 12:26:01,287 [logicalNode collector02.data-48] INFO
      thrift.ThriftEventSource: Starting blocking thread pool server on port
      35853...

      tip:
      The stub source indicates a failure to translate a configuration. This
      is because you have set a configuration before starting nodes. Since
      the node has not come up, the master does not know its ip/hostname and
      thus cannot automatically. In the current implementation the first
      heart beat from a node needs to appear so that logical names can be
      translated to a valid host/ip/port combination for the actual rpcs.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              flume_sturlese Disabled imported user
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: