[FLUME-554] Attempting to open a Stub Source!" when starting a collector sink

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 0.9.3
• Fix Version/s: 0.9.5
• Component/s: Sinks+Sources
• Labels: collector flume sink
• Assignee: Unassigned
• Reporter: Disabled imported user
• Created: 10/Mar/11 18:03
• Updated: 05/Nov/14 09:42
• Resolved: 05/Nov/14 09:42

Description

I'm always getting this error with a fresh instalation using a
collectorSink which writes to HDFS. The steps to reproduce it are
1.- start master (flume master)
2.- configure a collector sink:
exec config collector02.data data-flow autoCollectorSource
'collectorSink("hdfs://name.host/user/flume/input", "data-", 30000)'
on the flume shell
*collector02.data is already spawned to the physical node properly.
3.- start node (flume node).
Once the node is started I get this error:
2011-03-09 16:51:03,604 [logicalNode collector02.data-26] INFO
agent.LogicalNode: Connector started: LazyOpenSource |
LazyOpenDecorator
2011-03-09 16:51:03,606 [logicalNode collector02.data-26] ERROR
connector.DirectDriver: Driving src/sink failed! LazyOpenSource |
LazyOpenDecorator because Attempting to open a Stub Source!
java.io.IOException: Attempting to open a Stub Source!
at com.cloudera.flume.core.EventSource
$StubSource.open(EventSource.java:75)
at
com.cloudera.flume.handlers.debug.LazyOpenSource.next(LazyOpenSource.java:
53)
at com.cloudera.flume.core.connector.DirectDriver
$PumperThread.run(DirectDriver.java:89)
2011-03-09 16:51:03,608 [logicalNode collector02.data-26] INFO
agent.LogicalNode: Connector collector02.data exited with error
Attempting to open a Stub Source!
2011-03-09 16:51:03,608 [logicalNode collector02.data-26] ERROR
agent.LogicalNode: Error closing collector02.data source: Attempting
to close a Stub Source!
2011-03-09 16:51:03,608 [logicalNode collector02.data-26] ERROR
agent.LogicalNode: Driver on collector02.data closed LazyOpenSource |
LazyOpenDecorator because of Attempting to open a Stub Source!
java.io.IOException: Attempting to open a Stub Source!
at com.cloudera.flume.core.EventSource
$StubSource.open(EventSource.java:75)
at
com.cloudera.flume.handlers.debug.LazyOpenSource.next(LazyOpenSource.java:
53)
at com.cloudera.flume.core.connector.DirectDriver
$PumperThread.run(DirectDriver.java:89)
If i restart collector02.data (exec refresh collector02.data) the
error apears again when refreshig, but, if I refresh again the error
disapears (so refreshing twice the error does not apear anymore). The
output looks like:
2011-03-10 12:26:01,254 [SpawningLogicalNode collector02.data] INFO
rolling.RollSink: Created RollSink: trigger=[TimeTrigger: maxAge=30000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@28a50da4]
checkPeriodMs = 250 spec='escapedCustomDfs("hdfs://name.host/user/
flume/input","data-%

{rolltag}

" )'
2011-03-10 12:26:01,255 [SpawningLogicalNode collector02.data] INFO
thrift.ThriftEventSource: Closed server on port 35853...
2011-03-10 12:26:01,256 [SpawningLogicalNode collector02.data] INFO
thrift.ThriftEventSource: Queue still has 0 elements ...
2011-03-10 12:26:01,285 [logicalNode collector02.data-37] INFO
thrift.ThriftEventSource: Closed server on port 35853...
2011-03-10 12:26:01,285 [logicalNode collector02.data-37] INFO
thrift.ThriftEventSource: Queue still has 0 elements ...
2011-03-10 12:26:01,285 [logicalNode collector02.data-37] WARN
debug.LazyOpenDecorator: Closing a lazy sink that was not logically
opened
2011-03-10 12:26:01,285 [logicalNode collector02.data-37] INFO
agent.LogicalNode: collector02.data: Connector stopped: LazyOpenSource

LazyOpenDecorator 2011-03-10 12:26:01,285 [SpawningLogicalNode collector02.data] INFO agent.LogicalNode: Node config successfully set to com.cloudera.flume.conf.FlumeConfigData@41f6321 2011-03-10 12:26:01,287 [logicalNode collector02.data-48] INFO agent.LogicalNode: Connector started: LazyOpenSource

LazyOpenDecorator
2011-03-10 12:26:01,287 [logicalNode collector02.data-48] INFO
thrift.ThriftEventSource: Starting blocking thread pool server on port
35853...

tip:
The stub source indicates a failure to translate a configuration. This
is because you have set a configuration before starting nodes. Since
the node has not come up, the master does not know its ip/hostname and
thus cannot automatically. In the current implementation the first
heart beat from a node needs to appear so that logical names can be
translated to a valid host/ip/port combination for the actual rpcs.

Sub-Tasks

• stub source error: Resolved
• Attempting to open a Stub Source!" when starting a collector sink: Resolved
• Attempting to open a Stub Source!" when starting a collector: Resolved