void progress(Tuple input)
This would send a message back to the spout (through the acker) to increase the timeout for the roots of the tuple. Would be useful if the processing times of a tuple is highly variable. The timeout should reset to TOPOLOGY_MESSAGE_TIMEOUT plus the current time.
xumingming: what's the content of the tuple passed to the method progress?
nathanmarz: It would be the tuple that was passed to "execute". For example, in pseudocode:
// do some processing
So the progress method would extend the timeout for the spout tuples at the root of "input"
Just a note – a "progress" method could be a key to implementing MapReduce on top of Storm (along with more powerful grouping/scheduling capabilities and the ability to make use of local disk).
rohitprasad15: I wanted to take up this issue. I learnt Clojure and Storm a week back, but want to get deep into both.
I can briefly describe my approach -
1. Modify task.clj. Implement ^void progress [this ^Tuple tuple] as part of output-collector. Send a message to the acker, in a way similar to how ack method does it.
2. Modify acker.clj. Change execute() of IBolt implementation. Need to add ACKER-PROGRESS-STREAM-ID, to distinguish that its a progress message, and then finally reset the timeout to maxTopologyMessageTimeout.
Is this approach in the right direction?