Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4802

ReceiverInfo removal at ReceiverTracker upon deregistering receiver

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.1.2, 1.2.1, 1.3.0
    • Component/s: DStreams
    • Labels:
      None

      Description

      When the streaming receiver is deregistered, the ReceiverTracker doesn't remove the corresponding receiverInfo entry for the receiver.

      When the receiver is stopped at the executor and the ReceiverTrackerActor that processes the 'DeregisterReceiver' message. Shouldn't it remove the receiverInfo entry for that receiver as the receiver is actually deregistered?

      Not sure if there is any specific reason for not removing it.
      Currently, I see this warning if we don't remove it:

      WARN main-EventThread scheduler.ReceiverTracker - All of the receivers have not deregistered, Map(0 -> ReceiverInfo(0,MyReceiver-0,null,false,localhost,Stopped by driver,))

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ilayaperumalg Ilayaperumal Gopinathan
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: