Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12513

SocketReceiver hang in Netcat example

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.5.2
    • 2.0.0
    • DStreams
    • None

    Description

      I add a SocketReceiver test based on the NetworkWordCount.

      Using pipeline and tail the continuous output to netcat
      tail -f xxx.log | nc -lk 9999

      and create a SocketReceiver to receive the continuous output from remote netcat.
      After about 10 hours, SocketReceiver hang and can not receive no more data.

      Netcat only accept one socket connection and push "tail -f xxx.log" to connected socket. other connection is wating in the netcat queue.
      When restart SocketReceive, new socket connection is created to connect to Netcat. However old connection is not closed properly. new connection can not read anything from Netcat.

      Attachments

        Activity

          People

            guoxu1231 Shawn Guo
            guoxu1231 Shawn Guo
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: