Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-857

CollectorSink can get stuck into perpetual close() if the underlying sinks throws IOExceptions for both append and close.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.3, 0.9.4
    • Fix Version/s: 0.9.5
    • Component/s: None
    • Labels:
      None

      Description

      The CollectionSink create a chain of sinks on top of the underlying sink to perform roll and retries upon failures. The stubborn append reopens the downstream sink and retries the append once. The insistent append keeps retrying the append.
      Now if any errors (like network or HDFS datanodes shutdown) cause the downstream sink to throw IOException on append, the stubborn append will try to close that sink and reopen. If the close also throws an IOException, then sink chain doesn't get properly closed. This results into the collectorsink skipping the open and perpetually failing on append/close.

        Attachments

          Activity

            People

            • Assignee:
              prasadm Prasad Suresh Mujumdar
              Reporter:
              prasadm Prasad Suresh Mujumdar
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: