Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-3097

CLONE - WAL data grows forever even though data is delivered in E2E

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Blocker
    • Resolution: Unresolved
    • None
    • None
    • Master, Node, Sinks+Sources
    • None

    Description

      With a heavy enough write load, it appears that the E2E agent WAL will get into a state where data just gets constantly shuffled around between the various directories / states (e.g. writing, logged, sending, sent). When this happens, the WAL directories grow indefinitely until the disk is exhausted, no matter how much data caused the problem.

      To reproduce:

      • Use the supplied config (or something similar).
      • Write to the agent source at a rate of > 1MB/s for a short burst (using something like the provided generator below).
      • Note that data is delivered to the collectorSink but the agent WAL manager constantly grows the data.

      The config:

      n1 : execStream("tail -F datafile") | agentE2ESink("host", 12345);
      n2 : collectorSource(12345) | collectorSink("file://...", "n2-");
      

      Generator:

      perl -e 'while (1) { print $i++, "\n"; }' >> datafile
      

      This looks and smells just like FLUME-430. I haven't yet examined the WAL or destination data for duplicates / missing events.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              skysword_li jason.lee
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 12h
                  12h
                  Remaining:
                  Remaining Estimate - 12h
                  12h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified