Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1819

ExecSource don't flush the cache if there is no input entries

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.4.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      ExecSource has a default batchSize: 20, exec source read data from the source, then put it into the cache, after the cache is full, push it to the channel.

      but if exec source's cache is not full, and there isn't any input for a long time, then these entries always kept in the cache, there is no chance to the channel until the source's cache is full.

      so, the patch added a new config line: batchTimeout for ExecSource, and default is 3 seconds, if batchTimeout exceeded, push all cached data to the channel even the cache is not full.

        Attachments

        1. FLUME-1819.patch
          5 kB
          Fengdong Yu
        2. FLUME-1819.patch.1
          7 kB
          Venkatesh Sivasubramanian
        3. FLUME-1819.patch.2
          15 kB
          Venkatesh Sivasubramanian
        4. FLUME-1819-3.patch
          25 kB
          Venkatesh Sivasubramanian

          Activity

            People

            • Assignee:
              venkyz Venkatesh Sivasubramanian
              Reporter:
              azuryy Fengdong Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: