Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-3020

Improve HDFSEventSink Escape Ingestion by more then 10x by not getting InetAddress on every record

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.8.0
    • Component/s: None
    • Labels:
      None

      Description

      If you are use escaping the current code will call InetAddress on every record. Which will result is a huge impact to performance.

      TotalTime,8403,
      totalEventTakeTime,1498,
      totalWriteTime,1981,
      totalWriterSetupTime,65,
      commitTime,201,
      flushTime,18,
      startTrans,7,

      The rest is all InetAddress

        Attachments

        1. flume-3020.patch
          2 kB
          Theodore michael Malaska
        2. flume-3020.patch.2
          2 kB
          Theodore michael Malaska

          Issue Links

          links to

          Web Link review board

            Activity

              People

              • Assignee:
                ted.m Theodore michael Malaska
                Reporter:
                ted.m Theodore michael Malaska
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: