Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1688

Impala 'INSERT IGNORE' is slow due to VLOG(1) logging in client

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.0.0
    • 1.1.0
    • impala, perf
    • None

    Description

      Currently Batcher::ProcessWriteResponse will log a VLOG(1) entry for every row error it experiences. When using INSERT IGNORE in Impala, this can yield thousands of logs per second and makes it extremely slow.

      We should consider bumping this to a higher vlog level, or pushing the 'insert ignore' flag into the operations themselves (even before KUDU-1563 is finished we could just drop the errors inside the client side)

      Attachments

        Activity

          People

            aserbin Alexey Serbin
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: