Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1802

Deserializing scan tokens should avoid round-trip to master

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.2.0
    • 1.13.0
    • client, perf

    Description

      Currently, KuduScanToken::DeserializeIntoScanner calls KuduClient::OpenTable() which makes a GetTableSchema call to the master. This round trip is a bit expensive because it's always a "thundering herd" for an Impala query or Spark job – every host deserializes a bunch of scan tokens at the same time and ends up having to back off.

      We should consider some ways to avoid this.

      Attachments

        Issue Links

          Activity

            People

              granthenke Grant Henke
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: