Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10323

use snprintf instead of lexical_cast when casting int to string, to improve multi-thread performance

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 3.1.0, Impala 3.2.0, Impala 3.3.0, Impala 3.4.0
    • Impala 4.0.0
    • Backend
    • None
    • 3 nodes ( 24 core, 64 GB mem)
      impala 3.4
    • ghx-label-3

    Description

      For float_type, lexical_cast is replace by snprintf (issue https://issues.apache.org/jira/browse/IMPALA-1738),but why not do the same replacement for num_type.

      Test is done in 2 sql case :

      1)  group by cast int to string

      select cast(f1 as string) as kk, count ( * ) from test.my_table group by kk;

      2) group by int : 

      select f1 as kk, count( *  ) from test.my_table  group by kk;

      from the benchmark, we can see that performance decreased seriously with more thread.

      But using snprintf, performance improved significantly with more thread .

       

      Attachments

        1. image-2020-11-12-20-43-47-010.png
          59 kB
          wesleydeng
        2. image-2020-11-12-20-36-50-082.png
          55 kB
          wesleydeng

        Activity

          People

            wesleydeng_tencent wesleydeng
            wesleydeng_tencent wesleydeng
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 12h
                12h
                Remaining:
                Remaining Estimate - 12h
                12h
                Logged:
                Time Spent - Not Specified
                Not Specified