Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4196

Certain builtin functions throw symbol does not exist error when used inside aggregate function

    Details

      Description

      Steps to reproduce this:

      [x.y.z.p:21000] > describe sample_07;
      Query: describe sample_07
      +-------------+--------+---------+
      | name        | type   | comment |
      +-------------+--------+---------+
      | code        | string |         |
      | description | string |         |
      | total_emp   | int    |         |
      | salary      | int    |         |
      +-------------+--------+---------+
      Fetched 4 row(s) in 0.02s
      
      [x.y.z.p:21000] > select count(bitand(total_emp,0)) from sample_07;
      Query: select count(bitand(total_emp,0)) from sample_07
      +-----------------------------+
      | count(bitand(total_emp, 0)) |
      +-----------------------------+
      | 823                         |
      +-----------------------------+
      Fetched 1 row(s) in 0.53s
      [x.y.z.p:21000] > select count(shiftleft(total_emp,0)) from sample_07;
      Query: select count(shiftleft(total_emp,0)) from sample_07
      WARNINGS: 
      Builtin 'shiftleft' with symbol '_ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_6IntValES6_' does not exist. Verify that all your impalads are the same version.
      
      [x.y.z.p:21000] > select count(shiftright(total_emp,0)) from sample_07;
      Query: select count(shiftright(total_emp,0)) from sample_07
      WARNINGS: 
      Builtin 'shiftright' with symbol '_ZN6impala16BitByteFunctions10ShiftRightEPN10impala_udf15FunctionContextERKNS1_6IntValES6_' does not exist. Verify that all your impalads are the same version.
      

        Issue Links

          Activity

          Hide
          tarmstrong Tim Armstrong added a comment -

          Do you know if this is running with codegen enabled or disabled?

          Show
          tarmstrong Tim Armstrong added a comment - Do you know if this is running with codegen enabled or disabled?
          Hide
          tarmstrong Tim Armstrong added a comment -

          Also, what operating system?

          Show
          tarmstrong Tim Armstrong added a comment - Also, what operating system?
          Hide
          mala_ck Mala Chikka Kempanna added a comment -

          DISABLE_CODEGEN: [0]
          2.6.32-358.el6.x86_64

          Show
          mala_ck Mala Chikka Kempanna added a comment - DISABLE_CODEGEN: [0] 2.6.32-358.el6.x86_64
          Hide
          tarmstrong Tim Armstrong added a comment -

          I was able to reproduce this. It's very strange since the symbol seems to be defined in the binary's symbol table:

          tarmstrong@tarmstrong-box:~/impala-4196-shiftright$ ls
          etc  impala-2.6.0+cdh5.8.0+0-1.cdh5.8.0.p0.119.el6.x86_64.rpm  usr  var
          tarmstrong@tarmstrong-box:~/impala-4196-shiftright$ objdump -T ./usr/lib/impala/sbin-retail/impalad  | grep ShiftLeft
          0000000000878350 g    DF .text	0000000000000039  Base        _ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_10TinyIntValERKNS1_6IntValE
          0000000001a8a9b0 g    DF .text	000000000000009b  Base        _ZN4llvm5APInt11tcShiftLeftEPmjj
          0000000000878410 g    DF .text	000000000000003a  Base        _ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_9BigIntValERKNS1_6IntValE
          00000000008783d0 g    DF .text	0000000000000038  Base        _ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_6IntValES6_
          0000000000878390 g    DF .text	0000000000000039  Base        _ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_11SmallIntValERKNS1_6IntValE
          

          Also the function can be called if it's outside the count():

          select shiftleft(total_emp,0) from sample_07;
          
          1	134354250
          2	6003930
          3	299160
          4	1655410
          5	61110
          6	36300
          7	165240
          8	322170
          9	47210
          10	239360
          11	264990
          12	484390
          13	41780
          14	28170
          15	58100
          
          Show
          tarmstrong Tim Armstrong added a comment - I was able to reproduce this. It's very strange since the symbol seems to be defined in the binary's symbol table: tarmstrong@tarmstrong-box:~/impala-4196-shiftright$ ls etc impala-2.6.0+cdh5.8.0+0-1.cdh5.8.0.p0.119.el6.x86_64.rpm usr var tarmstrong@tarmstrong-box:~/impala-4196-shiftright$ objdump -T ./usr/lib/impala/sbin-retail/impalad | grep ShiftLeft 0000000000878350 g DF .text 0000000000000039 Base _ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_10TinyIntValERKNS1_6IntValE 0000000001a8a9b0 g DF .text 000000000000009b Base _ZN4llvm5APInt11tcShiftLeftEPmjj 0000000000878410 g DF .text 000000000000003a Base _ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_9BigIntValERKNS1_6IntValE 00000000008783d0 g DF .text 0000000000000038 Base _ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_6IntValES6_ 0000000000878390 g DF .text 0000000000000039 Base _ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_11SmallIntValERKNS1_6IntValE Also the function can be called if it's outside the count(): select shiftleft(total_emp,0) from sample_07; 1 134354250 2 6003930 3 299160 4 1655410 5 61110 6 36300 7 165240 8 322170 9 47210 10 239360 11 264990 12 484390 13 41780 14 28170 15 58100
          Hide
          tarmstrong Tim Armstrong added a comment -

          Ok, I can reproduce this on master:

          [tarmstrong-box.ca.cloudera.com:21000] > select count(shiftleft(int_col, 1)) from functional_parquet.alltypes;
          Query: select count(shiftleft(int_col, 1)) from functional_parquet.alltypes
          Query submitted at: 2016-09-23 16:28:40 (Coordinator: http://tarmstrong-box:25000)
          Query progress can be monitored at: http://tarmstrong-box:25000/query_plan?query_id=c84ea7a023f46364:4c2b82cf00000000
          WARNINGS: 
          Builtin 'shiftleft' with symbol '_ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_6IntValES6_' does not exist. Verify that all your impalads are the same version.
          
          Show
          tarmstrong Tim Armstrong added a comment - Ok, I can reproduce this on master: [tarmstrong-box.ca.cloudera.com:21000] > select count(shiftleft(int_col, 1)) from functional_parquet.alltypes; Query: select count(shiftleft(int_col, 1)) from functional_parquet.alltypes Query submitted at: 2016-09-23 16:28:40 (Coordinator: http: //tarmstrong-box:25000) Query progress can be monitored at: http: //tarmstrong-box:25000/query_plan?query_id=c84ea7a023f46364:4c2b82cf00000000 WARNINGS: Builtin 'shiftleft' with symbol '_ZN6impala16BitByteFunctions9ShiftLeftEPN10impala_udf15FunctionContextERKNS1_6IntValES6_' does not exist. Verify that all your impalads are the same version.
          Hide
          bharathv bharath v added a comment -

          Commit: 0a96ead664b8866a128b370fab9b793e28685df8
          http://github.mtv.cloudera.com/CDH/Impala/commit/0a96ead664b8866a128b370fab9b793e28685df8
          Author: Bharath Vissapragada <bharathv@cloudera.com>
          Date: 2016-09-30 (Fri, 30 Sep 2016)

          Changed paths:
          M be/src/codegen/impala-ir.cc
          M be/src/exprs/CMakeLists.txt
          A be/src/exprs/bit-byte-functions-ir.cc
          R be/src/exprs/bit-byte-functions.cc
          M testdata/workloads/functional-query/queries/QueryTest/exprs.test

          Log Message:
          -----------
          IMPALA-4196: Cross compile bit-byte-functions

          Change-Id: I5a1291bfd202b500405a884e4a62f0ca2447244a
          Reviewed-on: http://gerrit.cloudera.org:8080/4557
          Reviewed-by: Bharath Vissapragada <bharathv@cloudera.com>
          Tested-by: Internal Jenkins

          Show
          bharathv bharath v added a comment - Commit: 0a96ead664b8866a128b370fab9b793e28685df8 http://github.mtv.cloudera.com/CDH/Impala/commit/0a96ead664b8866a128b370fab9b793e28685df8 Author: Bharath Vissapragada <bharathv@cloudera.com> Date: 2016-09-30 (Fri, 30 Sep 2016) Changed paths: M be/src/codegen/impala-ir.cc M be/src/exprs/CMakeLists.txt A be/src/exprs/bit-byte-functions-ir.cc R be/src/exprs/bit-byte-functions.cc M testdata/workloads/functional-query/queries/QueryTest/exprs.test Log Message: ----------- IMPALA-4196 : Cross compile bit-byte-functions Change-Id: I5a1291bfd202b500405a884e4a62f0ca2447244a Reviewed-on: http://gerrit.cloudera.org:8080/4557 Reviewed-by: Bharath Vissapragada <bharathv@cloudera.com> Tested-by: Internal Jenkins

            People

            • Assignee:
              bharathv bharath v
              Reporter:
              mala_ck Mala Chikka Kempanna
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development