Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16074

Parser can not deal with complex sql

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.1.1
    • 2.3.0
    • Parser
    • None

    Description

      Parser will stuck when parsing the sql into ast, eg:
      select intType,floatType,stringType,varchar32Type,byteType,shortType,dateType,timestampType,longType , max(intType | 0) from orctest where ((((((((((((((((((((((((((((((((((((((1=2)) and ((1=1))) or (((2=2)) and ((1=2)))) and (((1=1)) or ((0=0)))) or (((1=0)) or ((1=2)))) or (((1=1)) or ((0=0)))) or (((1=1)) and ((0=0)))) or (((2=2)) or ((2=2)))) and (((2=2)) or ((2=2)))) and (((2=2)) or ((1=2)))) or (((0=0)) and ((1=0)))) or (((1=1)) and ((1=2)))) or (((1=1)) and ((2=2)))) or (((1=2)) and ((1=1)))) or (((1=1)) and ((2=2)))) or (((1=1)) or ((1=1)))) and (((0=0)) and ((0=0)))) or (((0=0)) or ((1=2)))) and (((1=2)) and ((1=1)))) and (((0=0)) or ((2=2)))) or (((1=2)) or ((2=2)))) and (((0=0)) and ((1=2)))) or (((1=2)) or ((1=0)))) or (((2=2)) or ((1=0)))) or (((2=2)) or ((1=2)))) or (((1=2)) or ((1=1)))) and (((1=2)) or ((1=2)))) and (((1=0)) or ((2=2)))) and (((2=2)) or ((1=0)))) and (((1=2)) or ((1=1)))) and (((1=2)) or ((1=2)))) and (((2=2)) and ((1=0)))) and (((2=2)) or ((1=1)))) or (((0=0)) and ((1=1)))) and (((2=2)) or ((1=2)))) and (((1=2)) or ((0=0)))) or (((1=0)) and ((1=1)))) or (((1=1)) and ((1=2))) group by intType,floatType,stringType,varchar32Type,byteType,shortType,dateType,timestampType,longType;

      The jvm of Server was working fine, with continues minor gc.
      The cpu kept 100%.
      It means the java logic generated by antlr is too complex.
      The way to solve the problem may be optimize the syntax file or update the version of antlr.

      Attachments

        Issue Links

          Activity

            This might be a dup of HIVE-15388

            gopalv Gopal Vijayaraghavan added a comment - This might be a dup of HIVE-15388

            I tried the following query

            select key,value from src where ((((((((((((((((((((((((((((((((((((((1=2)) and ((1=1))) or (((2=2)) and ((1=2)))) and (((1=1)) or ((0=0)))) or (((1=0)) or ((1=2)))) or (((1=1)) or ((0=0)))) or (((1=1)) and ((0=0)))) or (((2=2)) or ((2=2)))) and (((2=2)) or ((2=2)))) and (((2=2)) or ((1=2)))) or (((0=0)) and ((1=0)))) or (((1=1)) and ((1=2)))) or (((1=1)) and ((2=2)))) or (((1=2)) and ((1=1)))) or (((1=1)) and ((2=2)))) or (((1=1)) or ((1=1)))) and (((0=0)) and ((0=0)))) or (((0=0)) or ((1=2)))) and (((1=2)) and ((1=1)))) and (((0=0)) or ((2=2)))) or (((1=2)) or ((2=2)))) and (((0=0)) and ((1=2)))) or (((1=2)) or ((1=0)))) or (((2=2)) or ((1=0)))) or (((2=2)) or ((1=2)))) or (((1=2)) or ((1=1)))) and (((1=2)) or ((1=2)))) and (((1=0)) or ((2=2)))) and (((2=2)) or ((1=0)))) and (((1=2)) or ((1=1)))) and (((1=2)) or ((1=2)))) and (((2=2)) and ((1=0)))) and (((2=2)) or ((1=1)))) or (((0=0)) and ((1=1)))) and (((2=2)) or ((1=2)))) and (((1=2)) or ((0=0)))) or (((1=0)) and ((1=1)))) or (((1=1)) and ((1=2))) group by key, value;
            

            it is reproducible without HIVE-15388. After I applied the patch, it is resolved. jinzheng, Please upgrade to the latest master.

            pxiong Pengcheng Xiong added a comment - I tried the following query select key,value from src where ((((((((((((((((((((((((((((((((((((((1=2)) and ((1=1))) or (((2=2)) and ((1=2)))) and (((1=1)) or ((0=0)))) or (((1=0)) or ((1=2)))) or (((1=1)) or ((0=0)))) or (((1=1)) and ((0=0)))) or (((2=2)) or ((2=2)))) and (((2=2)) or ((2=2)))) and (((2=2)) or ((1=2)))) or (((0=0)) and ((1=0)))) or (((1=1)) and ((1=2)))) or (((1=1)) and ((2=2)))) or (((1=2)) and ((1=1)))) or (((1=1)) and ((2=2)))) or (((1=1)) or ((1=1)))) and (((0=0)) and ((0=0)))) or (((0=0)) or ((1=2)))) and (((1=2)) and ((1=1)))) and (((0=0)) or ((2=2)))) or (((1=2)) or ((2=2)))) and (((0=0)) and ((1=2)))) or (((1=2)) or ((1=0)))) or (((2=2)) or ((1=0)))) or (((2=2)) or ((1=2)))) or (((1=2)) or ((1=1)))) and (((1=2)) or ((1=2)))) and (((1=0)) or ((2=2)))) and (((2=2)) or ((1=0)))) and (((1=2)) or ((1=1)))) and (((1=2)) or ((1=2)))) and (((2=2)) and ((1=0)))) and (((2=2)) or ((1=1)))) or (((0=0)) and ((1=1)))) and (((2=2)) or ((1=2)))) and (((1=2)) or ((0=0)))) or (((1=0)) and ((1=1)))) or (((1=1)) and ((1=2))) group by key, value; it is reproducible without HIVE-15388 . After I applied the patch, it is resolved. jinzheng , Please upgrade to the latest master.
            jinzheng jinzheng added a comment -

            thanks pxiong

            jinzheng jinzheng added a comment - thanks pxiong

            People

              pxiong Pengcheng Xiong
              jinzheng jinzheng
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 12h
                  12h
                  Remaining:
                  Remaining Estimate - 12h
                  12h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified