Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18035

NullPointerException on querying a table with a compact index

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.1.0
    • Fix Version/s: None
    • Component/s: Indexing
    • Labels:
      None
    • Environment:

      CDH 5.8.3 on RHEL Server 6.8

      Description

      After creating an index on a table (sys_created_on is a STRING column):

      CREATE INDEX test_sys_audit_index_sys_created_on
      ON TABLE servicenow_stg.sys_audit_distinct_tmp (sys_created_on)
      AS 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler'
      WITH DEFERRED REBUILD;
      ALTER INDEX test_sys_audit_index_sys_created_on ON servicenow_stg.sys_audit_distinct_tmp REBUILD;
      

      ... the following query fails:

      SELECT sys_created_on FROM servicenow_stg.sys_audit_distinct_tmp
      WHERE sys_created_on > '2017-01-01 00:00:00'
      

      ... with this stack trace:

      [Cloudera][HiveJDBCDriver](500051) ERROR processing query/statement. Error Code: 40000, SQL state: TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Error while compiling statement: FAILED: NullPointerException null:17:16, org.apache.hive.service.cli.operation.Operation:toSQLException:Operation.java:385, org.apache.hive.service.cli.operation.SQLOperation:prepare:SQLOperation.java:136, org.apache.hive.service.cli.operation.SQLOperation:runInternal:SQLOperation.java:206, org.apache.hive.service.cli.operation.Operation:run:Operation.java:327, org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementInternal:HiveSessionImpl.java:424, org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementAsync:HiveSessionImpl.java:401, org.apache.hive.service.cli.CLIService:executeStatementAsync:CLIService.java:258, org.apache.hive.service.cli.thrift.ThriftCLIService:ExecuteStatement:ThriftCLIService.java:500, org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1313, org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1298, org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39, org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39, org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:746, org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286, java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1142, java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:617, java.lang.Thread:run:Thread.java:745, *java.lang.NullPointerException:null:40:24, org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler:generateIndexQuery:CompactIndexHandler.java:199, org.apache.hadoop.hive.ql.optimizer.physical.index.IndexWhereProcessor:rewriteForIndexes:IndexWhereProcessor.java:215, org.apache.hadoop.hive.ql.optimizer.physical.index.IndexWhereProcessor:process:IndexWhereProcessor.java:146, org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher:dispatch:DefaultRuleDispatcher.java:90, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:dispatchAndReturn:DefaultGraphWalker.java:94, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:dispatch:DefaultGraphWalker.java:78, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:walk:DefaultGraphWalker.java:132, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:startWalking:DefaultGraphWalker.java:109, org.apache.hadoop.hive.ql.optimizer.physical.index.IndexWhereTaskDispatcher:dispatch:IndexWhereTaskDispatcher.java:102, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:dispatchAndReturn:DefaultGraphWalker.java:94, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:dispatch:DefaultGraphWalker.java:78, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:walk:DefaultGraphWalker.java:132, org.apache.hadoop.hive.ql.lib.DefaultGraphWalker:startWalking:DefaultGraphWalker.java:109, org.apache.hadoop.hive.ql.optimizer.physical.IndexWhereResolver:resolve:IndexWhereResolver.java:38, org.apache.hadoop.hive.ql.optimizer.physical.PhysicalOptimizer:optimize:PhysicalOptimizer.java:107, org.apache.hadoop.hive.ql.parse.MapReduceCompiler:optimizeTaskPlan:MapReduceCompiler.java:273, org.apache.hadoop.hive.ql.parse.TaskCompiler:compile:TaskCompiler.java:225, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:analyzeInternal:SemanticAnalyzer.java:10109, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer:analyzeInternal:SemanticAnalyzer.java:9902, org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer:analyze:BaseSemanticAnalyzer.java:223, org.apache.hadoop.hive.ql.Driver:compile:Driver.java:446, org.apache.hadoop.hive.ql.Driver:compile:Driver.java:312, org.apache.hadoop.hive.ql.Driver:compileInternal:Driver.java:1201, org.apache.hadoop.hive.ql.Driver:compileAndRespond:Driver.java:1188, org.apache.hive.service.cli.operation.SQLOperation:prepare:SQLOperation.java:134], sqlState:42000, errorCode:40000, errorMessage:Error while compiling statement: FAILED: NullPointerException null), Query: SELECT sys_created_on FROM servicenow_stg.sys_audit_distinct_tmp WHERE sys_created_on > from_unixtime(unix_timestamp('2017-01-01')).
      

      I'll be able to test this with CDH 5.11.2 in two weeks.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              brechtm Brecht Machiels
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: