Hive
  1. Hive
  2. HIVE-5456

Queries fail on avro backed table with empty partition

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0, 0.13.1
    • Fix Version/s: 0.14.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      The following query fails

      DROP TABLE IF EXISTS episodes_partitioned;
      CREATE TABLE episodes_partitioned
      PARTITIONED BY (doctor_pt INT)
      ROW FORMAT
      SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
      STORED AS
      INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
      OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
      TBLPROPERTIES ('avro.schema.literal'='{
        "namespace": "testing.hive.avro.serde",
        "name": "episodes",
        "type": "record",
        "fields": [
          {
            "name":"title",
            "type":"string",
            "doc":"episode title"
          },
          {
            "name":"air_date",
            "type":"string",
            "doc":"initial date"
          },
          {
            "name":"doctor",
            "type":"int",
            "doc":"main actor playing the Doctor in episode"
          }
        ]
      }');
      
      ALTER TABLE episodes_partitioned ADD PARTITION (doctor_pt=4);
      ALTER TABLE episodes_partitioned ADD PARTITION (doctor_pt=5);
      SELECT COUNT(*) FROM episodes_partitioned;
      

      with following exception

      java.io.IOException: org.apache.hadoop.hive.serde2.avro.AvroSerdeException: Neither avro.schema.literal nor avro.schema.url specified, can't determine table schema
              at org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat.getHiveRecordWriter(AvroContainerOutputFormat.java:61)
              at org.apache.hadoop.hive.ql.exec.Utilities.createEmptyFile(Utilities.java:2869)
              at org.apache.hadoop.hive.ql.exec.Utilities.createDummyFileForEmptyPartition(Utilities.java:2901)
              at org.apache.hadoop.hive.ql.exec.Utilities.getInputPaths(Utilities.java:2825)
              at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:381)
              at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151)
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65)
              at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1409)
              at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1187)
              at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1015)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:883)
              at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
              at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
              at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:446)
              at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:456)
              at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:737)
              at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
              at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
      
      1. HIVE-5456.patch
        4 kB
        Chaoyu Tang
      2. HIVE-5456.patch
        4 kB
        Chaoyu Tang

        Issue Links

          Activity

          Hide
          Prasad Mujumdar added a comment -

          The problem seems to be with the logic for creating dummy "empty" files for empty partitions in case of avro tables.

          Show
          Prasad Mujumdar added a comment - The problem seems to be with the logic for creating dummy "empty" files for empty partitions in case of avro tables.
          Hide
          Sean Busbey added a comment -

          The current patch for HIVE-5302 fixes this same error.

          Show
          Sean Busbey added a comment - The current patch for HIVE-5302 fixes this same error.
          Hide
          Prasad Mujumdar added a comment -

          Thanks Sean! I will go ahead and close this as duplicate.

          Show
          Prasad Mujumdar added a comment - Thanks Sean! I will go ahead and close this as duplicate.
          Hide
          Chaoyu Tang added a comment -

          This problem is still existing in trunk Hive-0.14.0 as of today (7/17/2014). The cause is that the properties used in createDummyFileForEmptyPartition only contain those from a partition, thus missing the Avro schema url/literal which might only exist as a table property.

          Show
          Chaoyu Tang added a comment - This problem is still existing in trunk Hive-0.14.0 as of today (7/17/2014). The cause is that the properties used in createDummyFileForEmptyPartition only contain those from a partition, thus missing the Avro schema url/literal which might only exist as a table property.
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12656556/HIVE-5456.patch

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/856/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/856/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-856/

          Messages:

          **** This message was trimmed, see log for full details ****
          As a result, alternative(s) 2 were disabled for that input
          warning(200): IdentifiersParser.g:68:4: 
          Decision can match input such as "LPAREN KW_NOT TinyintLiteral" using multiple alternatives: 1, 2
          
          As a result, alternative(s) 2 were disabled for that input
          warning(200): IdentifiersParser.g:68:4: 
          Decision can match input such as "LPAREN LPAREN Number" using multiple alternatives: 1, 2
          
          As a result, alternative(s) 2 were disabled for that input
          warning(200): IdentifiersParser.g:68:4: 
          Decision can match input such as "LPAREN KW_EXISTS LPAREN" using multiple alternatives: 1, 2
          
          As a result, alternative(s) 2 were disabled for that input
          warning(200): IdentifiersParser.g:115:5: 
          Decision can match input such as "KW_CLUSTER KW_BY LPAREN" using multiple alternatives: 1, 2
          
          As a result, alternative(s) 2 were disabled for that input
          warning(200): IdentifiersParser.g:127:5: 
          Decision can match input such as "KW_PARTITION KW_BY LPAREN" using multiple alternatives: 1, 2
          
          As a result, alternative(s) 2 were disabled for that input
          warning(200): IdentifiersParser.g:138:5: 
          Decision can match input such as "KW_DISTRIBUTE KW_BY LPAREN" using multiple alternatives: 1, 2
          
          As a result, alternative(s) 2 were disabled for that input
          warning(200): IdentifiersParser.g:149:5: 
          Decision can match input such as "KW_SORT KW_BY LPAREN" using multiple alternatives: 1, 2
          
          As a result, alternative(s) 2 were disabled for that input
          warning(200): IdentifiersParser.g:166:7: 
          Decision can match input such as "STAR" using multiple alternatives: 1, 2
          
          As a result, alternative(s) 2 were disabled for that input
          warning(200): IdentifiersParser.g:179:5: 
          Decision can match input such as "KW_UNIONTYPE" using multiple alternatives: 5, 6
          
          As a result, alternative(s) 6 were disabled for that input
          warning(200): IdentifiersParser.g:179:5: 
          Decision can match input such as "KW_STRUCT" using multiple alternatives: 4, 6
          
          As a result, alternative(s) 6 were disabled for that input
          warning(200): IdentifiersParser.g:179:5: 
          Decision can match input such as "KW_ARRAY" using multiple alternatives: 2, 6
          
          As a result, alternative(s) 6 were disabled for that input
          warning(200): IdentifiersParser.g:261:5: 
          Decision can match input such as "KW_FALSE" using multiple alternatives: 3, 8
          
          As a result, alternative(s) 8 were disabled for that input
          warning(200): IdentifiersParser.g:261:5: 
          Decision can match input such as "KW_DATE StringLiteral" using multiple alternatives: 2, 3
          
          As a result, alternative(s) 3 were disabled for that input
          warning(200): IdentifiersParser.g:261:5: 
          Decision can match input such as "KW_NULL" using multiple alternatives: 1, 8
          
          As a result, alternative(s) 8 were disabled for that input
          warning(200): IdentifiersParser.g:261:5: 
          Decision can match input such as "KW_TRUE" using multiple alternatives: 3, 8
          
          As a result, alternative(s) 8 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "KW_BETWEEN KW_MAP LPAREN" using multiple alternatives: 8, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER KW_BY" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT KW_OVERWRITE" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE KW_BY" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_LATERAL KW_VIEW" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER KW_BY" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT KW_INTO" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_UNION KW_ALL" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:393:5: 
          Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP KW_BY" using multiple alternatives: 2, 9
          
          As a result, alternative(s) 9 were disabled for that input
          warning(200): IdentifiersParser.g:518:5: 
          Decision can match input such as "{AMPERSAND..BITWISEXOR, DIV..DIVIDE, EQUAL..EQUAL_NS, GREATERTHAN..GREATERTHANOREQUALTO, KW_AND, KW_ARRAY, KW_BETWEEN..KW_BOOLEAN, KW_CASE, KW_DOUBLE, KW_FLOAT, KW_IF, KW_IN, KW_INT, KW_LIKE, KW_MAP, KW_NOT, KW_OR, KW_REGEXP, KW_RLIKE, KW_SMALLINT, KW_STRING..KW_STRUCT, KW_TINYINT, KW_UNIONTYPE, KW_WHEN, LESSTHAN..LESSTHANOREQUALTO, MINUS..NOTEQUAL, PLUS, STAR, TILDE}" using multiple alternatives: 1, 3
          
          As a result, alternative(s) 3 were disabled for that input
          [INFO] 
          [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-exec ---
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-exec ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] Copying 2 resources
          [INFO] Copying 3 resources
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-exec ---
          [INFO] Executing tasks
          
          main:
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-exec ---
          [INFO] Compiling 1716 source files to /data/hive-ptest/working/apache-svn-trunk-source/ql/target/classes
          [INFO] -------------------------------------------------------------
          [WARNING] COMPILATION WARNING : 
          [INFO] -------------------------------------------------------------
          [WARNING] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java: Some input files use or override a deprecated API.
          [WARNING] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java: Recompile with -Xlint:deprecation for details.
          [WARNING] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Query.java: Some input files use unchecked or unsafe operations.
          [WARNING] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Query.java: Recompile with -Xlint:unchecked for details.
          [INFO] 4 warnings 
          [INFO] -------------------------------------------------------------
          [INFO] -------------------------------------------------------------
          [ERROR] COMPILATION ERROR : 
          [INFO] -------------------------------------------------------------
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/java/org/apache/hadoop/hive/ql/processors/CommandUtil.java:[71,25] method checkPrivileges in interface org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizer cannot be applied to given types;
            required: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveOperationType,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzContext
            found: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveOperationType,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,<nulltype>
            reason: actual and formal argument lists differ in length
          [INFO] 1 error
          [INFO] -------------------------------------------------------------
          [INFO] ------------------------------------------------------------------------
          [INFO] Reactor Summary:
          [INFO] 
          [INFO] Hive .............................................. SUCCESS [9.852s]
          [INFO] Hive Ant Utilities ................................ SUCCESS [6.022s]
          [INFO] Hive Shims Common ................................. SUCCESS [4.859s]
          [INFO] Hive Shims 0.20 ................................... SUCCESS [3.003s]
          [INFO] Hive Shims Secure Common .......................... SUCCESS [4.347s]
          [INFO] Hive Shims 0.20S .................................. SUCCESS [2.933s]
          [INFO] Hive Shims 0.23 ................................... SUCCESS [6.975s]
          [INFO] Hive Shims ........................................ SUCCESS [0.842s]
          [INFO] Hive Common ....................................... SUCCESS [21.084s]
          [INFO] Hive Serde ........................................ SUCCESS [12.337s]
          [INFO] Hive Metastore .................................... SUCCESS [32.270s]
          [INFO] Hive Query Language ............................... FAILURE [1:04.911s]
          [INFO] Hive Service ...................................... SKIPPED
          [INFO] Hive JDBC ......................................... SKIPPED
          [INFO] Hive Beeline ...................................... SKIPPED
          [INFO] Hive CLI .......................................... SKIPPED
          [INFO] Hive Contrib ...................................... SKIPPED
          [INFO] Hive HBase Handler ................................ SKIPPED
          [INFO] Hive HCatalog ..................................... SKIPPED
          [INFO] Hive HCatalog Core ................................ SKIPPED
          [INFO] Hive HCatalog Pig Adapter ......................... SKIPPED
          [INFO] Hive HCatalog Server Extensions ................... SKIPPED
          [INFO] Hive HCatalog Webhcat Java Client ................. SKIPPED
          [INFO] Hive HCatalog Webhcat ............................. SKIPPED
          [INFO] Hive HCatalog Streaming ........................... SKIPPED
          [INFO] Hive HWI .......................................... SKIPPED
          [INFO] Hive ODBC ......................................... SKIPPED
          [INFO] Hive Shims Aggregator ............................. SKIPPED
          [INFO] Hive TestUtils .................................... SKIPPED
          [INFO] Hive Packaging .................................... SKIPPED
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD FAILURE
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 2:52.185s
          [INFO] Finished at: Fri Jul 18 15:41:28 EDT 2014
          [INFO] Final Memory: 85M/539M
          [INFO] ------------------------------------------------------------------------
          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hive-exec: Compilation failure
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/java/org/apache/hadoop/hive/ql/processors/CommandUtil.java:[71,25] method checkPrivileges in interface org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizer cannot be applied to given types;
          [ERROR] required: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveOperationType,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzContext
          [ERROR] found: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveOperationType,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,<nulltype>
          [ERROR] reason: actual and formal argument lists differ in length
          [ERROR] -> [Help 1]
          [ERROR] 
          [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
          [ERROR] Re-run Maven using the -X switch to enable full debug logging.
          [ERROR] 
          [ERROR] For more information about the errors and possible solutions, please read the following articles:
          [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
          [ERROR] 
          [ERROR] After correcting the problems, you can resume the build with the command
          [ERROR]   mvn <goals> -rf :hive-exec
          + exit 1
          '
          

          This message is automatically generated.

          ATTACHMENT ID: 12656556

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12656556/HIVE-5456.patch Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/856/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/856/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-856/ Messages: **** This message was trimmed, see log for full details **** As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:68:4: Decision can match input such as "LPAREN KW_NOT TinyintLiteral" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:68:4: Decision can match input such as "LPAREN LPAREN Number" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:68:4: Decision can match input such as "LPAREN KW_EXISTS LPAREN" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:115:5: Decision can match input such as "KW_CLUSTER KW_BY LPAREN" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:127:5: Decision can match input such as "KW_PARTITION KW_BY LPAREN" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:138:5: Decision can match input such as "KW_DISTRIBUTE KW_BY LPAREN" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:149:5: Decision can match input such as "KW_SORT KW_BY LPAREN" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:166:7: Decision can match input such as "STAR" using multiple alternatives: 1, 2 As a result, alternative(s) 2 were disabled for that input warning(200): IdentifiersParser.g:179:5: Decision can match input such as "KW_UNIONTYPE" using multiple alternatives: 5, 6 As a result, alternative(s) 6 were disabled for that input warning(200): IdentifiersParser.g:179:5: Decision can match input such as "KW_STRUCT" using multiple alternatives: 4, 6 As a result, alternative(s) 6 were disabled for that input warning(200): IdentifiersParser.g:179:5: Decision can match input such as "KW_ARRAY" using multiple alternatives: 2, 6 As a result, alternative(s) 6 were disabled for that input warning(200): IdentifiersParser.g:261:5: Decision can match input such as "KW_FALSE" using multiple alternatives: 3, 8 As a result, alternative(s) 8 were disabled for that input warning(200): IdentifiersParser.g:261:5: Decision can match input such as "KW_DATE StringLiteral" using multiple alternatives: 2, 3 As a result, alternative(s) 3 were disabled for that input warning(200): IdentifiersParser.g:261:5: Decision can match input such as "KW_NULL" using multiple alternatives: 1, 8 As a result, alternative(s) 8 were disabled for that input warning(200): IdentifiersParser.g:261:5: Decision can match input such as "KW_TRUE" using multiple alternatives: 3, 8 As a result, alternative(s) 8 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "KW_BETWEEN KW_MAP LPAREN" using multiple alternatives: 8, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER KW_BY" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT KW_OVERWRITE" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE KW_BY" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_LATERAL KW_VIEW" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER KW_BY" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT KW_INTO" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_UNION KW_ALL" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:393:5: Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP KW_BY" using multiple alternatives: 2, 9 As a result, alternative(s) 9 were disabled for that input warning(200): IdentifiersParser.g:518:5: Decision can match input such as "{AMPERSAND..BITWISEXOR, DIV..DIVIDE, EQUAL..EQUAL_NS, GREATERTHAN..GREATERTHANOREQUALTO, KW_AND, KW_ARRAY, KW_BETWEEN..KW_BOOLEAN, KW_CASE, KW_DOUBLE, KW_FLOAT, KW_IF, KW_IN, KW_INT, KW_LIKE, KW_MAP, KW_NOT, KW_OR, KW_REGEXP, KW_RLIKE, KW_SMALLINT, KW_STRING..KW_STRUCT, KW_TINYINT, KW_UNIONTYPE, KW_WHEN, LESSTHAN..LESSTHANOREQUALTO, MINUS..NOTEQUAL, PLUS, STAR, TILDE}" using multiple alternatives: 1, 3 As a result, alternative(s) 3 were disabled for that input [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-exec --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-exec --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 2 resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-exec --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-exec --- [INFO] Compiling 1716 source files to /data/hive-ptest/working/apache-svn-trunk-source/ql/target/classes [INFO] ------------------------------------------------------------- [WARNING] COMPILATION WARNING : [INFO] ------------------------------------------------------------- [WARNING] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java: Some input files use or override a deprecated API. [WARNING] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java: Recompile with -Xlint:deprecation for details. [WARNING] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Query.java: Some input files use unchecked or unsafe operations. [WARNING] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Query.java: Recompile with -Xlint:unchecked for details. [INFO] 4 warnings [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/java/org/apache/hadoop/hive/ql/processors/CommandUtil.java:[71,25] method checkPrivileges in interface org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizer cannot be applied to given types; required: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveOperationType,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzContext found: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveOperationType,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,<nulltype> reason: actual and formal argument lists differ in length [INFO] 1 error [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Hive .............................................. SUCCESS [9.852s] [INFO] Hive Ant Utilities ................................ SUCCESS [6.022s] [INFO] Hive Shims Common ................................. SUCCESS [4.859s] [INFO] Hive Shims 0.20 ................................... SUCCESS [3.003s] [INFO] Hive Shims Secure Common .......................... SUCCESS [4.347s] [INFO] Hive Shims 0.20S .................................. SUCCESS [2.933s] [INFO] Hive Shims 0.23 ................................... SUCCESS [6.975s] [INFO] Hive Shims ........................................ SUCCESS [0.842s] [INFO] Hive Common ....................................... SUCCESS [21.084s] [INFO] Hive Serde ........................................ SUCCESS [12.337s] [INFO] Hive Metastore .................................... SUCCESS [32.270s] [INFO] Hive Query Language ............................... FAILURE [1:04.911s] [INFO] Hive Service ...................................... SKIPPED [INFO] Hive JDBC ......................................... SKIPPED [INFO] Hive Beeline ...................................... SKIPPED [INFO] Hive CLI .......................................... SKIPPED [INFO] Hive Contrib ...................................... SKIPPED [INFO] Hive HBase Handler ................................ SKIPPED [INFO] Hive HCatalog ..................................... SKIPPED [INFO] Hive HCatalog Core ................................ SKIPPED [INFO] Hive HCatalog Pig Adapter ......................... SKIPPED [INFO] Hive HCatalog Server Extensions ................... SKIPPED [INFO] Hive HCatalog Webhcat Java Client ................. SKIPPED [INFO] Hive HCatalog Webhcat ............................. SKIPPED [INFO] Hive HCatalog Streaming ........................... SKIPPED [INFO] Hive HWI .......................................... SKIPPED [INFO] Hive ODBC ......................................... SKIPPED [INFO] Hive Shims Aggregator ............................. SKIPPED [INFO] Hive TestUtils .................................... SKIPPED [INFO] Hive Packaging .................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2:52.185s [INFO] Finished at: Fri Jul 18 15:41:28 EDT 2014 [INFO] Final Memory: 85M/539M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hive-exec: Compilation failure [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/ql/src/java/org/apache/hadoop/hive/ql/processors/CommandUtil.java:[71,25] method checkPrivileges in interface org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizer cannot be applied to given types; [ERROR] required: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveOperationType,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzContext [ERROR] found: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveOperationType,java.util.List<org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject>,<nulltype> [ERROR] reason: actual and formal argument lists differ in length [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hive-exec + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12656556
          Hide
          Chaoyu Tang added a comment -

          Looks like the build failed due to compilation errors which were not from this patch.

          Show
          Chaoyu Tang added a comment - Looks like the build failed due to compilation errors which were not from this patch.
          Hide
          Szehon Ho added a comment -

          +1, pending tests. Yea those compilation errors dont look related.

          Show
          Szehon Ho added a comment - +1, pending tests. Yea those compilation errors dont look related.
          Hide
          Chaoyu Tang added a comment -

          Reattach patch to trigger the tests.

          Show
          Chaoyu Tang added a comment - Reattach patch to trigger the tests.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12657365/HIVE-5456.patch

          ERROR: -1 due to 2 failed/errored test(s), 5751 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/25/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/25/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-25/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12657365

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12657365/HIVE-5456.patch ERROR: -1 due to 2 failed/errored test(s), 5751 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/25/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/25/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-25/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12657365
          Hide
          Szehon Ho added a comment -

          Committed to trunk. Thanks Chaoyu for the contribution!

          Show
          Szehon Ho added a comment - Committed to trunk. Thanks Chaoyu for the contribution!
          Hide
          Thejas M Nair added a comment -

          This has been fixed in 0.14 release. Please open new jira if you see any issues.

          Show
          Thejas M Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.

            People

            • Assignee:
              Chaoyu Tang
              Reporter:
              Prasad Mujumdar
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development