Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4296

Cannot filter partition by 'matches'

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.12.0
    • None
    • None
    • None
    • Cloudera CDH 5.2, hive 0.13, pig 0.12

    Description

      I use Cloudera CDH 5.2 with hive 0.13 and pig 0.12, and I have a hive table with partition named 'g'. When i try to run:

      A = LOAD 'test' USING org.apache.hcatalog.pig.HCatLoader() AS (a:chararray, b:chararray, game:chararray, g:chararray);
      B = FILTER A BY g matches 'abc';
      DUMP B;

      I receive error:

      Caused by: java.io.IOException: MetaException(message:Error parsing partition filter; lexer error: null; exception NoViableAltException(11@[]))
      at org.apache.hcatalog.mapreduce.HCatInputFormat.setFilter(HCatInputFormat.java:111)
      at org.apache.hcatalog.pig.HCatLoader.setLocation(HCatLoader.java:120)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:498)
      ... 19 more
      Caused by: MetaException(message:Error parsing partition filter; lexer error: null; exception NoViableAltException(11@[]))
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_filter_result$get_partitions_by_filter_resultStandardScheme.read(ThriftHiveMetastore.java:64602)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_filter_result$get_partitions_by_filter_resultStandardScheme.read(ThriftHiveMetastore.java:64569)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_filter_result.read(ThriftHiveMetastore.java:64503)
      at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_partitions_by_filter(ThriftHiveMetastore.java:1942)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partitions_by_filter(ThriftHiveMetastore.java:1926)
      at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.listPartitionsByFilter(HiveMetaStoreClient.java:914)
      at org.apache.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:113)
      at org.apache.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
      at org.apache.hcatalog.mapreduce.HCatInputFormat.setFilter(HCatInputFormat.java:109)
      ... 21 more

      I tested 'matches' with any field (not partition) it's ok.
      With my old Cloudera CDH 4.6.0 hive 0.11 and pig 0.11 it's still OK with 'matches' partition.

      Please check if I have any mistake. Thank you.

      Attachments

        Activity

          People

            Unassigned Unassigned
            lhphuc210291 Luong Hoang Phuc
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: