[PIG-3961] Adding HBaseStorage cell value filters

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Fix Version/s: 0.15.0
• Patch Info: Patch Available
• Assignee: Mike Welch
• Reporter: Mike Welch
• Created: 23/May/14 21:42
• Updated: 28/Apr/15 23:05
• Resolved: 28/Apr/15 23:05

Description

Adding three additional server side filtering options when loading data with HBaseStorage:

1. specified cf:col does not exist
   -null cf:col
2. specified cf:col must exist
   -notnull cf:col
3. specified cf:col contains the given value
   -val cf:col=value

These are meant to replace (and optimize by reducing data transfer) the frequent paradigm in pig of loading data and immediately filtering for a specific condition. For example

data = load 'hbase://mytable' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:*') as (cf:map[]) ;
data_with_value = filter data by cf#'col' = 'value' ;

Can be replaced with:

data_with_value = load 'hbase://mytable' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:*', '-val cf:col=value') as (cf:map[]) ;