Pig
  1. Pig
  2. PIG-1167

[zebra] Zebra does not support Hadoop Globs

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6.0, 0.7.0
    • Component/s: None
    • Labels:
      None

      Description

      Pssing the following path to Zebra causing error but works with Hadoop directly: /projects/FETL/sample/ABF1/

      {2009120204}

        Activity

        Hide
        Yan Zhou added a comment -

        Zebra's TableLoader, the implementation of PIG's LoadFunc, does support glob by design. But Map/Reduce interface does not. TableLoader just expands the glob and passes the paths
        to Map/Reduce interface so a union of underlying Zebra tables will be loaded. It looks like that not enough test coverage is present in this area.

        Show
        Yan Zhou added a comment - Zebra's TableLoader, the implementation of PIG's LoadFunc, does support glob by design. But Map/Reduce interface does not. TableLoader just expands the glob and passes the paths to Map/Reduce interface so a union of underlying Zebra tables will be loaded. It looks like that not enough test coverage is present in this area.
        Hide
        Olga Natkovich added a comment -

        Just to clarify. Pig does support it for all loaders that use default pig slicer. Zebra, however, uses its own and that's why it is not getting this functionality for free.

        Show
        Olga Natkovich added a comment - Just to clarify. Pig does support it for all loaders that use default pig slicer. Zebra, however, uses its own and that's why it is not getting this functionality for free.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12428935/PIG-1167.patch
        against trunk revision 893827.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 11 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/162/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/162/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/162/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12428935/PIG-1167.patch against trunk revision 893827. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/162/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/162/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/162/console This message is automatically generated.
        Hide
        Chao Wang added a comment -

        Patch looks good +1.

        Show
        Chao Wang added a comment - Patch looks good +1.
        Hide
        Yan Zhou added a comment -

        Committed to both Apache trunk and 6.0 branch.

        Show
        Yan Zhou added a comment - Committed to both Apache trunk and 6.0 branch.

          People

          • Assignee:
            Yan Zhou
            Reporter:
            Olga Natkovich
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development