Pig
  1. Pig
  2. PIG-3162

PigTest.assertOutput doesn't allow non-default delimiter

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.11
    • Fix Version/s: None
    • Component/s: tools
    • Labels:

      Description

      PigTest.assertInput(String aliasInput, String[] input, String alias, String[] expected) assumes that the default delimiter is used (i.e. tab char) in input data.

      TestPig.java
      override(aliasInput, String.format(
          "%s = LOAD '%s' AS %s;", aliasInput, destination, sb.toString()));
      

      But it will be useful to be able to use a non-default delimiter. For example, here is an email from the user mailing list:
      http://search-hadoop.com/m/Pxcfq1TrnIb/PigUnit+test+for+script+with+non-default+PigStorage+delimiter&subj=PigUnit+test+for+script+with+non+default+PigStorage+delimiter

      1. PIG-3162-2.nows.patch.txt
        3 kB
        Johnny Zhang
      2. PIG-3162-2.patch.txt
        3 kB
        Johnny Zhang
      3. PIG-3162.patch.txt
        3 kB
        Johnny Zhang
      4. PIG-3162.nosw.patch.txt
        2 kB
        Johnny Zhang

        Activity

        Cheolsoo Park created issue -
        Cheolsoo Park made changes -
        Field Original Value New Value
        Labels newbie
        Johnny Zhang made changes -
        Assignee Johnny Zhang [ dreambird ]
        Hide
        Prashant Kommireddi added a comment -

        Cheolsoo, it would also be useful if Load/Store Func and associated delimiter (if any) is picked up from the pig script itself instead of specifying it manually?

        Show
        Prashant Kommireddi added a comment - Cheolsoo, it would also be useful if Load/Store Func and associated delimiter (if any) is picked up from the pig script itself instead of specifying it manually?
        Johnny Zhang made changes -
        Attachment PIG-3162.nosw.patch.txt [ 12569427 ]
        Attachment PIG-3162.patch.txt [ 12569428 ]
        Hide
        Johnny Zhang added a comment -

        I added one more method to pass in delimiter in class PigTest.java, and add one more test case in TestPigTest.java to verify it (delimiter as ",").

        ant clean test -Dtestcase=TestPigTest -Dtest.junit.output.format=xml
        

        pass for me.

        Show
        Johnny Zhang added a comment - I added one more method to pass in delimiter in class PigTest.java, and add one more test case in TestPigTest.java to verify it (delimiter as ","). ant clean test -Dtestcase=TestPigTest -Dtest.junit.output.format=xml pass for me.
        Johnny Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Cheolsoo Park added a comment -

        Johnny Zhang, can you please get rid of duplicate code? You can do something like this, for example:

        public void assertOutput(String aliasInput, String[] input, String alias, String[] expected) throws IOException, ParseException {
            assertOutput(aliasInput, input, alias, expected, "\\t");
        }
        

        Prashant Kommireddi, that sounds like a good suggestion. Can we do it in another jira?

        Show
        Cheolsoo Park added a comment - Johnny Zhang , can you please get rid of duplicate code? You can do something like this, for example: public void assertOutput( String aliasInput, String [] input, String alias, String [] expected) throws IOException, ParseException { assertOutput(aliasInput, input, alias, expected, "\\t" ); } Prashant Kommireddi , that sounds like a good suggestion. Can we do it in another jira?
        Cheolsoo Park made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Prashant Kommireddi added a comment -

        Sounds good Cheolsoo Park

        Show
        Prashant Kommireddi added a comment - Sounds good Cheolsoo Park
        Hide
        Johnny Zhang added a comment -

        Cheolsoo Park, thanks a lot for review the patch. The revised patches are posted.

        Show
        Johnny Zhang added a comment - Cheolsoo Park , thanks a lot for review the patch. The revised patches are posted.
        Johnny Zhang made changes -
        Attachment PIG-3162-2.patch.txt [ 12571318 ]
        Attachment PIG-3162-2.nows.patch.txt [ 12571319 ]
        Johnny Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Cheolsoo Park added a comment -

        +1. I will commit it after running tests.

        Show
        Cheolsoo Park added a comment - +1. I will commit it after running tests.
        Cheolsoo Park made changes -
        Issue Type Bug [ 1 ] Improvement [ 4 ]
        Hide
        Cheolsoo Park added a comment -

        Committed to trunk. Thanks Johnny!

        Show
        Cheolsoo Park added a comment - Committed to trunk. Thanks Johnny!
        Cheolsoo Park made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        12d 23h 7m 1 Cheolsoo Park 28/Feb/13 00:01
        Open Open Patch Available Patch Available
        9d 2h 16m 2 Johnny Zhang 28/Feb/13 00:36
        Patch Available Patch Available Resolved Resolved
        1d 18h 9m 1 Cheolsoo Park 01/Mar/13 18:46

          People

          • Assignee:
            Johnny Zhang
            Reporter:
            Cheolsoo Park
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development