Pig
  1. Pig
  2. PIG-3162

PigTest.assertOutput doesn't allow non-default delimiter

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.11
    • Fix Version/s: None
    • Component/s: tools
    • Labels:

      Description

      PigTest.assertInput(String aliasInput, String[] input, String alias, String[] expected) assumes that the default delimiter is used (i.e. tab char) in input data.

      TestPig.java
      override(aliasInput, String.format(
          "%s = LOAD '%s' AS %s;", aliasInput, destination, sb.toString()));
      

      But it will be useful to be able to use a non-default delimiter. For example, here is an email from the user mailing list:
      http://search-hadoop.com/m/Pxcfq1TrnIb/PigUnit+test+for+script+with+non-default+PigStorage+delimiter&subj=PigUnit+test+for+script+with+non+default+PigStorage+delimiter

      1. PIG-3162-2.patch.txt
        3 kB
        Johnny Zhang
      2. PIG-3162-2.nows.patch.txt
        3 kB
        Johnny Zhang
      3. PIG-3162.patch.txt
        3 kB
        Johnny Zhang
      4. PIG-3162.nosw.patch.txt
        2 kB
        Johnny Zhang

        Activity

        Hide
        Prashant Kommireddi added a comment -

        Cheolsoo, it would also be useful if Load/Store Func and associated delimiter (if any) is picked up from the pig script itself instead of specifying it manually?

        Show
        Prashant Kommireddi added a comment - Cheolsoo, it would also be useful if Load/Store Func and associated delimiter (if any) is picked up from the pig script itself instead of specifying it manually?
        Hide
        Johnny Zhang added a comment -

        I added one more method to pass in delimiter in class PigTest.java, and add one more test case in TestPigTest.java to verify it (delimiter as ",").

        ant clean test -Dtestcase=TestPigTest -Dtest.junit.output.format=xml
        

        pass for me.

        Show
        Johnny Zhang added a comment - I added one more method to pass in delimiter in class PigTest.java, and add one more test case in TestPigTest.java to verify it (delimiter as ","). ant clean test -Dtestcase=TestPigTest -Dtest.junit.output.format=xml pass for me.
        Hide
        Cheolsoo Park added a comment -

        Johnny Zhang, can you please get rid of duplicate code? You can do something like this, for example:

        public void assertOutput(String aliasInput, String[] input, String alias, String[] expected) throws IOException, ParseException {
            assertOutput(aliasInput, input, alias, expected, "\\t");
        }
        

        Prashant Kommireddi, that sounds like a good suggestion. Can we do it in another jira?

        Show
        Cheolsoo Park added a comment - Johnny Zhang , can you please get rid of duplicate code? You can do something like this, for example: public void assertOutput( String aliasInput, String [] input, String alias, String [] expected) throws IOException, ParseException { assertOutput(aliasInput, input, alias, expected, "\\t" ); } Prashant Kommireddi , that sounds like a good suggestion. Can we do it in another jira?
        Hide
        Prashant Kommireddi added a comment -

        Sounds good Cheolsoo Park

        Show
        Prashant Kommireddi added a comment - Sounds good Cheolsoo Park
        Hide
        Johnny Zhang added a comment -

        Cheolsoo Park, thanks a lot for review the patch. The revised patches are posted.

        Show
        Johnny Zhang added a comment - Cheolsoo Park , thanks a lot for review the patch. The revised patches are posted.
        Hide
        Cheolsoo Park added a comment -

        +1. I will commit it after running tests.

        Show
        Cheolsoo Park added a comment - +1. I will commit it after running tests.
        Hide
        Cheolsoo Park added a comment -

        Committed to trunk. Thanks Johnny!

        Show
        Cheolsoo Park added a comment - Committed to trunk. Thanks Johnny!

          People

          • Assignee:
            Johnny Zhang
            Reporter:
            Cheolsoo Park
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development