Pig
  1. Pig
  2. PIG-3091

Make schema, header and stats file configurable in JsonMetadata

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      JsonMetadata currently sets schema, header and stats file to the following

          private String schemaFileName = ".pig_schema";
          private String headerFileName = ".pig_header";
          private String statFileName = ".pig_stats";
      

      This could be made configurable so users can create custom schema files (used by custom Load/StoreFuncs)

      1. PIG-3091.patch
        2 kB
        Prashant Kommireddi
      2. PIG-3091_2.patch
        2 kB
        Prashant Kommireddi
      3. PIG-3091_1.patch
        2 kB
        Prashant Kommireddi

        Activity

        Hide
        Prashant Kommireddi added a comment -

        Minor change, please review.

        Show
        Prashant Kommireddi added a comment - Minor change, please review.
        Hide
        Prashant Kommireddi added a comment -

        Removed a comment that does not hold good anymore.

        Show
        Prashant Kommireddi added a comment - Removed a comment that does not hold good anymore.
        Hide
        Jonathan Coveney added a comment -

        A minor nit, but since you add the new constructor, you can just do the following:

        public JsonMetadata() {
            this(".pig_schema",".pig_header",".pig_stats");
        }
        

        The advantage being that if we change the logic around what to do with those file names, it can be kept in one place.

        Show
        Jonathan Coveney added a comment - A minor nit, but since you add the new constructor, you can just do the following: public JsonMetadata() { this ( ".pig_schema" , ".pig_header" , ".pig_stats" ); } The advantage being that if we change the logic around what to do with those file names, it can be kept in one place.
        Hide
        Prashant Kommireddi added a comment -

        That makes sense, will add a new patch.

        Show
        Prashant Kommireddi added a comment - That makes sense, will add a new patch.
        Hide
        Prashant Kommireddi added a comment -

        Jon, ready for your review!

        Show
        Prashant Kommireddi added a comment - Jon, ready for your review!
        Hide
        Jonathan Coveney added a comment -

        +1. I'll commit shortly.

        Show
        Jonathan Coveney added a comment - +1. I'll commit shortly.
        Hide
        Jonathan Coveney added a comment -

        Committed. Thanks, Prashant!

        Show
        Jonathan Coveney added a comment - Committed. Thanks, Prashant!

          People

          • Assignee:
            Prashant Kommireddi
            Reporter:
            Prashant Kommireddi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development