Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-1717

pig needs to call setPartitionFilter if schema is null but getPartitionKeys is not

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.9.0
    • 0.9.0
    • impl
    • None

    Description

      I'm writing a loader that works with hive style partitioning e.g. /logs/type1/daydate=2010-11-01
      The loader does not know the schema upfront and this is something that the user adds in the script using the AS clause.

      The problem is that this user defined schema is not available to the loader, so the loader cannot return any schema, the Loader does know what the partition keys are and pig needs in some way to know about these partition keys.

      Currently if the schema is null pig never calls the LoadMetaData:getPartitionKeys method or the setPartitionFilter method.

      Attachments

        1. PIG-1717.patch
          8 kB
          Gerrit Jansen van Vuuren
        2. PIG-1717.v1.patch
          12 kB
          Gerrit Jansen van Vuuren
        3. patchReleaseAuditWarnings.txt.gz
          114 kB
          Alan Gates
        4. testlog.tgz
          3 kB
          Alan Gates
        5. trunkReleaseAuditWarnings.txt.gz
          114 kB
          Alan Gates
        6. PIG-1717.v2.patch
          13 kB
          Gerrit Jansen van Vuuren

        Activity

          People

            gerritjvv Gerrit Jansen van Vuuren
            gerritjvv Gerrit Jansen van Vuuren
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: