Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2579

Support for multiple input schemas in AvroStorage

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.9.2, 0.11
    • 0.11
    • piggybank
    • None
    • Patch Available
    • Reviewed

    Description

      This is a barebones patch for AvroStorage which enables support of multiple input schemas. The assumption is that the input consists of avro files having different schemas that can be unioned, e.g., flat records.

      A simple illustrative example is attached (avro_storage_union_schema_test.tar.gz): run create_avro1.pig, followed by create_avro2.pig, followed by read_avro.pig.

      Attachments

        1. avro_storage_union_schema.patch
          50 kB
          Stan Rosenberg
        2. avro_storage_union_schema_test.tar.gz
          0.6 kB
          Stan Rosenberg
        3. PIG-2579-2.patch
          51 kB
          Cheolsoo Park
        4. PIG-2579-2-avro_test_files.tar.gz
          1 kB
          Cheolsoo Park
        5. PIG-2579-3.patch
          51 kB
          Cheolsoo Park
        6. PIG-2579-4.patch
          58 kB
          Cheolsoo Park
        7. PIG-2579-5.patch
          58 kB
          Cheolsoo Park
        8. PIG-2579-6.patch
          58 kB
          Cheolsoo Park

        Activity

          People

            cheolsoo Cheolsoo Park
            srosenberg Stan Rosenberg
            Votes:
            3 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: