Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5080

Support store alias as spark table

    Details

    • Type: New Feature
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.16.0
    • Fix Version/s: 0.17.1
    • Component/s: spark
    • Labels:
      None

      Description

      The purpose is that I'd like to take advantage of both pig and hive. Pig-latin has powerful data flow expression ability which is useful for ETL while hive is good at query.

      The scenario is that I'd like to store pig alias as spark temporary table (cache can be optional). And I have an another spark engine which share the same SparkContext (in the same JVM) to query the table.

      Please close this ticket if it is already supported. I didn't go through all the features of pig-spark.

        Attachments

        1. PIG-5080-2.patch
          14 kB
          Jeff Zhang
        2. PIG-5080-1.patch
          15 kB
          Jeff Zhang

          Issue Links

            Activity

              People

              • Assignee:
                zjffdu Jeff Zhang
                Reporter:
                zjffdu Jeff Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: