Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-824

SQL interface for Pig

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • None
    • None

    Description

      In the last 18 month PigLatin has gained significant popularity within the open source community. Many users like its data flow model, its rich type system and its ability to work with any data available on HDFS or outside. We have also heard from many users that having Pig speak SQL would bring many more users. Having a single system that exports multiple interfaces is a big advantage as it guarantees consistent semantics, custom code reuse, and reduces the amount of maintenance. This is especially relevant for project where using both interfaces for different parts of the system is relevant. For instance, in a
      data warehousing system, you would have ETL component that brings data into the warehouse and a component that analyzes the data and produces reports. PigLatin is uniquely suited for ETL processing while SQL might be a better fit for report generation.

      To start, it would make sense to implement a subset of SQL92 standard and to be as much as possible standard compliant. This would include all the standard constructs: select, from, where, group-by + having, order by, limit, join (inner + outer). Several extensions such as support for pig's UDFs and possibly streaming, multiquery and support for pig's complex types would be helpful.

      This work is dependent on metadata support outlined in https://issues.apache.org/jira/browse/PIG-823

      Attachments

        1. PIG-824.binfiles.tar.gz
          2.30 MB
          Thejas Nair
        2. PIG-824.1.patch
          664 kB
          Thejas Nair
        3. SQL_IN_PIG.html
          3 kB
          Thejas Nair
        4. pigsql.patch
          1.42 MB
          Thejas Nair
        5. pig_sql_beta.pdf
          86 kB
          Thejas Nair
        6. java-cup-11a-runtime.jar
          13 kB
          Thejas Nair
        7. java-cup-11a.jar
          94 kB
          Thejas Nair
        8. students2.bin
          0.2 kB
          Thejas Nair
        9. students_attr.bin
          0.6 kB
          Thejas Nair
        10. pigsql_tutorial.txt
          5 kB
          Thejas Nair

        Issue Links

          Activity

            People

              thejas Thejas Nair
              olgan Olga Natkovich
              Votes:
              0 Vote for this issue
              Watchers:
              30 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: