Hive
  1. Hive
  2. HIVE-140

Event Based Infrastructure for Syntax Trees in the compiler

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.3.0
    • Component/s: Query Processor
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In order to refactor SemanticAnalyzer into a bunch of different components, we need to move more modular mechanisms of doing semantic analysis. One of the missing pieces here is the event based infrastructure for analyzing ASTs in the Semantic Analyzer. We will first use this infrastructure for lineage tools and then migrate the rest of the Semantic Analyzer to this infrastructure over time.

      1. patch-140.txt
        16 kB
        Ashish Thusoo

        Activity

        Ashish Thusoo created issue -
        Hide
        Ashish Thusoo added a comment -

        This is a better framework for parse tree analysis and we should migrate to this gradually. I had to cook this up for Suresh who is doing lineage extraction from the parse tree for databee.

        The following are the basic framework elements:

        It has three interfaces:
        1. ParseTreeProcessor (with a default implementation of DefaultParseTreeProcessor that does a pre order traversal)
        2. ParseTreeEventDispatcher (with a default implementation of DefaultParseTreeEventDispatcher) which dispatches events to a ParseTreeEventProcessor
        3. ParseTreeEventProcessor (with a default implementation of DefaultParseTreeEventProcessor).

        The ParseTreeEvent class defines the different kind of parse tree events. These are basically things like saw a select clause, source table, subquery, limit etc... and we can keep adding to these.

        The dispatcher code registers for the interested events the event processor that needs to be called. The default dispatcher calls these processors for the event in the order that they were registered.

        The process function in the tree processor calls the dispatcher for these events and the default one does that in a pre order way.

        I think a framework like this will help us modularize SemanticAnalyzer in a much better way.

        Show
        Ashish Thusoo added a comment - This is a better framework for parse tree analysis and we should migrate to this gradually. I had to cook this up for Suresh who is doing lineage extraction from the parse tree for databee. The following are the basic framework elements: It has three interfaces: 1. ParseTreeProcessor (with a default implementation of DefaultParseTreeProcessor that does a pre order traversal) 2. ParseTreeEventDispatcher (with a default implementation of DefaultParseTreeEventDispatcher) which dispatches events to a ParseTreeEventProcessor 3. ParseTreeEventProcessor (with a default implementation of DefaultParseTreeEventProcessor). The ParseTreeEvent class defines the different kind of parse tree events. These are basically things like saw a select clause, source table, subquery, limit etc... and we can keep adding to these. The dispatcher code registers for the interested events the event processor that needs to be called. The default dispatcher calls these processors for the event in the order that they were registered. The process function in the tree processor calls the dispatcher for these events and the default one does that in a pre order way. I think a framework like this will help us modularize SemanticAnalyzer in a much better way.
        Ashish Thusoo made changes -
        Field Original Value New Value
        Attachment patch-140.txt [ 12395618 ]
        Hide
        Ashish Thusoo added a comment -

        submitting.

        Show
        Ashish Thusoo added a comment - submitting.
        Ashish Thusoo made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Zheng Shao added a comment -

        +1

        Show
        Zheng Shao added a comment - +1
        Hide
        Zheng Shao added a comment -

        Committed revision 724613.

        Show
        Zheng Shao added a comment - Committed revision 724613.
        Hide
        Zheng Shao added a comment -

        HIVE-140. Event Based Infrastructure for Syntax Trees in the compiler.
        (Ashish Thusoo through zshao)

        Show
        Zheng Shao added a comment - HIVE-140 . Event Based Infrastructure for Syntax Trees in the compiler. (Ashish Thusoo through zshao)
        Zheng Shao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Fix Version/s 0.19.0 [ 12313513 ]
        Hadoop Flags [Reviewed]
        Zheng Shao made changes -
        Fix Version/s 0.6.0 [ 12314524 ]
        Fix Version/s 0.2.0 [ 12313565 ]
        Carl Steinbach made changes -
        Fix Version/s 0.3.0 [ 12313637 ]
        Fix Version/s 0.6.0 [ 12314524 ]
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        24m 58s 1 Ashish Thusoo 09/Dec/08 03:36
        Patch Available Patch Available Resolved Resolved
        2h 10m 1 Zheng Shao 09/Dec/08 05:47
        Resolved Resolved Closed Closed
        1102d 18h 21m 1 Carl Steinbach 17/Dec/11 00:09

          People

          • Assignee:
            Ashish Thusoo
            Reporter:
            Ashish Thusoo
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development