Hive
  1. Hive
  2. HIVE-140

Event Based Infrastructure for Syntax Trees in the compiler

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.3.0
    • Component/s: Query Processor
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In order to refactor SemanticAnalyzer into a bunch of different components, we need to move more modular mechanisms of doing semantic analysis. One of the missing pieces here is the event based infrastructure for analyzing ASTs in the Semantic Analyzer. We will first use this infrastructure for lineage tools and then migrate the rest of the Semantic Analyzer to this infrastructure over time.

      1. patch-140.txt
        16 kB
        Ashish Thusoo

        Activity

        Hide
        Ashish Thusoo added a comment -

        This is a better framework for parse tree analysis and we should migrate to this gradually. I had to cook this up for Suresh who is doing lineage extraction from the parse tree for databee.

        The following are the basic framework elements:

        It has three interfaces:
        1. ParseTreeProcessor (with a default implementation of DefaultParseTreeProcessor that does a pre order traversal)
        2. ParseTreeEventDispatcher (with a default implementation of DefaultParseTreeEventDispatcher) which dispatches events to a ParseTreeEventProcessor
        3. ParseTreeEventProcessor (with a default implementation of DefaultParseTreeEventProcessor).

        The ParseTreeEvent class defines the different kind of parse tree events. These are basically things like saw a select clause, source table, subquery, limit etc... and we can keep adding to these.

        The dispatcher code registers for the interested events the event processor that needs to be called. The default dispatcher calls these processors for the event in the order that they were registered.

        The process function in the tree processor calls the dispatcher for these events and the default one does that in a pre order way.

        I think a framework like this will help us modularize SemanticAnalyzer in a much better way.

        Show
        Ashish Thusoo added a comment - This is a better framework for parse tree analysis and we should migrate to this gradually. I had to cook this up for Suresh who is doing lineage extraction from the parse tree for databee. The following are the basic framework elements: It has three interfaces: 1. ParseTreeProcessor (with a default implementation of DefaultParseTreeProcessor that does a pre order traversal) 2. ParseTreeEventDispatcher (with a default implementation of DefaultParseTreeEventDispatcher) which dispatches events to a ParseTreeEventProcessor 3. ParseTreeEventProcessor (with a default implementation of DefaultParseTreeEventProcessor). The ParseTreeEvent class defines the different kind of parse tree events. These are basically things like saw a select clause, source table, subquery, limit etc... and we can keep adding to these. The dispatcher code registers for the interested events the event processor that needs to be called. The default dispatcher calls these processors for the event in the order that they were registered. The process function in the tree processor calls the dispatcher for these events and the default one does that in a pre order way. I think a framework like this will help us modularize SemanticAnalyzer in a much better way.
        Hide
        Ashish Thusoo added a comment -

        submitting.

        Show
        Ashish Thusoo added a comment - submitting.
        Hide
        Zheng Shao added a comment -

        +1

        Show
        Zheng Shao added a comment - +1
        Hide
        Zheng Shao added a comment -

        Committed revision 724613.

        Show
        Zheng Shao added a comment - Committed revision 724613.
        Hide
        Zheng Shao added a comment -

        HIVE-140. Event Based Infrastructure for Syntax Trees in the compiler.
        (Ashish Thusoo through zshao)

        Show
        Zheng Shao added a comment - HIVE-140 . Event Based Infrastructure for Syntax Trees in the compiler. (Ashish Thusoo through zshao)

          People

          • Assignee:
            Ashish Thusoo
            Reporter:
            Ashish Thusoo
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development