Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24206

DefaultGraphWalker IdentityHashMap causes thread idling

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.3.4
    • Fix Version/s: None
    • Component/s: Parser
    • Labels:
      None

      Description

      the sql is very long,about 3M size,use hive parse to parse the sql, it costs more than one hour to finish it. The reason is hive parse use IdentityHashMap, is not multi-thread safe, than it costs more than one hour  to execute the code "while(true)"

       

      my code is :

      ParseDriver pd = new ParseDriver();
      ASTNode tree = pd.parse(query, context);

      while ((tree.getToken() == null) && (tree.getChildCount() > 0))

      { tree = (ASTNode) tree.getChild(0); }

      Map<Rule, NodeProcessor> rules = new LinkedHashMap<>();
      Dispatcher disp = new DefaultRuleDispatcher(this, rules, null);
      GraphWalker ogw = new DefaultGraphWalker(disp);
      List<Node> topNodes = new ArrayList<>();
      topNodes.add(tree);
      ogw.startWalking(topNodes, null);

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Xiaojie Zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Due:
                Created:
                Updated: