Hive
  1. Hive
  2. HIVE-3408

A race condition is caused within QueryPlan class

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.7.1
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
    • Environment:
      • Java 1.6.0_21, Java HotSpot(TM) 64-Bit Server VM
      • Ubuntu 10.04

      Description

      Hive's threads are stopped at HashMap.getEntry(..) that is used within QueryPlan#extractCounters() and QueryPlan#updateCountersInQueryPlan(). It seems that a race condition problem of a HashSet object is caused when extractCounters() and updateCountersInQueryPlan() are concurrently executed. I hit the problem with Hive 0.7.1 but, I think that it also is caused with 0.8.1.

      The problem is reported by several persons on mailing list.
      http://mail-archives.apache.org/mod_mbox/hive-dev/201201.mbox/%3CCAKTRiE+3x31FDy+3F0c+jZSXQrhxBgT4DOyfZddm7sdX+cu=Zg@mail.gmail.com%3E

      http://mail-archives.apache.org/mod_mbox/hive-user/201202.mbox/%3CFC28CCD9-9C75-4F8D-B272-3D50F663A634@gmail.com%3E

      The following is a part of my thread dump.

      "Thread-1091" prio=10 tid=0x00007fd17112b000 nid=0x1100 runnable [0x00007fd175f60000]
      java.lang.Thread.State: RUNNABLE
      at java.util.HashMap.getEntry(HashMap.java:347)
      at java.util.HashMap.containsKey(HashMap.java:335)
      at java.util.HashSet.contains(HashSet.java:184)
      at org.apache.hadoop.hive.ql.QueryPlan.extractCounters(QueryPlan.java:342)
      at org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan(QueryPlan.java:419)
      at org.apache.hadoop.hive.ql.QueryPlan.toString(QueryPlan.java:592)
      at org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress(HiveHistory.java:493)
      at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:395)
      at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:686)
      at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123)
      at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
      at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
      at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)

      "Thread-1090" prio=10 tid=0x00007fd17012f000 nid=0x10ff runnable [0x00007fd175152000]
      java.lang.Thread.State: RUNNABLE
      at java.util.HashMap.getEntry(HashMap.java:347)
      at java.util.HashMap.containsKey(HashMap.java:335)
      at java.util.HashSet.contains(HashSet.java:184)
      at org.apache.hadoop.hive.ql.QueryPlan.updateCountersInQueryPlan(QueryPlan.java:297)
      at org.apache.hadoop.hive.ql.QueryPlan.getQueryPlan(QueryPlan.java:420)
      at org.apache.hadoop.hive.ql.QueryPlan.toString(QueryPlan.java:592)
      at org.apache.hadoop.hive.ql.history.HiveHistory.logPlanProgress(HiveHistory.java:493)
      at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:395)
      at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:686)
      at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123)
      at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
      at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
      at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)

        Issue Links

          Activity

          Hide
          Muga Nishizawa added a comment -

          To solve the problem, I attached a patch on this ticket. Please check it.

          Show
          Muga Nishizawa added a comment - To solve the problem, I attached a patch on this ticket. Please check it.
          Hide
          Kazuki Ohta added a comment -

          Found that other HashSet within a same class could cause the race condition.

          Show
          Kazuki Ohta added a comment - Found that other HashSet within a same class could cause the race condition.

            People

            • Assignee:
              Unassigned
              Reporter:
              Muga Nishizawa
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development