Hadoop Common
  1. Hadoop Common
  2. HADOOP-9115

Deadlock in configuration when writing configuration to hdfs

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.1.1
    • Fix Version/s: 1.1.2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      This fixes a bug where Hive could trigger a deadlock condition in the Hadoop configuration management code.

      Description

      This was noticed when using hive with hadoop-1.1.1 and running

      select count(*) from tbl;
      

      This would cause a deadlock configuration.

      1. hive-jstack.log
        8 kB
        Arpit Gupta
      2. HADOOP-7082.b1.patch
        6 kB
        Jing Zhao
      3. HADOOP-7082.b1.002.patch
        6 kB
        Jing Zhao

        Issue Links

          Activity

          Arpit Gupta created issue -
          Hide
          Arpit Gupta added a comment -

          Thread 1640 and 1577 show the deadlock.

          Show
          Arpit Gupta added a comment - Thread 1640 and 1577 show the deadlock.
          Arpit Gupta made changes -
          Field Original Value New Value
          Attachment hive-jstack.log [ 12555961 ]
          Arpit Gupta made changes -
          Description This was noticed when using hive with hadoop-1.1.1 and running select count(*) from tbl;

          This would cause a deadlock configuration.
          This was noticed when using hive with hadoop-1.1.1 and running

          {code}
          select count(*) from tbl;
          {code}

          This would cause a deadlock configuration.
          Jing Zhao made changes -
          Assignee Jing Zhao [ jingzhao ]
          Arpit Gupta made changes -
          Affects Version/s 1.1.1 [ 12321658 ]
          Hide
          Jing Zhao added a comment -

          The deadlock seems to be the same issue with HDFS-1542 and HADOOP-7082. Thus we need to backport these two jiras to branch-1. Upload the backport patch first.

          Show
          Jing Zhao added a comment - The deadlock seems to be the same issue with HDFS-1542 and HADOOP-7082 . Thus we need to backport these two jiras to branch-1. Upload the backport patch first.
          Jing Zhao made changes -
          Attachment HADOOP-7082.b1.patch [ 12555967 ]
          Hide
          Jing Zhao added a comment -

          The backport patch passed all the unit tests and the test-patch result is:

          -1 overall.  
              +1 @author.  The patch does not contain any @author tags.
              +1 tests included.  The patch appears to include 3 new or modified tests.
              +1 javadoc.  The javadoc tool did not generate any warning messages.
              +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
              -1 findbugs.  The patch appears to introduce 225 new Findbugs (version 2.0.1) warnings.
          
          Show
          Jing Zhao added a comment - The backport patch passed all the unit tests and the test-patch result is: -1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 225 new Findbugs (version 2.0.1) warnings.
          Hide
          Arpit Gupta added a comment -

          Built the jar using the patch attached and the hive query ran through without any issues.

          Show
          Arpit Gupta added a comment - Built the jar using the patch attached and the hive query ran through without any issues.
          Jing Zhao made changes -
          Link This issue is related to HDFS-4161 [ HDFS-4161 ]
          Arpit Gupta made changes -
          Target Version/s 1.2.0, 1.1.2 [ 12321659, 12323596 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Comparing the patch with trunk:

          • The line "Properties properties = getProps();" is in asXmlDocument() for the patch but it is in writeXml(Writer out) in trunk. Is it intended?
          • In trunk, writeXml(Writer out) is declared before asXmlDocument() but the patch put them the other way. Could you move the code to make it the same as trunk?
          Show
          Tsz Wo Nicholas Sze added a comment - Comparing the patch with trunk: The line "Properties properties = getProps();" is in asXmlDocument() for the patch but it is in writeXml(Writer out) in trunk. Is it intended? In trunk, writeXml(Writer out) is declared before asXmlDocument() but the patch put them the other way. Could you move the code to make it the same as trunk?
          Hide
          Jing Zhao added a comment -

          Thanks for the comments Nicholas. The line "Properties properties = getProps();" is moved there to make sure properties is not null (otherwise if we directly call writeXml after creating a configuration the properties can be null). The new patch addressed the function sequence comment.

          Show
          Jing Zhao added a comment - Thanks for the comments Nicholas. The line "Properties properties = getProps();" is moved there to make sure properties is not null (otherwise if we directly call writeXml after creating a configuration the properties can be null). The new patch addressed the function sequence comment.
          Jing Zhao made changes -
          Attachment HADOOP-7082.b1.002.patch [ 12555992 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Jing!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Jing!
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 1.1.2 [ 12323596 ]
          Resolution Fixed [ 1 ]
          Matt Foley made changes -
          Target Version/s 1.2.0, 1.1.2 [ 12321659, 12323596 ] 1.1.2 [ 12323596 ]
          Matt Foley made changes -
          Release Note This fixes a bug where Hive could trigger a deadlock condition in the Hadoop configuration management code.
          Hide
          Matt Foley added a comment -

          Closed upon successful release of 1.1.2.

          Show
          Matt Foley added a comment - Closed upon successful release of 1.1.2.
          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Jing Zhao
              Reporter:
              Arpit Gupta
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development