Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1542

Deadlock in Configuration.writeXml when serialized form is larger than one DFS block

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.22.0, 0.23.0
    • Fix Version/s: 0.22.0
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Configuration.writeXml holds a lock on itself and then writes the XML to an output stream, during which DFSOutputStream will try to get a lock on ackQueue/dataQueue. Meanwihle the DataStreamer thread will call functions like conf.getInt() and deadlock against the other thread, since it could be the same conf object.

      This causes a deterministic deadlock whenever the serialized form is larger than block size.

      1. Test.java
        0.5 kB
        Todd Lipcon
      2. deadlock.txt
        82 kB
        Amit Nithian
      3. hdfs-1542.txt
        6 kB
        Todd Lipcon
      4. hdfs1542_cdh3b3.txt
        1 kB
        Amit Nithian
      5. hdfs-1542.txt
        6 kB
        Todd Lipcon
      6. hdfs-1542.txt
        6 kB
        Todd Lipcon
      7. hdfs-1542.txt
        2 kB
        Todd Lipcon

        Issue Links

          Activity

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development