Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1542

Deadlock in Configuration.writeXml when serialized form is larger than one DFS block

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.22.0, 0.23.0
    • Fix Version/s: 0.22.0
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Configuration.writeXml holds a lock on itself and then writes the XML to an output stream, during which DFSOutputStream will try to get a lock on ackQueue/dataQueue. Meanwihle the DataStreamer thread will call functions like conf.getInt() and deadlock against the other thread, since it could be the same conf object.

      This causes a deterministic deadlock whenever the serialized form is larger than block size.

        Attachments

        1. deadlock.txt
          82 kB
          Amit Nithian
        2. hdfs1542_cdh3b3.txt
          1 kB
          Amit Nithian
        3. hdfs-1542.txt
          2 kB
          Todd Lipcon
        4. hdfs-1542.txt
          6 kB
          Todd Lipcon
        5. hdfs-1542.txt
          6 kB
          Todd Lipcon
        6. hdfs-1542.txt
          6 kB
          Todd Lipcon
        7. Test.java
          0.5 kB
          Todd Lipcon

          Issue Links

            Activity

              People

              • Assignee:
                tlipcon Todd Lipcon
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                1 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: