Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1380

The append pipeline does not followed TSP principal

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Incomplete
    • Affects Version/s: 0.20-append
    • Fix Version/s: None
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      1. Say we have 2 racks: rack-0 and rack-1.
      Rack-0 has dn1, dn2, dn3. Rack-0 has dn4, dn5, dn6.

      2. Suppose client is in rack-0, and the write pipeline is:
      client --> localnode --> other rack --> other rack
      In this example we have the pipeline client-dn1-dn4-dn6.
      That is rack0-rack0-rack1-rack1. So far so good.

      3. Now other client comes, and append to file.
      This client is also in rack-0. Interestingly,
      the append pipeline is client-dn6-dn4-dn1.
      That is the new client (from rack0) sends packet
      to the first node in pipeline (dn6) which belongs to rack1.

      This bug was found by our Failure Testing Service framework:
      http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html
      For questions, please email us: Thanh Do (thanhdo@cs.wisc.edu) and
      Haryadi Gunawi (haryadi@eecs.berkeley.edu)

        Activity

        Allen Wittenauer made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Incomplete [ 4 ]
        Thanh Do made changes -
        Field Original Value New Value
        Description 1. Say we have 2 racks: rack-0 and rack-1.
        Rack-0 has dn1, dn2, dn3. Rack-0 has dn4, dn5, dn6.
         
        2. Suppose client is in rack-0, and the write pipeline is:
        client --> localnode --> other rack --> other rack
        In this example we have the pipeline client-dn1-dn4-dn6.
        That is rack0-rack0-rack1-rack1. So far so good.
         
        3. Now other client comes, and append to file.
        This client is also in rack-0. Interestingly,
        the append pipeline is client-dn6-dn4-dn1.
        That is the new client (from rack0) sends packet
        to the first node in pipeline (dn6) which belongs to rack1.
        1. Say we have 2 racks: rack-0 and rack-1.
        Rack-0 has dn1, dn2, dn3. Rack-0 has dn4, dn5, dn6.
         
        2. Suppose client is in rack-0, and the write pipeline is:
        client --> localnode --> other rack --> other rack
        In this example we have the pipeline client-dn1-dn4-dn6.
        That is rack0-rack0-rack1-rack1. So far so good.
         
        3. Now other client comes, and append to file.
        This client is also in rack-0. Interestingly,
        the append pipeline is client-dn6-dn4-dn1.
        That is the new client (from rack0) sends packet
        to the first node in pipeline (dn6) which belongs to rack1.

        This bug was found by our Failure Testing Service framework:
        http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-98.html
        For questions, please email us: Thanh Do (thanhdo@cs.wisc.edu) and
        Haryadi Gunawi (haryadi@eecs.berkeley.edu)
        Thanh Do created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Thanh Do
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development