Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3310

Lease recovery for append

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.18.0
    • None
    • None
    • Incompatible change, Reviewed
    • Implemented Lease Recovery to sync the last bock of a file. Added ClientDatanodeProtocol for client trigging block recovery. Changed DatanodeProtocol to support block synchronization. Changed InterDatanodeProtocol to support block update.

    Description

      In order to support file append, a GenerationStamp is associated with each block. Lease recovery will be performed when there is a possibility that the replicas of a block in a lease may have different GenerationStamp values.

      For more details, see the documentation in HADOOP-1700.

      Attachments

        1. 3310_20080514.patch
          38 kB
          Tsz-wo Sze
        2. 3310_20080516b.patch
          49 kB
          Tsz-wo Sze
        3. 3310_20080516c.patch
          47 kB
          Tsz-wo Sze
        4. 3310_20080519.patch
          51 kB
          Tsz-wo Sze
        5. 3310_20080519b.patch
          62 kB
          Tsz-wo Sze
        6. 3310_20080520.patch
          76 kB
          Tsz-wo Sze
        7. 3310_20080521.patch
          78 kB
          Tsz-wo Sze
        8. 3310_20080522b.patch
          81 kB
          Tsz-wo Sze
        9. 3310_20080522c.patch
          82 kB
          Tsz-wo Sze
        10. 3310_20080523.patch
          85 kB
          Tsz-wo Sze
        11. 3310_20080524_dhruba.patch
          88 kB
          Dhruba Borthakur
        12. 3310_20080527.patch
          100 kB
          Tsz-wo Sze
        13. 3310_20080528.patch
          102 kB
          Tsz-wo Sze
        14. 3310_20080528b.patch
          101 kB
          Tsz-wo Sze
        15. 3310_20080528c.patch
          104 kB
          Tsz-wo Sze
        16. 3310_20080529.patch
          96 kB
          Dhruba Borthakur
        17. 3310_20080529b.patch
          105 kB
          Dhruba Borthakur
        18. 3310_20080529c.patch
          108 kB
          Tsz-wo Sze

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3113 DFSOututStream.flush() should flush data to real block file on DataNode.

          • Major - Major loss of function.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3375 Lease paths are sometimes not removed from LeaseManager.sortedLeasesByPath

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-1700 Append to files in HDFS

          • Major - Major loss of function.
          • Closed

          Activity

            People

              szetszwo Tsz-wo Sze
              szetszwo Tsz-wo Sze
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: