Hadoop Common
  1. Hadoop Common
  2. HADOOP-5554

DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.19.2
    • Fix Version/s: 0.19.2, 0.20.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-5384 makes DataNodeCluster to create blocks with generation stamp Block#GRANDFATHER_GENERATION_STAMP(0) so simuated datanodes do not crash NameNode any more. But there is still a problem. CreateEditLogs creates blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP (1000). Because of the generation stamp mismatch, all injected blocks are marked as invalid when NameNode processes block reports.

      1. blockGS.patch
        2 kB
        Hairong Kuang
      2. blockGS-0.19.patch
        2 kB
        Hairong Kuang

        Activity

        Hide
        Hairong Kuang added a comment -

        This patch makes both CreateEditsLog and DataNodeCluster to create blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP.

        Show
        Hairong Kuang added a comment - This patch makes both CreateEditsLog and DataNodeCluster to create blocks with generation stamp GenerationStamp#FIRST_VALID_STAMP.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1

        Show
        Tsz Wo Nicholas Sze added a comment - +1
        Hide
        Tsz Wo Nicholas Sze added a comment -

        This patch changes tools for manual tests. I think we don't have junit for these test tools. So, I suggest to do some manual tests instead of waiting for Hudson.

        Show
        Tsz Wo Nicholas Sze added a comment - This patch changes tools for manual tests. I think we don't have junit for these test tools. So, I suggest to do some manual tests instead of waiting for Hudson.
        Hide
        Hairong Kuang added a comment -

        I have tested the patch manually.

        Show
        Hairong Kuang added a comment - I have tested the patch manually.
        Hide
        Hairong Kuang added a comment -

        This is a patch for 0.19.

        Show
        Hairong Kuang added a comment - This is a patch for 0.19.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this to 0.19 and above. Thanks, Hairong!

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this to 0.19 and above. Thanks, Hairong!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk #789 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/789/)
        . DataNodeCluster and CreateEditsLog should create blocks with the same generation stamp value. (hairong via szetszwo)

        Show
        Hudson added a comment - Integrated in Hadoop-trunk #789 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/789/ ) . DataNodeCluster and CreateEditsLog should create blocks with the same generation stamp value. (hairong via szetszwo)

          People

          • Assignee:
            Hairong Kuang
            Reporter:
            Hairong Kuang
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development